Touch interaction module and algorithm

Abstract

The invention relates to the technical field of data processing, and particularly discloses a touch interaction module and algorithm, the module comprises a substrate layer, a touch function layer and a protection layer; the base material layer is used for bearing the upper touch function layer and the protective layer; the touch function layer is arranged on the surface of one side of the base material layer; the protective layer is arranged on the surface of the other side of the touch function layer; the algorithm comprises the following steps: collecting an original touch signal output by a touch function layer; performing filtering and denoising processing on the original touch signal; analyzing the processed touch signal, and dynamically adjusting a lock point threshold value; and the use habit of the user is learned and adapted in real time. According to the method, the convolutional neural network is used for deeply analyzing the touch signal and dynamically adjusting the lock point threshold value, so that the accuracy of touch recognition is improved; in addition, false touch points are eliminated through space matching of the hand position, the false touch rate is reduced, and the anti-interference performance of touch control is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and more specifically, to a touch interaction module and algorithm. Background Art

[0002] Touch interaction technology has become an important means of human-computer interaction with its intuitive and efficient user experience.

[0003] Existing public literature 1 (Design of a Desktop Touch Projection System Based on Depth Infrared Information Fusion, 2024) discloses a touch algorithm. This algorithm first uses a background subtraction algorithm based on depth images for gesture segmentation, combines improved Canny edge detection and contour area analysis to extract the hand contour, effectively adapting to different lighting environments and improving the anti-noise ability. Then, according to the touch type, the centroid detection (single finger) or convex hull detection (multiple fingers) algorithm is used to accurately extract the contact point position. However, this algorithm is prone to false touch problems during multi-finger operations.

[0004] Existing public literature 2 (Design of a High-Performance Multi-Channel Touch System for Large-Size Capacitive Screens, 2024) discloses a high-performance multi-channel touch system for large-size capacitive screens. Through hardware innovations such as double-edge integration and differential output, this system supports parallel processing of 112×72 channels, solving the problems of low scanning frame rate and slow response speed caused by a large number of channels. At the same time, this system combines a touch prediction scanning algorithm based on long short-term memory network (LSTM) to reduce the invalid scanning area and further improve the scanning frame rate. However, in the multi-user close collaboration operation multi-touch scenario, this system cannot determine which user the contact point belongs to, increasing the difficulty of signal processing and resulting in a lack of precision in multi-touch.

[0005] Therefore, there is an urgent need for a touch interaction module and algorithm that can reduce the false touch rate in the scenario of multi-person collaborative touch. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art, the present invention provides a touch interaction module and algorithm, which removes low-frequency interference through a high-pass filter, dynamically adjusts the lock point threshold, improves the accuracy of touch recognition, and eliminates false touch points through spatial matching of the hand position, reducing the false touch rate generated by multi-person collaborative operation, and effectively improving the anti-interference performance of touch. To solve the problems raised in the above background art.

[0007] To achieve the above object, the present invention provides the following technical solutions: A touch interaction algorithm, characterized by comprising the following steps: Step Z1, collecting the original touch signal output by the touch function layer; Step Z2, performing filtering and denoising processing on the original touch signal; Step Z3: Analyze the processed touch signal and dynamically adjust the lock point threshold; Step Z4: Continuously learn and adapt to the user's usage habits in real time; The touch algorithm also introduces a method that combines multi-touch and hand tracking to reduce the signal interference of multi-touch during multi-person operation. The specific steps are as follows: Step S1: Detect the infrared light leakage generated by finger touch in the rear projection desktop constructed by the acrylic light guide plate, extract the touch point data by combining image difference and binarization, and assign a unique ID. Introduce a delayed reporting mechanism to improve the recognition accuracy and anti-interference ability; Step S2: Determine the relative position relationship between the finger and the arm in the picture by analyzing the Y-axis distribution characteristics of the hand contour in the image, and infer the spatial position and direction of the hand accordingly; Step S3: Match each hand position with the touch point set space, logically group the simultaneously occurring touch points, and eliminate the "spatially isolated" touch points to reduce misjudgment.

[0008] As a further solution of the present invention, in the step Z1, the sensor array in the touch function layer continuously obtains the touch operation actions of the user, converts the touch operation actions into electrical signals, thereby forming initial raw touch signal data; the touch signal data is stored in the form of a voltage signal matrix, where each element in the matrix corresponds to a specific sensing point in the touch sensor array, and records the voltage or capacitance change corresponding to this position during the touch operation.

[0009] As a further solution of the present invention, in the step Z2, since there may be many environmental interferences and device noises during the user's touch process, the raw touch signal data carries interference signals and noises. If it is directly used for recognition without being processed, the accuracy of touch recognition will be reduced; therefore, the collected raw touch signal data is filtered and denoised. Specifically, a high-pass filter is used to preprocess the touch signal data to remove low-frequency interference signals or DC offsets. The cut-off frequency of the high-pass filter is between 200 Hz and 400 Hz.

[0010] As a further solution of the present invention, in the step Z3, the filtered touch signal data is input into a convolutional neural network for in-depth analysis. The structure of the convolutional neural network consists of an input layer, three convolutional layers, two pooling layers, a fully connected layer, and a Softmax output layer. The input layer receives a touch signal matrix in the form of M×N, where M and N respectively represent the number of electrodes of the touch sensor array in the X-axis and Y-axis directions. In the first convolution, 32 3×3 convolutional kernels are used to extract local spatial features, and the ReLU activation function is connected after convolution to enhance the non-linear expression ability; immediately following is the first 2×2 max pooling layer, which is used to reduce the spatial resolution of the feature map while retaining significant features. The second convolutional layer increases to 64 3×3 convolutional kernels, also using ReLU as the activation function, and then the second 2×2 max pooling layer is connected for further downsampling. The third convolutional layer uses 128 3×3 convolutional kernels to perform high-order abstraction on the previously extracted features. After the convolution operation, the ReLU activation function is connected again, and no pooling layer is added to retain more high-level information. Subsequently, all the convolved feature maps are converted into a one-dimensional vector through a flattening operation and sent to a fully connected layer containing 256 neurons for global information fusion and non-linear mapping. Finally, a Softmax output layer is connected to perform multi-classification prediction on the touch action categories.

[0011] The convolutional neural network is trained using a labeled touch dataset based on supervised learning. The dataset consists of touch operation signal matrices and their corresponding action category labels, and each sample is a two-dimensional matrix with a size of 32x32. Before training, the original data is first normalized, and the signal intensity is normalized to the range of [0,1]. The dataset is divided into a training set, a validation set, and a test set, and the ratio of the training set, the validation set, and the test set is 7:2:1. The cross-entropy loss function is used as the objective function in the training phase; the Adam optimizer is selected, and the initial learning rate is set to 0.001; during the training process, each batch contains multiple touch samples. The batch data is sequentially sent into the neural network for forward propagation to generate prediction results. Subsequently, the loss is calculated by comparing with the true labels, and the gradient of the convolutional kernel weights and the fully connected layer parameters is updated using the backpropagation algorithm.

[0012] The touch interaction algorithm involves a dynamic adjustment mechanism for the lock point threshold, specifically using the exponential weighted average method for dynamic adjustment. The real-time prediction probability output by the convolutional neural network is weighted and averaged with the threshold at the previous moment to ensure the smoothness and fast response ability of the threshold adjustment. The weighting coefficient takes values between 0.5 and 0.7.

[0013] As a further aspect of the present invention, in step Z4, the touch interaction algorithm introduces a user behavior learning module. The user behavior learning module is based on a long short-term memory network, using the historical user touch behavior sequence as the training input to capture the changing trends of the user's long-term or short-term operation habits, and accordingly, the convolutional neural network model and the lock point threshold parameters are adjusted and optimized in a personalized manner. By this method, the touch interaction algorithm can adapt to the personalized differences shown by the user in daily touch operations, such as finger size, touch force, frequency, and speed. After learning the user's habits and optimizing its own model parameters, it can actively predict the user's touch action tendency and optimize the signal recognition result accordingly, thereby improving the personalized touch response accuracy and user experience satisfaction.

[0014] As a further aspect of the present invention, in step S1, a rear projection interactive desktop based on the FTIR principle is used as the touch platform, which is composed of a 1.2-meter × 0.9-meter acrylic diffusion plate laminated with an infrared light guide plate and installed at waist height. Infrared light is injected along the edge of the light guide plate and propagates in the plate through total internal reflection. When a finger touches the surface, the total internal reflection is broken, forming a bright spot that can be captured by the infrared camera below. The system extracts the bright spot area through image difference and threshold binaryzation, determines the touch point coordinates using contour analysis, and records the timestamp. To improve stability, a delay reporting mechanism is introduced, and only the contacts that stably exist in consecutive frames are reported as valid events. Each touch point is assigned a unique ID for subsequent trajectory tracking until the finger leaves the light guide plate, and the ID is automatically released and reset.

[0015] As a further aspect of the present invention, in step S2, a skin-color-based RGB threshold segmentation method is used for hand detection and tracking. After the upper visible light camera captures an image, the skin color range of each pixel is judged to generate a binary image, and then the hand area is identified through contour extraction. The system filters out small-area noises and only retains the main contours that meet the shape and area characteristics, and extracts the key points of the "arm end" and "finger end" according to the Y-axis coordinate to determine the user's standing direction, thereby establishing the attribution relationship between the hand and the touch point. To adapt to different lighting and projection contents, the system has the ability to dynamically adjust the skin color threshold in real time, continuously optimizing the recognition accuracy and stability.

[0016] As a further solution of the present invention, in step S3, based on the principle of the minimum Euclidean distance, each touch point is spatially matched with the hand position. If the distance between a touch point and a certain hand point is less than a threshold d0 (30 - 50 pixels), it is considered to belong to that user. After the matching is completed, it enters the fusion and optimization stage. According to the hand attribution relationship, the touch points that are temporally and spatially close are grouped, and it is judged whether they form a composite gesture, such as two-finger pinch, three-finger rotation, etc., through gesture library matching and convolutional or temporal algorithms to achieve dynamic recognition. At the same time, the system automatically eliminates isolated touch points that exceed the distance d0 and have no corresponding hand to avoid false touches caused by environmental interference and improve the accuracy and robustness of gesture recognition.

[0017] To improve the collaboration efficiency and interaction personalization, the system assigns a unique ID to each hand and determines the operation direction based on its spatial position at the screen boundary. By analyzing the Y-axis relationship between the "finger end" and the "table edge end", the user's orientation (such as side one, side two, or horizontal boundary) is identified. Based on the hand ID and orientation information, the system constructs a user role mapping table to achieve dynamic touch response optimization and differential allocation of function permissions in a multi-user environment.

[0018] A touch interaction module includes a substrate layer, a touch function layer, and a protective layer. The substrate layer is used to carry the upper touch function layer and the protective layer, and the material of the substrate layer is a tempered glass substrate with a thickness controlled within the range of 0.5 mm to 1.5 mm.

[0019] The touch function layer is disposed on one side surface of the substrate layer and is used to monitor the user's touch actions in real time and accurately convert the touch actions into recognizable electrical signals for output. The touch function layer includes two interleaved sensor arrays, an X electrode array and a Y electrode array. Among them, the width of each electrode array is between 3 and 10 μm, and the array pitch is set between 200 and 500 μm.

[0020] To protect the touch function layer from the impact of external forces and daily wear, a protective layer is disposed on the other side surface of the touch function layer. The protective layer includes a main body part and a hollow part. The main body part is a continuous surface structure for blocking external mechanical impacts and preventing environmental pollutants from damaging the touch function layer, thereby extending the service life of the module; the hollow part adopts a honeycomb-like hollow structure, which increases the actual contact area between the protective layer and the connection layer located below it, so that the structure of the protective layer, while maintaining the original protection function, improves the overall adhesion, structural stability, and signal transmission sensitivity of the touch module. The diameter of the hollow holes is controlled between 50 μm and 150 μm, and the hole pitch is between 100 μm and 300 μm.

[0021] Technical effects and advantages of a touch interaction module and algorithm of the present invention: The present invention proposes a comprehensive solution. In terms of hardware, the touch interaction module uses a tempered glass substrate as the base layer, combined with a silver nanowire conductive layer structure with high light transmittance and fast response, ensuring the accurate acquisition and transmission of touch signals; the protective layer uses a transparent polymer material and is designed with a honeycomb-like hollow structure, enhancing the adhesion to the connection layer and the signal transmission sensitivity. In terms of algorithm, low-frequency interference is removed through a high-pass filter, and a convolutional neural network is used to deeply analyze touch signals and dynamically adjust the lock point threshold, significantly improving the accuracy of touch recognition; the user behavior learning module adapts to the user's personalized operation habits based on a long short-term memory network, further optimizing the touch response. Particularly importantly, by combining the methods of multi-touch and hand tracking, the touch points are accurately detected and tracked using the FTIR principle and image processing technology, and false touch points are eliminated through spatial matching of the hand position, reducing the misjudgment rate by about 40% in a multi-user collaboration scenario, effectively solving the problem of touch point attribution and improving the anti-interference performance of touch. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 FIG. is a flowchart of a touch interaction algorithm of the present invention.

[0023] Figure 2 FIG. is a schematic diagram of a touch algorithm in the prior art.

[0024] Figure 3 FIG. is a schematic diagram of a capacitive touch screen system in the prior art.

[0025] Figure 4 FIG. is a comparison chart of the touch recognition accuracy of the present invention.

[0026] Figure 5 FIG. is a schematic diagram of multi-user touch point attribution and abnormal point filtering of the present invention.

[0027] Figure 6 FIG. is a comparison chart of the misjudgment rates of the present invention with and without hand tracking. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0029] Example 1. Refer to Figure 1 the flowchart shown. An embodiment of the present invention provides a touch interaction algorithm, which includes the following steps: Step Z1: Collect the original touch signal output by the touch function layer.

[0030] Step Z2: Perform filtering and denoising processing on the original touch signal.

[0031] Step Z3: Analyze the processed touch signal and dynamically adjust the lock point threshold.

[0032] Step Z4: Continuously learn and adapt to the user's usage habits in real time.

[0033] In this embodiment, refer to Figure 2 , which discloses a touch algorithm in the prior art. This algorithm first uses a background subtraction algorithm based on depth images for gesture segmentation, combines improved Canny edge detection and contour area analysis to extract the hand contour, effectively adapts to different lighting environments and enhances the anti-noise ability; then, according to the touch type, it uses a centroid detection (single finger) or convex hull detection (multiple fingers) algorithm to accurately extract the contact point position. However, this algorithm is prone to false touch problems during multi-finger operations. Refer to Figure 3 , which discloses a high-performance multi-channel touch system for large-size capacitive screens in the prior art. This system supports parallel processing of 112×72 channels through hardware innovations such as double-edge integration and differential output, solving the problems of low scan frame rate and slow response speed caused by a large number of channels. At the same time, this system combines a touch prediction scanning algorithm based on long short-term memory network (LSTM) to reduce the invalid scanning area and further improve the scan frame rate. However, in the multi-user close cooperation operation multi-touch scenario, this system cannot determine which user the contact point belongs to, increasing the difficulty of signal processing and resulting in a lack of accuracy in multi-touch.

[0034] Furthermore, in the step Z1, the sensor array in the touch function layer continuously obtains the user's touch operation actions, converts the touch operation actions into electrical signals, thereby forming initial original touch signal data; the touch signal data is stored in the form of a voltage signal matrix, where each element in the matrix corresponds to a specific sensing point in the touch sensor array, recording the voltage or capacitance change at that position during the touch operation.

[0035] In the step Z2, since there may be many environmental interferences and device noises during the user's touch process, the original touch signal data carries interference signals and noises. If it is directly used for recognition without processing, the accuracy of touch recognition will be reduced; therefore, the collected original touch signal data is subjected to filtering and denoising processing. Specifically, a high-pass filter is used to preprocess the touch signal data to remove low-frequency interference signals or DC offsets. The cut-off frequency of the high-pass filter is between 200 Hz and 400 Hz.

[0036] In step Z3, the touch signal data after filtering is input into a convolutional neural network for in-depth analysis. The structure of the convolutional neural network consists of an input layer, three convolutional layers, two pooling layers, a fully connected layer, and a Softmax output layer. The input layer receives a touch signal matrix in the form of M×N, where M and N represent the number of electrodes of the touch sensor array in the X-axis and Y-axis directions, respectively. In the first convolution, 32 3×3 convolutional kernels are used to extract local spatial features, and a ReLU activation function is connected after convolution to enhance the non-linear expression ability. Immediately following is the first 2×2 max pooling layer, which is used to reduce the spatial resolution of the feature map while retaining significant features. The second convolutional layer increases to 64 3×3 convolutional kernels, also using ReLU as the activation function, and a second 2×2 max pooling layer is connected afterwards for further downsampling. The third convolutional layer uses 128 3×3 convolutional kernels to perform high-order abstraction on the previously extracted features. After the convolution operation, the ReLU activation function is connected again, and no pooling layer is added to retain more high-level information. Subsequently, all the convolved feature maps are converted into a one-dimensional vector through a flattening operation and sent into a fully connected layer containing 256 neurons for global information fusion and non-linear mapping. Finally, a Softmax output layer is connected to perform multi-classification prediction on the touch action categories.

[0037] In step Z3, the convolutional neural network is trained using a labeled touch dataset based on supervised learning. The dataset consists of touch operation signal matrices and their corresponding action category labels, and each sample is a two-dimensional matrix with a size of 32x32. Before training, the original data is first normalized, and the signal strength is normalized to the range of [0,1]. The dataset is divided into a training set, a validation set, and a test set, and the ratio of the training set, validation set, and test set is 7:2:1. The cross-entropy loss function is used as the objective function during the training phase; the Adam optimizer is selected, and the initial learning rate is set to 0.001. During the training process, each batch contains multiple touch samples. The batch data is sequentially fed into the neural network for forward propagation to generate prediction results. Subsequently, the loss is calculated by comparing with the true labels, and the gradient of the convolutional kernel weights and the fully connected layer parameters is updated using the backpropagation algorithm.

[0038] In step Z3, the touch interaction algorithm involves a dynamic adjustment mechanism for the lock point threshold, specifically using the exponential weighted average method for dynamic adjustment. The real-time prediction probability output by the convolutional neural network is weighted and averaged with the threshold at the previous moment to ensure the smoothness and fast response ability of the threshold adjustment. The weighting coefficient takes values between 0.5 and 0.7. Refer to Figure 4, which shows the comparison of the accuracy change trends of the "static threshold strategy" and the "dynamic threshold adjustment mechanism based on convolutional neural network" in the touch recognition task within 20 training cycles. The recognition accuracy of the static threshold strategy fluctuates between 75% and 79% throughout the training process, while the accuracy of the dynamic threshold adjustment mechanism continuously improves as the training progresses and reaches approximately 94.5% in the 20th round, an increase compared to the static threshold strategy, with an overall improvement of more than 20%. The experimental results show that the dynamic threshold adjustment mechanism effectively enhances the model's adaptability to different touch environments and user behavior differences and reduces the misjudgment rate.

[0039] In step Z4, the touch interaction algorithm introduces a user behavior learning module. The user behavior learning module is based on a long short-term memory network, uses the historical user touch behavior sequence as the training input, captures the changing trends of the user's long-term or short-term operation habits, and accordingly makes personalized adjustments and optimizations to the convolutional neural network model and the lock point threshold parameters; through this method, the touch interaction algorithm can adapt to the personalized differences shown by the user in daily touch operations, such as finger size, touch force, frequency, and speed, etc. After learning the user's habits and optimizing its own model parameters, it can actively predict the user's touch action tendency and optimize the signal recognition result accordingly, thereby improving the personalized touch response accuracy and user experience satisfaction.

[0040] In this embodiment, the following is the Python code of the convolutional neural network structure: class TouchCNN(nn.Module): def __init__(self, input_size=32, num_classes=4): super(TouchCNN, self).__init__() self.conv1 = nn.Conv2d(in_channels=1, out_channels=32, kernel_size=3, stride=1, padding=1) self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2) self.conv2 = nn.Conv2d(in_channels=32, out_channels=64,kernel_size=3, stride=1, padding=1) self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2) self.conv3 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3, stride=1, padding=1) self.fc1 = nn.Linear(128 * (input_size / / 4) * (input_size / / 4), 256) self.fc2 = nn.Linear(256, num_classes) def forward(self, x): x = F.relu(self.conv1(x)) # First convolutional layer + ReLU x = self.pool1(x) # First pooling layer x = F.relu(self.conv2(x)) # Second convolutional layer + ReLU x = self.pool2(x) # Second pooling layer x = F.relu(self.conv3(x)) # Third convolutional layer + ReLU x = x.view(x.size(0), -1) # Flattening x = F.relu(self.fc1(x)) # First fully connected layer + ReLU x = self.fc2(x) # Output layer return F.log_softmax(x, dim=1) # Softmax activation + log output if __name__ == "__main__": model = TouchCNN(input_size=32, num_classes=4) print(model) dummy_input = torch.randn(8, 1, 32, 32) output = model(dummy_input) print("Output size:", output.shape) # Expected to be [8, 4] Further, in step S1, a back-projection interactive desktop constructed based on the FTIR principle is used as a touch detection platform, which is composed of an acrylic diffusion plate with a size of 1.2 meters × 0.9 meters, installed at the height of the user's waist. The diffusion plate projects visual content from below through a mirror system, and at the same time, another acrylic light guide plate injected with infrared light is closely installed above it. After the infrared light is injected along the edge of the light guide plate, total internal reflection is formed at the interface between the acrylic and the air, and the light circulates and propagates inside without leaking outwards. When the user's finger touches the upper surface of the light guide plate, this total internal reflection phenomenon will be locally destroyed at the touch point, resulting in some infrared light overflowing to form a bright spot. An infrared camera is arranged below the light guide plate, and the infrared bright spot is captured in real time through an optical channel shared with the projection optical path. Each frame of image is first subjected to image difference processing with a static background image, and after removing the invariant area, it is then subjected to binary conversion according to a set gray threshold to convert the bright spot area into a connected domain in a black-and-white image. Then, the geometric center of each non-connected bright area is extracted through a contour analysis algorithm as the precise position of the touch point. All touch point data are collected through a touch sensor , and each touch point contains its coordinate and contact timestamp information. In order to enhance data stability and filter the influence of short-term noise in the environment on recognition, a "delayed reporting mechanism" is introduced. If the contact point remains stable in consecutive frames, it is officially submitted to the upper-layer interaction logic as a valid touch event. Each confirmed touch point will be assigned a unique ID number for subsequent tracking of its movement trajectory and life cycle until the user's finger leaves the upper surface of the light guide plate, and the ID will be released and reset to zero

[0041] In step S2, during hand detection and tracking, a skin-color-based RGB threshold segmentation method is adopted as the recognition mechanism. By extracting the pixels belonging to the skin in the image, the positioning of the hand area is achieved. Specifically, each frame of the image is obtained through a visible light camera installed above the touch table, and then threshold judgment is performed on each pixel in the RGB color space of the image to screen out the pixel point area falling within the skin color range, and the pixel point area is converted into a binary image. The cvFindContours function in the OpenCV library is called to extract the contours of the white connected areas in the image, and each contour is regarded as a potential hand or arm area. To exclude the noise interference in the image, the contours with smaller areas are removed, and only the main contours that meet the area and shape characteristics are retained. Next, by scanning the point sets in each contour, the minimum value and the maximum value in the Y-axis direction are extracted, thereby determining two key points: one is the "arm end" close to the table edge (i.e., the end point close to the table), and the other is the "finger end" far from the table edge. The set of hand center points in each frame of the image is defined as . Based on the positional relationship of these geometric features, the relative standing position of the user is judged. For example, whether the user is on the first side (smaller Y-axis) or the second side (larger Y-axis) of the table, and the attribution relationship between the user and the touch point is established accordingly. To adapt to the possible impacts of different application backgrounds, lighting conditions, and tabletop projection contents on skin color detection, the system has the ability to dynamically adjust the skin color threshold in real time. By continuously monitoring the screen color change and the hand recognition stability, the RGB threshold range is automatically updated to ensure accurate and stable hand detection effects under different contents or environments.

[0042] In this embodiment, referring to Figure 5 , the red color represents the touch points determined as noise; the blue color represents the touch of user A; the green color represents the touch of user B; the white color represents the detected skin; the yellow circle represents the acceptable space area where the touch occurs.

[0043] In step S3, the Euclidean distance nearest principle is adopted to perform spatial matching between each hand position and the touch point set. For any touch point , if its distance from a hand point is less than the set threshold , then it is determined that the touch point belongs to the user and action group corresponding to the hand point . The setting of the threshold is calibrated according to the display panel size and the camera view angle, and is 30 - 50 pixels.

[0044] In step S3, after completing the spatial pairing of the touch points and the hand positions, it enters the fusion and optimization stage. First, based on the established hand attribution relationship, multiple touch points that are highly close in space-time and belong to the same hand are logically grouped. This logical grouping is not only based on position proximity but also incorporates the feature of time continuity, so as to be able to identify multi-point operations belonging to the same interaction intention. By judging the relative movement trajectories, distance change trends, and direction consistency among these touch points, it is further analyzed whether they form specific composite gestures, such as two-finger pinch, three-finger rotation, four-finger swipe, etc. This process is matched with the gesture pattern library, and convolutional or time-series analysis methods are used to evaluate the dynamic features of the touch trajectories in real time to support rich gesture recognition capabilities. At the same time, all "spatially isolated touch points" that are not effectively attributed to any hand are filtered. These touch points usually exceed a set threshold from the center point of all hands. , under the premise of no clear hand association, such touch points are very likely to be mis-touches caused by environmental interference (such as reflection, water droplets, non-user body part occlusion), so they are automatically marked as abnormal points by the system and excluded from the gesture recognition process.

[0045] To further improve the collaboration efficiency and personalized control of the interaction, a unique ID is assigned to each hand, and the operation direction of the user is determined in combination with the position of the hand at the screen boundary, so as to realize a dynamically optimized touch response and role assignment mechanism. By analyzing the Y-axis coordinate relationship between the "finger tip" and the "table edge" of the hand, it is judged whether the user is on the first side, the second side, or the horizontal boundary of the screen, so as to accurately identify the relative orientation of the user. On this basis, combining the hand ID with the user orientation, a user role mapping table is established, so that different functions and permissions can be assigned to different users. For example, in a multi-user collaboration environment, the system can set that the first user (UserA) standing above the screen controls the rotation and zoom operations of the interface view, while the second user (UserB) standing below focuses on details such as data screening and information annotation.

[0046] In this embodiment, referring to Figure 6 , for a system without hand tracking, the misjudgment rate increases significantly when the number of users increases. Especially when the number of concurrent users exceeds 3, the misjudgment rate climbs rapidly, reaching up to nearly 26%. After integrating hand tracking, the system can effectively distinguish the touch sources and combine the spatial pairing mechanism to judge the attribution of the touch points, reducing the overall misjudgment rate by about 40%, and the growth trend is more gentle, maintaining below 15%.

[0047] Embodiment 2. The embodiment of the present invention also provides a touch interaction module, which includes a substrate layer, a touch function layer, and a protective layer. The substrate layer is used to carry the upper touch function layer and the protective layer. The material of the substrate layer is a tempered glass substrate, and the thickness is controlled within the range of 0.5 mm to 1.5 mm. In this embodiment, a tempered glass substrate with a thickness of 1.0 mm is preferably used as the material of the substrate layer.

[0048] The touch function layer is disposed on one surface of the substrate layer and is used to monitor the user's touch actions in real time and accurately convert the touch actions into recognizable electrical signals for output. The touch function layer includes two interleaved sensor arrays, an X electrode array and a Y electrode array. Among them, the width of each electrode array is between 3 and 10 μm, and the spacing between the arrays is set between 200 and 500 μm.

[0049] In this embodiment, a silver nanowire conductive layer structure with a width of 5 μm and a spacing of 300 μm is preferably adopted. The silver nanowire conductive layer structure not only ensures conductivity and touch response speed, but also has excellent light transmittance, and the light transmittance can be higher than 90%. Therefore, it can effectively guarantee the optical performance and touch accuracy of the module.

[0050] In order to protect the touch function layer from external force impacts and daily wear, a protective layer is disposed on the other surface of the touch function layer. The protective layer includes a main body part and a hollowed-out part. The main body part is a continuous surface structure, which is used to block external mechanical impacts and prevent environmental pollutants from damaging the touch function layer, thereby extending the service life of the module; the hollowed-out part adopts a honeycomb-shaped hollow structure, which increases the actual contact area between the protective layer and the connection layer located below it, so that the structure of the protective layer improves the overall adhesion, structural stability, and signal transmission sensitivity of the touch module while maintaining the original protection function. The diameter of the hollowed-out holes is controlled between 50 μm and 150 μm, and the hole spacing is between 100 μm and 300 μm.

[0051] In this embodiment, the protective layer is made of a transparent polymer material, and the material includes high-performance polymer materials such as polycarbonate (PC) or polymethyl methacrylate (PMMA).

[0052] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0053] Finally, the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A touch interaction algorithm, characterized in that, It includes the following steps: Step Z1, collecting the original touch signal output by the touch function layer; Step Z2, performing filtering and denoising processing on the original touch signal; Step Z3, analyzing the processed touch signal and dynamically adjusting the lock point threshold; Step Z4, learning and adapting to the user's usage habits in real time; The touch interaction algorithm also introduces a method combining multi-touch and hand tracking to reduce the signal interference of multi-touch during multi-person operation. The specific steps are as follows: Step S1, detecting the infrared light leakage generated by finger touch in the rear projection desktop constructed by the acrylic light guide plate by using the FTIR principle, extracting the touch point data by combining image difference and binarization and assigning a unique ID, and introducing a delay reporting mechanism to improve the recognition accuracy and anti-interference ability; Step S2, determining the relative position relationship between the finger and the arm in the picture by analyzing the Y-axis distribution characteristics of the hand contour in the image, and inferring the spatial position and direction of the hand accordingly; Step S3, spatially matching each hand position with the touch point set, logically grouping the simultaneously occurring touch points, and eliminating the "spatially isolated" touch points to reduce misjudgment.

2. The touch interaction algorithm according to claim 1, wherein , By closely installing another acrylic light guide plate injected with infrared light above the acrylic diffusion plate with a size of 1.2 meters × 0.9 meters, the infrared light forms total internal reflection inside the light guide plate. When the user's finger touches the surface of the light guide plate, the total internal reflection phenomenon is locally damaged, and the infrared light leaks to generate a bright spot; the infrared camera set below the light guide plate captures each frame of infrared image, extracts the touch point data through image difference and gray-scale binarization technology, assigns a unique ID to each touch point, tracks the touch point trajectory, and cooperates with the hand space position tracking technology based on RGB skin color threshold segmentation to identify the user's hand contour, and determines the user's spatial orientation by analyzing the relative position between the finger and the arm. Then, according to the principle of the closest Euclidean distance, each touch point is spatially matched with the identified hand position, and the spatially isolated touch points that are irrelevant to the hand and exceed the set threshold of 30 - 50 pixels are eliminated to reduce the mis-touch points generated by environmental interference and multi-person simultaneous operation.

3. The touch interaction algorithm according to claim 1, wherein , In step S1, a rear projection interactive desktop based on the FTIR principle is used as the touch detection platform. The acrylic diffusion plate is installed at the height of the user's waist. The diffusion plate projects visual content from below through a mirror system, and at the same time, another acrylic light guide plate injected with infrared light is closely installed above it; after the infrared light is injected along the edge of the light guide plate, total internal reflection is formed at the interface between the acrylic and the air, and the light circulates inside without leaking outwards.

4. A touch interaction algorithm according to claim 1, characterized in that , When the user's finger touches the upper surface of the light guide plate, the total internal reflection phenomenon will be locally damaged at the touch point, resulting in some infrared light escaping to form a bright spot; an infrared camera is set below the light guide plate to capture the infrared bright spot in real time through the optical channel shared with the projection light path. Each frame of image is first subjected to image difference processing with the static background image, and after eliminating the unchanged area, it is then binarized according to the set gray-scale threshold, and the bright spot area is converted into a connected domain in the black and white image; Collect all touch point data through the touch sensor , and each touch point contains its coordinate and contact timestamp information; each confirmed touch point will be assigned a unique ID number for subsequent tracking of its movement trajectory and life cycle until the user's finger leaves the upper surface of the light guide plate, at which point the ID will be released and reset to zero.

5. A touch interaction algorithm according to claim 1, characterized in that, In the step S3, the Euclidean distance nearest principle is adopted to perform spatial matching between each hand position and the touch point set. For any touch point , if its distance from a certain hand point is less than the set threshold , it is determined that the touch point belongs to the user and action group corresponding to the hand point . The setting of the threshold is calibrated according to the display panel size and the camera view angle.

6. The touch interaction algorithm according to claim 1, wherein In the step S3, after the touch points and the hand positions are paired, based on the hand attribution relationship, the touch points that are spatially and temporally close and belong to the same hand are logically grouped, and by synthesizing the position proximity and time continuity, the multi-point operations with unified interaction intentions are identified; by analyzing the trajectories, distance changes and direction consistency between the touch points, combined with the gesture pattern library and convolutional real-time recognition of dynamic features; And automatically filter out the isolated touch points that have no effective association with any hand and whose distance exceeds the threshold d0, excluding the false touches caused by environmental interference.

7. A touch interaction algorithm according to claim 1, wherein By analyzing the Y-axis coordinate relationship between the "finger tip" and the "table edge" of the hand, it is judged whether the user is on the first side, the second side or the horizontal boundary of the screen, so as to accurately identify the relative orientation of the user. On this basis, combining the hand ID with the user orientation, a user role mapping table is established to assign different functional permissions to different users.

8. The touch interaction algorithm according to claim 1, wherein, In the step Z3, the touch interaction algorithm involves a dynamic adjustment mechanism for the lock point threshold, specifically using the exponential weighted average method for dynamic adjustment.

9. A touch interaction module, characterized in that, Applied to a touch interaction algorithm according to any one of claims 1-8, the module includes a substrate layer, a touch function layer and a protective layer; the substrate layer is used to carry the upper touch function layer and the protective layer, and the material of the substrate layer is a tempered glass substrate with a thickness controlled within the range of 0.5 mm to 1.5 mm; the touch function layer is arranged on one side surface of the substrate layer and is used to monitor the touch actions of the user in real time and accurately convert the touch actions into electrical signals for output; in order to protect the touch function layer from the influence of external force impact and daily wear, a protective layer is arranged on the other side surface of the touch function layer, and the protective layer includes a main body part and a hollow part. The main body part is a continuous surface structure for blocking external mechanical impacts and preventing environmental pollutants from damaging the touch function layer, and the hollow part adopts a honeycomb-shaped hollow structure, which increases the actual contact area between the protective layer and the connection layer located below it.

Images