A multifunctional touch screen assembly and signal processing system

Abstract

A multifunctional touch screen assembly and signal processing system, comprising a touch sensing module, a signal acquisition and transmission module, a signal preprocessing module, a high-level function calculation module, a touch event recognition and generation module, a system management and control module, and an output and interface module, the touch sensing module is used for detecting and collecting original touch event information, the signal acquisition and transmission module is used for acquiring and transmitting original signals, the signal preprocessing module is used for original signal preprocessing, the high-level function calculation module is used for touch point detection and positioning and signal processing, the touch event recognition and generation module is used for touch event conversion, the system management and control module is used for coordinated management of the system, and the output and interface module is used for touch event output. The application proposes an improved dual-domain adaptive capacitive touch positioning algorithm for touch point detection and positioning, and an improved multi-modal touch semantic fusion algorithm for signal processing, providing a more optimal solution for a multifunctional touch screen assembly and signal processing system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of signal processing and human-computer interaction, in particular to a multifunctional touch screen assembly and signal processing system. BACKGROUND

[0002] Signal processing technology is a technology that collects, transforms, enhances, compresses, and analyzes original physical signals (such as voltage, current, capacitance, and sound waves) through mathematical algorithms and hardware systems to extract useful information, suppress noise interference, recognize features, and finally output structured data for decision-making. Signal processing technology includes analog circuit filtering (such as RC active filter for noise suppression), Fourier analysis (time domain signal frequency domain processing), sampling theorem (Nyquist-Shannon theorem to solve the problem of discretization accuracy), linear system theory (Z-transform and Laplace transform modeling dynamic system), and digital signal processing hardware (DSP chip to realize real-time FFT operation). These technologies collectively solve the key problems of signal noise reduction, feature extraction, and real-time processing. The mutual cooperation of these technologies provides a more accurate signal processing method for a multifunctional touch screen assembly and signal processing system.

[0003] Human-computer interaction technology is a theory and method that studies, designs, and implements bidirectional information exchange between users and computer systems. It is a technology that converts user physical operations (such as touch and voice) into machine executable instructions through hardware interfaces (such as touch screens and sensors) and software logic (such as gesture recognition and intent analysis). Human-computer interaction technology includes input hardware innovation (such as mouse, touch screen, and force sensor), interaction paradigm breakthrough (command line → WIMP interface → natural user interface NUI), core theoretical model (Fitts law quantifying operation efficiency, Norman's seven principles of interaction), and multi-modal fusion (voice + gesture + vision). The mutual cooperation of these technologies lays a good hardware and algorithm foundation for semantic interaction of modern touch screens (such as pressure penmanship and gesture intent analysis). SUMMARY

[0004] To solve the above problems, the present application aims to provide a multifunctional touch screen assembly and signal processing system.

[0005] In order to achieve the above object, the present application provides the following technical scheme: a multifunctional touch screen assembly and signal processing system, comprising a touch sensing module, a signal acquisition and transmission module, a signal preprocessing module, a high-level function calculation module, a touch event recognition and generation module, a system management and control module, and an output and interface module, the touch sensing module is used for detecting and collecting original touch event information, the signal acquisition and transmission module comprises a signal acquisition unit and a signal transmission unit, the signal acquisition unit is used for acquiring original signals generated by the sensor module, the signal transmission unit is used for transmitting the original signals to the signal preprocessing module, the signal preprocessing module is used for initial processing of the original signals, the high-level function calculation module comprises a touch point detection and positioning unit and a signal processing unit, the touch point detection and positioning unit proposes an improved dual-domain adaptive capacitive touch positioning algorithm for detecting and positioning the touch point, the signal processing unit proposes an improved multi-modal touch control semantic fusion algorithm for multifunctional signal processing, the touch event recognition and generation module is used for converting the information calculated at the bottom layer into high-level touch events that can be understood by the operating system and application programs, the system management and control module is used for coordinating the work of the entire touch system, and the output and interface module is used for transmitting the finally generated touch events to the host system.

[0006] Further, the touch sensing module detects and collects original touch event information through a special physical sensor array integrated in the screen structure, the touch sensing module constitutes a physical layer of the touch screen assembly, and various interactive actions of a user with the screen surface are detected in real time, and the physical contact behaviors are converted into original electrical signals that can be processed by subsequent circuits.

[0007] Further, the signal acquisition unit is used for acquiring original analog signals generated by the touch sensing module, the original weak touch signals are amplified through an analog front-end circuit first, and then the analog signals are converted into high-precision digital signals through an analog-to-digital converter;

[0008] The signal transmission unit is used for reliably and efficiently transmitting the high-precision digital signals to the subsequent signal preprocessing module, the data converted by the analog-to-digital converter is packaged into a data packet in a predetermined format, and is transmitted in compliance with a specific communication protocol.

[0009] Further, the signal preprocessing module is used for initial processing of the original signals, irrelevant signals such as high-frequency electromagnetic interference and power supply noise are filtered out through a digital filter, random noise is suppressed through signal averaging and sliding window smoothing technology, and the clarity of the effective touch signal is enhanced.

[0010] Furthermore, the touch detection and positioning unit proposes an improved dual-domain adaptive capacitive touch positioning algorithm to detect and locate touch points. By analyzing the pre-processed signal, it determines the existence of effective touch points and their precise coordinate positions on the screen, and conducts touch durability tests on the multi-functional display assembly product.

[0011] Furthermore, the improved dual-domain adaptive capacitive touch positioning algorithm is as follows: the driving circuit sequentially activates each row of transmitting electrodes in a time-division multiplexing manner. When the first When the row transmitting electrode is activated, the column receiving electrode... Measurement of the emitter electrode using a charge integrating amplifier The mutual capacitance value, the mutual capacitance value ,in, For the first line, number The original capacitance value at the column intersection. This is the baseline capacitance value when there is no touch. The amount of capacitance decay caused by touch, then... Baseline correction and filtering are performed to extract the effective signal change matrix, i.e. ,in, For the first line, number The effective signal strength of the column, To suppress high-frequency noise, the digital filtering operation results in the following effective signal transformation matrix after noise reduction: ,in, For the effective signal change matrix, For the row emission electrode Total number of rows For column receiving electrodes The total number of columns, then scan. Areas where the signal strength exceeds a dynamic threshold are designated as candidate touch points, and the dynamic threshold is set to... ,in, For dynamic detection threshold, The peak attenuation coefficient is... It is a function with maximum value. The background noise figure represents the effective signal strength at each signal point. Perform threshold decision, i.e. ,in, For the binary mask matrix, Perform an 8-neighborhood scan and aggregate adjacent activation points into candidate regions. ,in, For candidate region indexing, the set of candidate regions labeled with connected components is: ,in, is the first candidate region, is the second candidate region, is the first candidate region, is the total number of candidate regions, for each candidate region , the weighted barycentric coordinates are calculated , wherein, is the horizontal coordinate of the weighted barycenter, is the physical center position coordinate of the first column electrode, is the intersection of the first row and the first column, is the vertical coordinate of the weighted barycenter, is the physical center position coordinate of the first row electrode, a preliminary coordinate set is obtained wherein, is the barycentric coordinate of the first candidate region, is the barycentric coordinate of the second candidate region, is the barycentric coordinate of the first candidate region;

[0012] Then, in order to solve the problem of asymmetric electrode distribution in the edge region of the touch screen, a nonlinear compensation function is proposed to correct the coordinates, i.e. , wherein, is the nonlinearly compensated horizontal coordinate of the weighted barycenter, is the compensation coefficient, is the horizontal coordinate of the nearest edge, is the nonlinearly compensated horizontal coordinate of the weighted barycenter, is the compensation coefficient, is the vertical coordinate of the nearest edge, is the edge attenuation function, i.e. wherein, is the exponential operation, is the attenuation coefficient, is the distance of the touch point from the edge, taking the left edge as an example, then wherein, is the absolute coordinate of the left edge of the physical boundary of the touch screen screen in the horizontal direction, the calculation method of the distance of the touch point from the right, upper and lower edges is consistent with the distance of the touch point from the left edge, in order to solve the problem of interaction confusion caused by noise, scanning delay and touch point concentration, a multi-point tracking and ID association is proposed to establish a spatiotemporal continuity model of the touch point, i.e. the current frame coordinates are associated with the last frame coordinates, a unique touch point ID is assigned, and the nearest neighbor association is wherein, is the ID assigned to the centroid coordinate of the is the corrected x-coordinate of the centroid coordinate of the is the corrected y-coordinate of the centroid coordinate of the is the corrected x-coordinate of the centroid coordinate of the is the corrected y-coordinate of the centroid coordinate of the is the Euclidean norm operation, is the calculation of the minimum value, and the final output is the set of touch physical coordinates with a unique ID , the improved dual-domain adaptive capacitive touch positioning algorithm first calculates the mutual capacitance value, then performs baseline correction and filtering to extract the effective signal change matrix, then marks the regions with signal strength exceeding the dynamic threshold as candidate touch points and calculates the weighted centroid coordinates of each candidate region, then proposes a non-linear compensation function to solve the problem of asymmetric electrode distribution in the edge area of the touch screen, and finally proposes multi-point tracking and ID association to solve the problem of interaction confusion caused by noise, scanning delay, and dense touch points, so as to detect and locate the accurate coordinate position of the effective touch point on the screen. Further, the signal processing unit proposes an improved multi-modal touch control semantic fusion algorithm for multi-functional signal processing, which converts the original touch physical position data into interactive information with semantic information, stability and reliability, and directly usable for application, and performs a vibration test on the multi-functional display screen assembly product.

[0013] Further, the improved multi-modal touch control semantic fusion algorithm is as follows: after processing by the touch detection and positioning unit, the set of touch physical coordinates with a unique ID is

[0014] , first, to suppress touch jitter, Kalman filtering is performed for touch trajectory optimization to provide a smooth motion trajectory, the state prediction is , wherein is the predicted state vector, is the state transition matrix, is the state vector at time t, and the covariance prediction is , wherein is the predicted state vector, To estimate the covariance a priori, for The posterior estimate of the covariance at time t. State transition matrix transpose, Let be the process noise covariance matrix, and let Kalman gain be... ,in, Here is the Kalman gain matrix. For the observation matrix, Observation matrix transpose, To observe the noise covariance matrix, the state is updated as follows: ,in, For posterior state estimation, i.e., the smoothed contact state, the smoothed contact coordinates are: Then, a feature extraction method is proposed to distinguish contact types, and the contact area of ​​the contact points is calculated as follows: ,in, For the first The contact area of ​​each contact point Let be the length of the major axis of the equivalent ellipse of the contact area. Let be the length of the minor axis of the equivalent ellipse of the contact region, and , ,in, and The second-order moment eigenvalues ​​of the signal are used for classification decisions. , ,in, This is a classification decision value used to determine the contact type. For the weight vector of a Support Vector Machine (SVM), for transpose, For feature vectors, For kernel function, For the bias of Support Vector Machine (SVM), The ratio of the major axis to the minor axis. The contact movement speed, The signal variation coefficient is used, and the contact type determination rule is as follows: ,in, For the first Contact type of each contact point , and These are the contact point types: fingertip, pen tip, and palm.

[0015] To improve interaction accuracy, pressure-sensitive mapping is proposed to map capacitive signals to pressure levels. The pressure-sensitive mapping process is as follows: ,in, For normalized pressure values, is the original capacitance signal, is the maximum background noise value when not touching, is the medium pressure demarcation point, is the maximum pressure saturation point, then gesture recognition is proposed to solve the problem of contact ambiguity, and trajectory feature extraction is wherein, is the motion trajectory feature vector of the contact , is the horizontal displacement increment of the contact at the moment , is the vertical displacement increment of the contact at the moment , is the horizontal displacement increment of the contact at the moment , is the vertical displacement increment of the contact at the moment , dynamic time warping (DTW) template matching is wherein, is the standard trajectory sequence of the predefined gesture template , is the minimum cumulative distance between and , is the set of regularized paths, is the first frame displacement vector of the current trajectory feature vector , is the first frame displacement vector of the standard trajectory sequence , is the Euclidean norm operation, and the final output is a multi-dimensional touch attribute information containing touch coordinates, touch type, pressure level, and gesture semantics. The improved multi-modal touch semantic fusion algorithm first optimizes the touch trajectory through Kalman filtering to provide a smooth motion trajectory, then extracts the features of the touch to distinguish the touch type, then proposes a pressure mapping to improve the interaction accuracy, and finally proposes a gesture recognition to analyze the correlation of multiple touches, so as to realize multi-functional signal processing, and convert the original touch physical position data into interactive information with semantic information, stable and reliable, which can be directly used by applications.

[0016] Further, the touch event recognition and generation module converts the semantic touch information into a standard interactive event that can be parsed by the operating system according to the processing result of the signal processing unit, realizing the bridge function from physical interaction to application logic.

[0017] Further, the system management and control module is used for coordinating hardware resources, optimizing algorithm parameters, guaranteeing system robustness and realizing energy efficiency management and control.

[0018] The output and interface module is used for encapsulating the processed high-dimensional interaction data into a standard protocol format and transmitting to the operating system / application layer in real time through a physical / logical channel.

[0019] Meanwhile, appearance / function inspection, high-temperature storage, high-temperature work, low-temperature storage, low-temperature work and constant damp heat test are carried out on the multifunctional display screen assembly product.

[0020] Compared with the prior art, the application has the following beneficial effects:

[0021] 1. The improved dual-domain adaptive capacitive touch positioning algorithm is used to detect and locate the touch point, the existence of the effective touch point and the accurate coordinate position of the touch point on the screen are determined by analyzing the preprocessed signal, the improvement of the dual-domain adaptive capacitive touch positioning algorithm is that the mutual capacitance value is calculated first, then baseline correction and filtering are performed to extract the effective signal change matrix, then the region with signal strength exceeding the dynamic threshold is marked as a candidate touch point, and the weighted centroid coordinates of each candidate region are calculated, then a nonlinear compensation function is proposed to solve the problem of asymmetric electrode distribution in the edge region of the touch screen, finally, multi-point tracking and ID association are proposed to solve the problem of interaction confusion caused by noise, scanning delay and touch point density, so as to detect and locate the touch point and determine the accurate coordinate position of the effective touch point on the screen.

[0022] 2. The improved multi-modal touch semantic fusion algorithm is used for multifunctional signal processing, the innovation of the application is that the multi-modal touch semantic fusion algorithm is improved, first, Kalman filtering is used for touch trajectory optimization to provide a smooth motion trajectory, then feature extraction is performed on the touch point to distinguish the touch point type, then pressure mapping is proposed to improve the interaction accuracy, finally, gesture recognition is proposed for multi-touch correlation analysis, so as to realize multifunctional signal processing and convert the original touch physical position data into interactive information with semantic information, stability and reliability, which can be directly used by applications. BRIEF DESCRIPTION OF DRAWINGS

[0023] The application is further illustrated by the drawings, but the embodiments in the drawings do not constitute any limitation on the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the following drawings.

[0024] Figure 1 X-axis vibration test detection results of the multifunctional display screen assembly;

[0025] Figure 2Y-axis vibration test detection results of the multifunctional display screen assembly;

[0026] Figure 3 Z-axis vibration test detection results of the multifunctional display screen assembly;

[0027] Figure 4 A working flowchart of a multifunctional touch screen assembly and a signal processing system;

[0028] Figure 5 Design drawings of the multifunctional display screen assembly product;

[0029] Figure 6 Physical drawings of the multifunctional display screen assembly product;

[0030] Figure 7 A rendering wireframe diagram of the multifunctional display screen assembly product. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0032] A multifunctional touch screen assembly and a signal processing system, comprising a touch sensing module, a signal acquisition and transmission module, a signal preprocessing module, a high-level function calculation module, a touch event recognition and generation module, a system management and control module, and an output and interface module. The touch sensing module is used for detecting and collecting original touch event information. The signal acquisition and transmission module comprises a signal acquisition unit and a signal transmission unit. The signal acquisition unit is used for acquiring original signals generated by the sensor module, and the signal transmission unit is used for transmitting the original signals to the signal preprocessing module. The signal preprocessing module is used for initial processing of the original signals. The high-level function calculation module comprises a touch point detection and positioning unit and a signal processing unit. The touch point detection and positioning unit proposes an improved dual-domain adaptive capacitive touch positioning algorithm to detect and position the touch point. The signal processing unit proposes an improved multi-modal touch semantic fusion algorithm for multifunctional signal processing. The touch event recognition and generation module is used for converting the information calculated at the bottom layer into high-level touch events that can be understood by the operating system and application programs. The system management and control module is used for coordinating the work of the entire touch system and processing tasks related to system performance and user experience but not related to touch core processes. The output and interface module is used for transmitting the finally generated touch events to the host system (the main processor / operating system of the device).

[0033] Preferably, the touch sensing module detects and collects the original touch event information through a dedicated physical sensor array integrated in the screen structure, which constitutes the physical layer of the touch screen assembly, including a capacitive sensor array, a resistive film layer, an infrared grid, a sound wave sensor, a force sensing element, different types of touch (finger, stylus, multi-point, pressure) are initially detected, real-time detection of various interactive actions (such as finger touch, press, hover, stylus operation) between the user and the screen surface, and the conversion of these physical contact behaviors into original electrical signals that can be processed by subsequent circuits.

[0034] Preferably, the signal acquisition unit is used to acquire the original analog signal generated by the touch sensing module (such as a capacitive sensor array, a resistive film, a force sensing element), which first amplifies the original weak touch signal (such as capacitance change, voltage fluctuation, pressure value) through an analog front-end circuit, and then converts the analog signal into a high-precision digital signal through an analog-to-digital converter (ADC).

[0035] The signal transmission unit is used to reliably and efficiently transmit the high-precision digital signal to the subsequent signal preprocessing module, and the data converted by the analog-to-digital converter (ADC) is packaged into a data packet in a predetermined format and transmitted in compliance with a specific communication protocol (such as I2C, SPI, USB standard serial bus protocol).

[0036] Preferably, the signal preprocessing module is used for initial processing of the original signal, which filters out irrelevant signals such as high-frequency electromagnetic interference and power noise through a digital filter (such as a low-pass, band-pass, or adaptive filter), while suppressing random noise through signal averaging and sliding window smoothing techniques to enhance the clarity of the effective touch signal.

[0037] Preferably, the touch detection and positioning unit proposes an improved dual-domain adaptive capacitive touch positioning algorithm for touch detection and positioning, which analyzes the preprocessed signal to determine the existence of an effective touch point and its precise coordinate position on the screen, and performs a touch durability test on the multifunctional display screen assembly product, and the test result data is shown in Table 1:

[0038] Table 1 Touch durability test detection results of multifunctional display screen assembly

[0039]

[0040] Specifically, the improved dual-domain adaptive capacitive touch positioning algorithm is as follows: the drive circuit activates each row of transmission electrodes in a time-division multiplexing manner When the first row of transmission electrodes is activated, the column of receiving electrodes measures the mutual capacitance value with the transmission electrode through a charge integration amplifier, and the mutual capacitance value is ,in, For the first line, number The original capacitance value at the column intersection. The baseline capacitance value when there is no touch (obtained through initial calibration). Capacitance attenuation caused by touch ( ), and then to Baseline correction and filtering are performed to extract the effective signal change matrix, i.e. ,in, For the first line, number The effective signal strength of the column, To suppress high-frequency noise, the digital filtering operation results in the following effective signal transformation matrix after noise reduction: ,in, For the effective signal change matrix, For the row emission electrode Total number of rows For column receiving electrodes The total number of columns, then scan. Areas where the signal strength exceeds a dynamic threshold are designated as candidate touch points, and the dynamic threshold is set to... ,in, For dynamic detection threshold, The peak attenuation coefficient is... It is a function with maximum value. The background noise figure represents the effective signal strength at each signal point. Perform threshold decision, i.e. ,in, For the binary mask matrix, Perform an 8-neighborhood scan and aggregate adjacent activation points into candidate regions. ,in, For candidate region indexing, the set of candidate regions labeled with connected components is: ,in, This is the first candidate region. This is the second candidate region. For the first Candidate regions, For each candidate region, the total number of candidate regions is [number]. Calculate its weighted centroid coordinates ,Right now , ,in, The x-coordinate of the weighted centroid For the first The physical center coordinates of the column electrodes. For the first Row, the intersection of column, the longitudinal coordinate of the weighted centroid, the longitudinal coordinate of the weighted centroid, the physical center position coordinate of the row electrode, obtain the preliminary coordinate set wherein, the centroid coordinate of the first candidate region, the centroid coordinate of the second candidate region, the centroid coordinate of the first candidate region;

[0041] Then, in order to solve the problem of asymmetric electrode distribution of the screen edge area of the touch screen, a nonlinear compensation function is proposed to correct the coordinates, that is, , wherein, the horizontal coordinate of the weighted centroid after nonlinear compensation, the compensation coefficient, the horizontal coordinate of the nearest edge, the horizontal coordinate of the weighted centroid after nonlinear compensation, the compensation coefficient, the longitudinal coordinate of the nearest edge, the edge attenuation function, that is wherein, the exponential operation, the attenuation coefficient, the distance of the touch point from the edge, taking the left edge as an example, then wherein, the absolute coordinate of the left edge of the physical boundary of the touch screen screen in the horizontal direction, the calculation method of the distance of the touch point from the right, upper and lower edges is consistent with the distance of the touch point from the left edge, in order to solve the problem of interaction confusion caused by noise, scanning delay and touch point density, a multi-point tracking and ID association is proposed to establish a space-time continuity model of the touch point, that is, the current frame coordinates are associated with the last frame coordinates, and a unique touch point ID is assigned, and the nearest neighbor association is wherein, the centroid coordinate of the first candidate region at the moment is assigned an ID, the horizontal coordinate of the centroid coordinate of the first candidate region at the moment after correction, the horizontal coordinate of the centroid coordinate of the first candidate region at the moment after correction, the longitudinal coordinate of the centroid coordinate of the first candidate region at the moment after correction, the horizontal coordinate of the centroid coordinate of the first candidate region at the moment after correction, the horizontal coordinate of the centroid coordinate of the first candidate region at the moment after correction, Time of the first The centroid coordinates of each candidate region are corrected to the ordinate. For Euclidean norm operations. For calculation When the minimum value is obtained The value is ultimately output as a set of physical coordinates of the touch points with unique IDs. The improved dual-domain adaptive capacitive touch positioning algorithm first calculates the mutual capacitance value, then performs baseline correction and filtering to extract the effective signal change matrix, then marks the region with signal strength exceeding the dynamic threshold as candidate touch points and calculates the weighted centroid coordinates of each candidate region, then proposes a nonlinear compensation function to solve the problem of asymmetrical electrode distribution in the screen edge region of the touch screen, and finally proposes multi-point tracking and ID association to solve the problem of interaction confusion caused by noise, scanning delay and dense touch points, so as to detect and locate touch points and determine the precise coordinate position of the effective touch point on the screen.

[0042] Preferably, the signal processing unit proposes an improved multimodal touch semantic fusion algorithm for multifunctional signal processing, transforming the original physical position data of the touch points into interactive information with semantic information, stable and reliable, and directly usable by applications. Vibration tests were conducted on the multifunctional display assembly product, and the test results are as follows: Figure 1 , Figure 2 , Figure 3 As shown:

[0043] Specifically, the improved multimodal touch semantic fusion algorithm is as follows: After processing by the touch point detection and positioning unit, the set of physical coordinates of touch points with unique IDs is... After processing by the contact detection and positioning unit, the contact coordinates have been located, therefore The candidate region index was transformed into a contact point index. First, to suppress contact point jitter, Kalman filtering was used to optimize the contact point trajectory to provide a smooth motion trajectory. The state prediction was... ,in, To predict the state vector, Here is the state transition matrix. for The posterior state vector at time t, with predicted covariance is: ,in, To estimate the covariance a priori, for The posterior estimate of the covariance at time t. State transition matrix transpose, Let be the process noise covariance matrix, and let Kalman gain be... ,in, Here is the Kalman gain matrix. For the observation matrix, Observation matrix transpose, To observe the noise covariance matrix, the state is updated as follows: ,in, For posterior state estimation, i.e., the smoothed contact state, the smoothed contact coordinates are: Then, a feature extraction method is proposed to distinguish contact types, and the contact area of ​​the contact points is calculated as follows: ,in, For the first The contact area of ​​each contact point Let be the length of the major axis of the equivalent ellipse of the contact area. Let be the length of the minor axis of the equivalent ellipse of the contact region, and , ,in, and The second-order moment eigenvalues ​​of the signal are used for classification decisions. , ,in, This is a classification decision value used to determine the contact type. For the weight vector of a Support Vector Machine (SVM), for transpose, For feature vectors, For kernel function, For the bias of Support Vector Machine (SVM), The ratio of the major axis to the minor axis. The contact movement speed, The signal variation coefficient is used, and the contact type determination rule is as follows: ,in, For the first Contact type of each contact point , and These are the contact point types: fingertip, pen tip, and palm.

[0044] To improve interaction accuracy, a pressure-sensitive mapping method is proposed to map capacitive signals to pressure levels (upgrading the capacitive signal to the physical pressure space). The pressure-sensitive mapping process is as follows: ,in, For normalized pressure values, This is the original capacitance signal. This represents the maximum background noise value when there is no contact. This marks the dividing point between moderate and moderate pressure. To determine the maximum pressure saturation point, and to address the ambiguity issue at the contact point, a gesture recognition approach is proposed for multi-contact correlation analysis, with trajectory feature extraction as follows: ,in, motion trajectory feature vector of the touch point, horizontal displacement increment of the touch point at time t, vertical displacement increment of the touch point at time t, horizontal displacement increment of the touch point at time t, vertical displacement increment of the touch point at time t, horizontal displacement increment of the touch point at time t, vertical displacement increment of the touch point at time t, wherein, is a standard trajectory sequence (e.g. scaling, rotation) of the predefined gesture template, is the minimum cumulative distance between the current trajectory feature vector and the standard trajectory sequence (the smaller the distance, the higher the matching degree), is a set of normalized paths, is the i-th frame displacement vector of the current trajectory feature vector, is the i-th frame displacement vector of the standard trajectory sequence, is the Euclidean norm operation, and the final output is a multi-dimensional touch attribute information including touch point coordinates, touch point type, pressure level, and gesture semantics. The improved multi-modal touch semantic fusion algorithm first optimizes the touch trajectory through Kalman filtering to provide a smooth motion trajectory, then extracts features of the touch point to distinguish the touch point type, then proposes a pressure mapping to improve the interaction accuracy, and finally proposes a gesture recognition to perform multi-touch correlation analysis, so as to realize multi-functional signal processing, and convert the original touch physical position data into interactive information with semantic information, stability and reliability, which can be directly used by applications.

[0045] Preferably, the touch event recognition and generation module converts the semantic touch information into a standard interactive event that can be parsed by the operating system according to the processing result of the signal processing unit, and realizes the bridge function from physical interaction to application logic.

[0046] Preferably, the system management and control module is used to coordinate hardware resources, optimize algorithm parameters, ensure system robustness, and realize energy efficiency management and control.

[0047] The output and interface module is used to encapsulate the processed high-dimensional interaction data into a standard protocol format, and transmit it to the operating system / application layer in real time through physical / logical channels. ​​​​​​​​​​​​​​

[0048] At the same time, the multifunctional display screen assembly product is subjected to appearance / function inspection, high temperature storage, high temperature work, low temperature storage, low temperature work, constant damp heat test, and the test result data are shown in Tables 2-7.

[0049] Table 2 Test result of appearance / function inspection of multifunctional display screen assembly

[0050]

[0051] Table 3 Test result of high temperature storage of multifunctional display screen assembly

[0052]

[0053] Table 4 Test result of high temperature work of multifunctional display screen assembly

[0054]

[0055] Table 5 Test result of low temperature storage of multifunctional display screen assembly

[0056]

[0057] Table 6 Test result of low temperature work of multifunctional display screen assembly

[0058]

[0059] Table 7 Test result of constant damp heat of multifunctional display screen assembly

[0060]

[0061] The embodiment provides a multifunctional touch screen assembly and signal processing system, which is used for realizing accurate and natural user intention recognition and response, and is characterized in that the multifunctional touch screen assembly and signal processing system is obtained by fusing a touch sensing module, a signal acquisition and transmission module, a signal preprocessing module, a high-level function calculation module, a touch event recognition and generation module, a system management and control module and an output and interface module, a double-domain adaptive capacitive touch positioning algorithm is improved to detect and position a touch point, the existence of an effective touch point and the accurate coordinate position of the touch point on a screen are determined by analyzing a preprocessed signal, the double-domain adaptive capacitive touch positioning algorithm is improved to firstly calculate a mutual capacitance value, then perform baseline correction and filtering to extract an effective signal change matrix, then mark a region with a signal strength exceeding a dynamic threshold value as a candidate touch point and calculate a weighted barycentric coordinate of each candidate region, then a nonlinear compensation function is proposed to solve the problem of asymmetric electrode distribution in the edge region of the touch screen, finally, multi-point tracking and ID association are proposed to solve the problem of interactive confusion caused by noise, scanning delay and dense touch points, so as to detect and position the touch point and determine the accurate coordinate position of the effective touch point on the screen, a multi-modal touch control semantic fusion algorithm is improved to perform multifunctional signal processing, the multi-modal touch control semantic fusion algorithm is improved to firstly perform touch point trajectory optimization by Kalman filtering to provide a smooth motion trajectory, then perform feature extraction on the touch point to distinguish the touch point type, then propose pressure sensing mapping to improve interaction accuracy, and finally propose gesture recognition to perform multi-touch point correlation analysis, so as to realize multifunctional signal processing, convert original touch point physical position data into interactive information with semantic information, stability and reliability, and provide application with direct use, effectively improve the working effect of the multifunctional touch screen assembly and signal processing system, provide more comprehensive and accurate technical support for the multifunctional touch screen assembly and signal processing system, provide better decision support for the scientific and efficient multifunctional touch screen assembly and signal processing system, meanwhile, the application relates to signal processing and human-computer interaction technology, provides an accurate and efficient multifunctional touch screen assembly and signal processing system, and contributes to important application value in the field of signal processing and human-computer interaction technology.

[0062] Although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features, and any modification, equivalent replacement, improvement and the like made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A multi-functional touch screen assembly and signal processing system, comprising: The touch sensing module, the signal acquisition and transmission module, the signal preprocessing module, the advanced function calculation module, the touch event recognition and generation module, the system management and control module, and the output and interface module are included. The touch sensing module is used for detecting and collecting original touch event information. The signal acquisition and transmission module includes a signal acquisition unit and a signal transmission unit. The signal acquisition unit is used for acquiring original signals generated by the sensor module. The signal transmission unit is used for transmitting the original signals to the signal preprocessing module. The signal preprocessing module is used for initial processing of the original signals. The advanced function calculation module includes a touch point detection and positioning unit and a signal processing unit. The touch point detection and positioning unit proposes an improved dual-domain adaptive capacitive touch positioning algorithm for detecting and positioning the touch point. The signal processing unit proposes an improved multi-modal touch control semantic fusion algorithm for multi-functional signal processing. The touch event recognition and generation module is used for converting the information calculated by the bottom layer into touch events that can be understood by the operating system and application programs. The system management and control module is used for coordinating the work of the entire touch system. The output and interface module is used for transmitting the finally generated touch events to the host system. The improved dual-domain adaptive capacitive touch positioning algorithm first calculates mutual capacitance values, then performs baseline correction and filtering to extract an effective signal change matrix, then marks regions with signal strength exceeding a dynamic threshold as candidate touch points and calculates the weighted centroid coordinates of each candidate region, then proposes a non-linear compensation function to solve the problem of asymmetric electrode distribution in the edge region of the touch screen, and finally proposes multi-point tracking and ID association. The improved multi-modal touch control semantic fusion algorithm first optimizes the touch point trajectory through Kalman filtering to provide a smooth motion trajectory, then extracts features of the touch point to distinguish the touch point type, then proposes pressure sensing mapping to improve interaction accuracy, and finally proposes gesture recognition for multi-touch association analysis, so as to realize multi-functional signal processing and convert the original touch point physical position data into interactive information with semantic information for direct use by the application.

2. The multi-functional touch screen assembly and signal processing system according to claim 1, wherein, The touch sensing module detects and collects original touch event information through a special physical sensor array integrated in the screen structure. The touch sensing module constitutes the physical layer of the touch screen assembly, and detects various interactive actions between the user and the screen surface in real time, and converts these physical contact behaviors into original electrical signals for subsequent circuit processing.

3. The multi-functional touch screen assembly and signal processing system of claim 1, wherein, The signal acquisition unit is used for acquiring original analog signals generated by the touch sensing module. The original weak touch signals are first amplified through an analog front-end circuit, and then the analog signals are converted into high-precision digital signals through an analog-to-digital converter. The signal transmission unit is used for transmitting the high-precision digital signals to the subsequent signal preprocessing module. The data converted by the analog-to-digital converter is packaged into data packets in a predetermined format and transmitted in compliance with the communication protocol.

4. The multi-functional touch screen assembly and signal processing system of claim 1, wherein, The signal preprocessing module is used for initial processing of the original signals. High-frequency electromagnetic interference and power supply noise are filtered out through a digital filter. Random noise is suppressed through signal averaging and sliding window smoothing techniques, and the clarity of the effective touch signal is enhanced.

5. The multi-functional touch screen assembly and signal processing system according to claim 1, wherein, The contact detection and positioning unit proposes an improved dual-domain adaptive capacitive touch positioning algorithm to detect and locate the touch point. By analyzing the pre-processed signal, the existence of the effective touch point and its accurate coordinate position on the screen are determined, and the touch durability test of the multifunctional display screen assembly product is carried out.

6. The multi-functional touch screen assembly and signal processing system according to claim 5, wherein, The improved dual-domain adaptive capacitive touch positioning algorithm is as follows: The driving circuit activates each row of transmitting electrodes sequentially in a time-division multiplexing manner. When the first When the row transmitting electrode is activated, the column receiving electrode... Measurement of the emitter electrode using a charge integrating amplifier The mutual capacitance value, the mutual capacitance value ,in, For the first line, number The original capacitance value at the column intersection. This is the baseline capacitance value when there is no touch. The amount of capacitance decay caused by touch, then... Baseline correction and filtering are performed to extract the effective signal change matrix, i.e. ,in, For the first line, number The effective signal strength of the column, To suppress high-frequency noise, the digital filtering operation results in the following effective signal transformation matrix after noise reduction: ,in, For the effective signal change matrix, For the row emission electrode Total number of rows For column receiving electrodes The total number of columns, then scan. Areas where the signal strength exceeds a dynamic threshold are designated as candidate touch points, and the dynamic threshold is set to... ,in, For dynamic detection threshold, The peak attenuation coefficient is... It is a function with maximum value. The background noise figure represents the effective signal strength at each signal point. Perform threshold decision, i.e. ,in, For the binary mask matrix, Perform an 8-neighborhood scan and aggregate adjacent activation points into candidate regions. ,in, For candidate region indexing, the set of candidate regions labeled with connected components is: ,in, This is the first candidate region. This is the second candidate region. For the first Candidate regions, For each candidate region, the total number of candidate regions is [number]. Calculate its weighted centroid coordinates ,Right now , ,in, The x-coordinate of the weighted centroid For the first The physical center coordinates of the column electrodes. For the first line, number The intersection of columns, The ordinate of the weighted centroid is... For the first The physical center coordinates of the horizontal electrodes are used to obtain a preliminary coordinate set. ,in, The centroid coordinates of the first candidate region are given. The centroid coordinates of the second candidate region are... For the first The centroid coordinates of each candidate region; Then, a nonlinear compensation function is proposed to correct the coordinates, namely... , ,in, The weighted centroid abscissa after nonlinear compensation. For compensation coefficient, The x-coordinate of the nearest edge. The weighted centroid ordinate after nonlinear compensation. The ordinate of the nearest edge. The edge decay function, i.e. ,in, This refers to exponentiation operations. The attenuation coefficient is... This refers to the distance from the contact point to the edge. Taking the left edge as an example, then... ,in, Given the absolute coordinates of the left edge of the touchscreen screen's physical boundary in the horizontal direction, the calculation method for the distances from the touchpoint to the right, top, and bottom edges is the same as that for the distance from the touchpoint to the left edge. A multi-point tracking and ID association model is proposed to establish the spatiotemporal continuity of the touchpoint, that is, associating the current frame coordinates with the previous frame coordinates, assigning a unique touchpoint ID, and using nearest neighbor association. ,in, for Time of the first The ID assigned to the centroid coordinates of each candidate region. for Time of the first The centroid coordinates of each candidate region are corrected to the x-coordinate. for Time of the first The centroid coordinates of each candidate region are corrected to the ordinate. for Time of the first The centroid coordinates of each candidate region are corrected to the x-coordinate. for Time of the first The centroid coordinates of each candidate region are corrected to the ordinate. For Euclidean norm operations. For calculation When the minimum value is obtained The value is ultimately output as a set of physical coordinates of the touch points with unique IDs. .

7. The multi-functional touch screen assembly and signal processing system of claim 1, wherein, The signal processing unit proposes an improved multi-modal touch semantic fusion algorithm for multifunctional signal processing, which converts the original touch physical position data into interactive information with semantic information for direct use by applications, and carries out the vibration test of the multifunctional display screen assembly product.

8. The multi-functional touch screen assembly and signal processing system according to claim 7, wherein, The improved multimodal touch semantic fusion algorithm is as follows: After processing by the touch detection and positioning unit, the set of physical coordinates of touch points with unique IDs is... First, to suppress contact jitter, Kalman filtering is used to optimize the contact trajectory to provide a smooth motion trajectory, and the state prediction is... ,in, To predict the state vector, Here is the state transition matrix. for The posterior state vector at time t, with predicted covariance is: ,in, To estimate the covariance a priori, for The posterior estimate of the covariance at time t. State transition matrix transpose, Let be the process noise covariance matrix, and let Kalman gain be... ,in, Here is the Kalman gain matrix. For the observation matrix, Observation matrix transpose, To observe the noise covariance matrix, the state is updated as follows: ,in, For posterior state estimation, i.e., the smoothed contact state, the smoothed contact coordinates are: Then, a feature extraction method is proposed to distinguish contact types, and the contact area of ​​the contact points is calculated as follows: ,in, For the first The contact area of ​​each contact point Let be the length of the major axis of the equivalent ellipse of the contact area. Let be the length of the minor axis of the equivalent ellipse of the contact region, and , ,in, and The second-order moment eigenvalues ​​of the signal are used for classification decisions. , ,in, This is a classification decision value used to determine the contact type. For the weight vector of the support vector machine, for transpose, For feature vectors, For kernel function, For the bias of the support vector machine The ratio of the major axis to the minor axis. The contact movement speed, The signal variation coefficient is CV, and the contact type decision rule is wherein, is the contact type of the th contact, , and are the contact types of the fingertip, the pen tip and the palm, respectively. A pressure mapping is proposed to map the capacitance signal to a pressure level, the pressure mapping process is wherein, is a normalized pressure value, is an original capacitance signal, is a maximum background noise value when no contact, is a medium pressure dividing point, is a maximum pressure saturation point, then to solve the problem of contact ambiguity, a gesture recognition is proposed to perform a multi-contact correlation analysis, the trajectory feature extraction is wherein, is a motion trajectory feature vector of a contact , is a horizontal displacement increment of a contact at time , is a vertical displacement increment of a contact at time , is a horizontal displacement increment of a contact at time , is a vertical displacement increment of a contact at time , the dynamic time warping template matching is wherein, is a standard trajectory sequence of a predefined gesture template , is a minimum cumulative distance of and , is a warping path set, is a frame displacement vector of a current trajectory feature vector , is a frame displacement vector of a standard trajectory sequence , is a Euclidean norm operation, the final output is a multi-dimensional contact attribute information including contact coordinates, contact type, pressure level, gesture semantics.

9. The multi-functional touch screen assembly and signal processing system of claim 1, wherein, The touch event recognition and generation module converts the semantic touch information into standard interactive events that can be parsed by the operating system according to the processing results of the signal processing unit, realizing the bridge function from physical interaction to application logic.

10. The multi-functional touch screen assembly and signal processing system of claim 1, wherein, The system management and control module is used to coordinate hardware resources, optimize algorithm parameters, ensure system robustness and realize energy efficiency management and control. The output and interface module is used to encapsulate the processed high-dimensional interaction data into a standard protocol format and transmit it to the operating system / application layer in real time through physical / logical channels. At the same time, the multifunctional display screen assembly product is subjected to appearance / function inspection, storage work and constant damp heat test.

Images