Electronic whiteboard all-in-one machine based on multi-point touch control technology

Abstract

The invention relates to an electronic whiteboard all-in-one machine based on a multi-point touch control technology, and belongs to the technical field of touch control. The electronic whiteboard all-in-one machine based on the multi-point touch control technology comprises a hardware system and an operating system, the operating system comprises a touch control identification unit, and the touch control identification unit is used for processing touch control data and identifying a touch control type and a corresponding touch control style according to a hidden Markov model; comprising the following steps: S1, data acquisition; s2, model construction and recognition: adopting a layered HMM structure, and decomposing a touch recognition task into two layers for gradual recognition; the first level identifies a touch type; the second level identifies a specific touch pattern; and the recognition capability of the operation system on the complex touch pattern is further improved.

Description

Technical Field

[0001] The present invention belongs to the field of touch technology, and in particular, relates to an all-in-one electronic whiteboard based on multi-touch technology. Background Art

[0002] With the popularity of smart devices, multi-touch technology has been widely used in various interactive devices, such as smart phones, tablet computers, electronic whiteboards and self-service terminals. In these devices, the touch screen has become the main way for users to interact with the device. How to improve the accuracy and flexibility of touch recognition has become the focus of current technology development. However, the existing touch recognition technology still has the following shortcomings when processing complex touch operations (such as multi-finger gestures, pen writing and stroke graphics, etc.):

[0003] Traditional touch recognition methods rely on simple pattern matching algorithms, which can only process a single or a few touch points and cannot effectively recognize complex touch types and patterns.

[0004] In the existing touch recognition system, during the pen writing process, the user experience is often affected by recognition delays or untimely changes in handwriting thickness, resulting in poor operation fluency. Summary of the invention

[0005] In order to solve the technical problems existing in the above background technology that the traditional touch control system lacks the ability to process complex touch control patterns and has low recognition accuracy, the present invention provides an all-in-one electronic whiteboard based on multi-touch control technology.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] An all-in-one electronic whiteboard based on multi-touch technology, the all-in-one electronic whiteboard includes a hardware system and an operating system;

[0008] The hardware system is used to collect, process and store multi-touch data according to multi-touch technology; the hardware system includes a touch screen, a sensor, a processor and a memory;

[0009] The operating system includes a touch recognition unit, which is used to process touch data and identify touch types and their corresponding touch patterns according to a hidden Markov model;

[0010] The implementation method of the touch recognition unit includes the following process:

[0011] S1. Data collection:

[0012] Initialize the touch system and record various touch feature data of all touch points;

[0013] Preprocessing the collected touch feature data, wherein the preprocessing includes data denoising and normalization processing;

[0014] Preferably, the touch feature data includes combined data of two or more of the touch coordinates of the touch point, the touch pressure, the touch area, the duration of the touch feature or the touch movement speed.

[0015] S2. Model construction and identification:

[0016] A hierarchical HMM structure is used, and the touch recognition task is decomposed into two layers of step-by-step recognition; the first layer recognizes the touch type, including finger touch or pen touch; the second layer recognizes the specific touch style, including specific gestures or geometric strokes;

[0017] Preferably, the finger touch includes specific gestures, specifically single-finger click, two-finger slide, two-finger zoom or long press;

[0018] Preferably, the pen touch comprises a geometric stroke, specifically drawing a straight line, drawing a polyline, drawing a circle, drawing a rectangle or drawing a triangle;

[0019] S21. At the first level, a hidden Markov model (HMM) is constructed to combine various touch feature data at multiple moments into a feature vector, expressed as:

[0020] O t =[x t ,y t , a t , p t ];

[0021] In the formula, O t : The observation vector generated at time point t.

[0022] x t ,y t : Coordinates of the touch point.

[0023] a t : Touch area.

[0024] p t : Touch pressure.

[0025] S22. During the touch control at multiple moments, the observation sequence of the touch control features can be expressed as:

[0026] O=[O1,O2,...,O t ];

[0027] Wherein, O represents an observation sequence of a touch type, including touch feature data at multiple moments;

[0028] S23. After the model is initialized, the observation sequence O is input into the model as training data of the hidden Markov model;

[0029] S24. Use the Baum-Welch algorithm to train the model to maximize the probability of the observation sequence; use the Viterbi algorithm to find the hidden state with the maximum probability. The hidden state with the maximum matching probability is the touch type of the observation sequence;

[0030] S25. At the second level, a semi-hidden Markov model (SHMM) is constructed, and the observation vector O generated at time point t t The duration and touch type of the touch feature are introduced to form a new observation vector, which is expressed as:

[0031] O t '=[x t ,y t , a t , p t , Q t , T t ];

[0032] In the formula, O t ': The new observation vector generated at time point t;

[0033] Q t : Touch type; (finger touch or pen touch)

[0034] T t : The duration of the touch feature;

[0035] S26. Then, during the touch control at multiple moments, the observation sequence of the touch control pattern can be expressed as:

[0036] O'=[O1',O2',...,O t ']

[0037] Wherein, O' is an observation sequence of a touch style, including the data of the touch features and duration of the touch type;

[0038] S27. Input the new observation sequence O' into the model as training data of the semi-hidden Markov model;

[0039] S28. Use the Baum-We1ch algorithm to train the model to maximize the probability of the observation sequence; use the Viterbi algorithm to find the hidden state with the maximum probability. The hidden state with the maximum matching probability is the touch pattern of the time observation sequence.

[0040] Preferably, before process S27, the method further includes performing dynamic time warping (DTW) processing on the new observation sequence; using DTW to align sequences of different lengths so that the semi-hidden Markov model can process different touch durations in the time dimension. The specific implementation process is as follows:

[0041] 1) Set the lengths of the two a ′ and O b The observation sequence of ′ is recorded as:

[0042] O a ′=[O a ′(1),O a ′(2), ..., O a ′(t)];

[0043] O b ′=[O b ′(1),O b ′(2), ..., O b ′(t)];

[0044] Where (t) represents each time point of the observation sequence.

[0045] 2) Use the Euclidean distance method to calculate the distance between each time point: that is, the time point O in the first sequence a '(i) and time point O in the second sequence b The distance between ′(j) is d(O a ′(i), O b ′(j)).

[0046] 3) Construct the cumulative distance matrix {D(i, j)} to record the sequence O a ′ and O b The minimum cumulative distance of the alignment path at each time point of ′: The calculation formula is:

[0047] D(i, j) = d(O a ′(i), O b ′(j))+min[D(i-1,j),D(i,j-1),D(i-1,j-1)];

[0048] Where D(i, j) is the minimum cumulative distance when positions i and j are aligned.

[0049] 4) Starting from the lower right corner of the matrix {D(i, j)}, backtrack to find the optimal alignment path q = {D(i, j) ...}, which corresponds to the optimal match between the time points in the two sequences;

[0050] During the backtracking process, the next time point to be aligned is selected according to min[D(i-1, j), D(i, j-1), D(i-1, j-1)] until D(1, 1) is reached.

[0051] 5) Through the optimal alignment path, the observation sequence O a ′ and O b ′ are aligned in the time dimension, even if they are of different lengths; for shorter sequences, their time points may be aligned with time points in the longer sequence multiple times, thereby normalizing them in the time dimension.

[0052] Preferably, it also includes S3. establishing a mapping relationship:

[0053] A mapping relationship table is established to map the touch styles to corresponding operation commands, including mapping and associating the touch styles with specific operation commands one by one.

[0054] Preferably, the mapping relationship includes: mapping a "two-finger slide" gesture to a "page move" command;

[0055] The "two-finger zoom" gesture is mapped to the "zoom" command to adjust the scale of the displayed content;

[0056] The "long press" gesture is mapped to the "select" command.

[0057] The "click left or right edge of the page" gesture is mapped to the "turn page left or right" command.

[0058] The "swipe on screen" gesture is mapped to the "exit" command.

[0059] Preferably, the operating system further comprises a pressure sensing unit, which is used to realize adaptive adjustment of the thickness of handwriting according to the pressure sensing data of the stylus. The specific implementation steps include:

[0060] Step 1) obtaining real-time pressure data of the stylus from the touch screen hardware system;

[0061] Step 2) De-noise the collected pressure data; use exponential smoothing method to smooth the pressure data, the formula is as follows:

[0062] Ps(t)=α·P(t)+(1-α)·Ps(t-1);

[0063] Where: Ps( t ) is the smoothed pressure value at time t;

[0064] P( t ) is the measured pressure value at time t;

[0065] α is the smoothing factor (ranging from 0 to 1), which controls the smoothness;

[0066] After smoothing, the generated smoothed pressure value ensures the stability of pressure changes and reduces fluctuations caused by noise.

[0067] Step 3) Set the linear relationship between the handwriting thickness and the smoothing pressure value, expressed as a function:

[0068] W(t)=k·Ps(t)+b

[0069] In the formula, W( t ) is the handwriting thickness value, k and b are constants; through this linear function, the pressure value of the stylus directly affects the handwriting thickness, and the greater the pressure, the thicker the handwriting.

[0070] Step 4) According to the linear relationship between the pressure value and the handwriting thickness, the handwriting thickness value is calculated and updated in real time to ensure that the handwriting thickness of the user is adjusted instantly as the pressure changes during the handwriting process.

[0071] Preferably, the pressure sensing unit further comprises step 5) abnormality processing: determining whether the current pressure data fluctuates abnormally; the system will suspend the adjustment of the handwriting thickness value and recalibrate the pressure sensing unit;

[0072] Monitor the smoothness of the handwriting thickness adjustment process. If there is any lag or delay, optimize the calculation process or adjust the smoothing parameters.

[0073] Beneficial effects of the present invention:

[0074] 1. The present invention collects touch data in real time based on multi-touch technology and combines a hierarchical hidden Markov model architecture to process touch data, identify touch types and touch patterns, and complete a series of touch analyses, making touch recognition more accurate and flexible.

[0075] 2. The present invention uses the hierarchical processing structure of HMM and SHMM to decompose and refine complex touch recognition tasks layer by layer, making touch recognition more accurate and flexible. HMM has a high accuracy in touch type recognition, and SHMM further improves the recognition ability of complex touch patterns by introducing feature duration and touch type of the previous level recognition result.

[0076] 3. The pressure sensing unit is combined with the exponential smoothing algorithm to ensure the stability of pressure perception, adjust the thickness of the handwriting in real time, and improve the fluency and accuracy of the user during the writing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0078] Figure 1 The flowchart of the first-level model construction and recognition in the electronic whiteboard integrated machine based on multi-touch technology of the present invention.

[0079] Figure 2 The flowchart of the second-level model construction and recognition in the electronic whiteboard integrated machine based on the multi-touch technology of the present invention. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] See also Figure 1-Figure 2 As shown, the electronic whiteboard all-in-one machine based on multi-touch technology includes a hardware system and an operating system; a combination of multi-touch technology and program code is used to realize multi-user interaction and intelligent touch style recognition.

[0082] The hardware system is used to collect, process and store multi-touch data according to multi-touch technology; the hardware system includes a touch screen, a sensor, a processor and a memory;

[0083] Specifically, the touch screen is used to detect the touch points and movement trajectories of fingers, styluses or other tools. Sensors are responsible for collecting multi-touch signals. Processors and memory are used to process touch data and execute operation commands, and support the storage and processing of related data.

[0084] The operating system includes a touch recognition unit, which is used to process touch data and recognize touch types and corresponding touch patterns according to a hidden Markov model;

[0085] The implementation method of the touch recognition unit includes the following process:

[0086] S1. Data collection:

[0087] Initialize the touch system and record various touch feature data of all touch points;

[0088] Preprocessing the collected touch feature data, wherein the preprocessing includes data denoising and normalization processing;

[0089] It should be noted that noise data (such as false touches) are removed and all feature data are normalized to make the value ranges of different features consistent, thereby reducing the model error caused by data differences.

[0090] Furthermore, the touch feature data includes combined data of two or more of the touch coordinates of the touch point, the touch pressure, the touch area, the duration of the touch feature, or the touch movement speed.

[0091] Specifically, the touch feature data provides input for subsequent touch type recognition and touch pattern recognition, wherein:

[0092] Touch coordinates: record the X and Y coordinates of the touch point; can directly represent the user's interaction position on the screen, helping to distinguish different touch behaviors, such as click location, sliding direction, or drawing a circle.

[0093] Touch Pressure: Gets touch pressure data at each point in time from the touch device. Pressure data is particularly important in distinguishing between finger touch and pen touch, because the pressure value of a pen is usually higher and more stable, while the pressure value of a finger is the opposite.

[0094] Touch area extraction: Record the size of the touch area at each time point; the size of the touch area can be used as an important feature to distinguish between fingers and pens. Finger touch usually presents a larger contact area, while the contact area of ​​the pen is relatively fixed and small.

[0095] Touch feature duration: Captures the duration of a touch feature and uses it as one of the features. The duration of a touch feature can be used to identify specific gestures or geometric strokes, such as a short touch for clicking, a long press for activating certain functions, and drawing geometric shapes such as circles and rectangles to activate the shape correction function.

[0096] Touch type: mainly divided into finger touch or pen touch; the touch type is expressed as the recognition result of the first-level model and serves as the input feature of the second-level model; it is used to distinguish between specific finger gestures and pen strokes.

[0097] S2. Model construction and identification:

[0098] Adopting a hierarchical HMM structure, complex touch recognition tasks are decomposed and refined layer by layer; each layer can handle specific touch recognition tasks, and by using the processing results of the previous layer as the input of the next layer, the touch recognition task is decomposed into two layers of step-by-step recognition: the first layer recognizes the touch type, including finger touch or pen touch; the second layer recognizes the specific touch style, including specific gestures or geometric strokes;

[0099] Model building and recognition use different HMM structures at different levels; Hidden Markov model processing is used at the touch type recognition level; Semi-Hidden Markov model processing is used at the touch style recognition level;

[0100] Further, the finger touch includes specific gestures, specifically single-finger click, two-finger slide, two-finger zoom or long press;

[0101] The pen touch control includes geometric strokes, specifically drawing a straight line, drawing a polyline, drawing a circle, drawing a rectangle or drawing a triangle;

[0102] S21. At the first level, a hidden Markov model (HMM) is constructed to combine various touch feature data at multiple moments into a feature vector, expressed as:

[0103] O t =[x t ,y t , a t , p t ];

[0104] In the formula, O t : The observation vector generated at time point t.

[0105] x t ,y t : Coordinates of the touch point.

[0106] a t : Touch area.

[0107] p t : Touch pressure.

[0108] S22. During the touch control at multiple moments, the observation sequence of the touch control features can be expressed as:

[0109] O=[O1,O2,...,O t ];

[0110] Wherein, O represents an observation sequence of a touch type, including touch feature data at multiple moments;

[0111] S23. After the model is initialized, the observation sequence O is input into the model as training data of the hidden Markov model;

[0112] S24. Use the Baum-Welch algorithm to train the model to maximize the probability of the observation sequence; use the Viterbi algorithm to find the hidden state with the maximum probability. The hidden state with the maximum matching probability is the touch type of the observation sequence;

[0113] Specifically, the Hidden Markov Model (HMM) initialization process includes:

[0114] The probability of each touch type being in the initial state is set; for example, the initial probabilities of finger touch and pen touch can be determined based on training data.

[0115] Based on historical data, define the transition probabilities between different touch types, for example, the probability of transitioning from finger touch to pen touch, or vice versa.

[0116] Defines the probability of generating a certain observation sequence under each touch type. It can be estimated through training data, such as the difference between the pressure and area distribution of finger touch and pen touch.

[0117] The Baum-Welch algorithm is used to train the HMM model. By comparing the current state of the model with the degree of fit between the observation sequence and the model parameters, the model parameters are repeatedly adjusted to maximize the probability of the observation sequence.

[0118] Next, the Viterbi algorithm is used to solve the hidden state with the highest probability of the HMM model. This is well documented in existing public literature and will not be described in detail here.

[0119] It is worth noting that the important concept and innovation of the present invention is that touch recognition is a complex task, which not only involves distinguishing touch types (such as finger and pen touch), but also requires identifying specific touch styles (such as different gestures or graphic strokes). By decomposing the task into two layers of processing, the hierarchical HMM structure can efficiently cope with this complexity.

[0120] The hierarchical HMM structure of the present invention enables the system to perform rough classification first and then gradually process complex tasks in depth;

[0121] At the first level, the system can quickly and accurately identify the touch type, thereby avoiding mixing finger touch and pen touch data in subsequent style recognition and reducing the impact of noise.

[0122] Then in the subsequent process, in the second layer, since the first layer has filtered the touch type, the system can focus on more complex gesture or graphic recognition, and use the SHMM model to effectively process the time duration of the gesture, thereby improving the accuracy of gesture recognition.

[0123] S25. At the second level, a semi-hidden Markov model (SHMM) is constructed, and the observation vector O generated at time point t t The duration and touch type of the touch feature are introduced to form a new observation vector, which is expressed as:

[0124] O t '=[x t ,y t , a t , p t, Q t , T t ];

[0125] In the formula, O t ': The new observation vector generated at time point t;

[0126] Q t : Touch type; (finger touch or pen touch)

[0127] T t : The duration of the touch feature;

[0128] S26. Then, during the touch control at multiple moments, the observation sequence of the touch control pattern can be expressed as:

[0129] O'=[O1',O2',...,O t ']

[0130] Wherein, O' is an observation sequence of a touch style, including the data of the touch features and duration of the touch type;

[0131] S27. Input the new observation sequence O' into the model as training data of the semi-hidden Markov model;

[0132] S28. Use the Baum-Welch algorithm to train the model to maximize the probability of the observation sequence; use the Viterbi algorithm to find the hidden state with the maximum probability. The hidden state with the maximum matching probability is the touch pattern of the time observation sequence.

[0133] In this specific implementation, the semi-hidden Markov model introduces the touch type and the duration of the touch feature into the observation vector to form a new observation sequence. This extension enables the SHMM to more accurately describe the dynamic characteristics of the touch style, especially for scenarios where the duration is critical to gesture recognition, such as recognition scenarios of specific gestures (long press, two-finger slide, two-finger zoom) and geometric strokes (drawing a straight line, drawing a polyline, drawing a circle, drawing a rectangle).

[0134] The hidden Markov model can be expressed as (N, M, π, A, B). N is the number of states of the Markov chain. The larger the number of states, the more states there are, and the better it can describe various touch feature data;

[0135] However, for specific gestures and geometric strokes, SHMM not only considers the sequence changes of hidden states, but also can handle the duration distribution under different states. SHMM can better handle long-term specific gestures and geometric strokes (such as pinch-to-zoom, circle drawing, etc.)

[0136] Furthermore, before process S27, the method further includes performing dynamic time warping (DTW) processing on the new observation sequence; using DTW to align sequences of different lengths so that the semi-hidden Markov model can process different touch durations in the time dimension.

[0137] Specifically, the dynamic time warping (DTW) method is used to align the new observation sequence so that observation sequences of different lengths can be compared and processed in the time dimension; the specific implementation process is as follows:

[0138] 1) Set the lengths of the two a ′ and O b The observation sequence of ′ is recorded as:

[0139] O a ′=[O a ′(1),O a ′(2), ..., O a ′(t)];

[0140] O b ′=[O b ′(1),O b ′(2), ..., O b ′(t)];

[0141] Where (t) represents each time point of the observation sequence.

[0142] 2) Use the Euclidean distance method to calculate the distance between each time point: that is, the time point O in the first sequence a '(i) and time point O in the second sequence b The distance between ′(j) is d(O a ′(i), O b ′(j)).

[0143] 3) Construct the cumulative distance matrix {D(i, j)} to record the sequence O a ′ and O b The minimum cumulative distance of the alignment path at each time point of ′: The calculation formula is:

[0144] D(i, j) = d(O a ′(i), O b ′(j))+min[D(i-1,j),D(i,j-1),D(i-1,j-1)];

[0145] Where D(i, j) is the minimum cumulative distance when positions i and j are aligned.

[0146] 4) Starting from the lower right corner of the matrix {D(i, j)}, backtrack to find the optimal alignment path q = {D(i, j) ...}, which corresponds to the optimal match between the time points in the two sequences;

[0147] During the backtracking process, the next time point to be aligned is selected according to min[D(i-1, j), D(i, j-1), D(i-1, j-1)] until D(1, 1) is reached.

[0148] 5) Through the optimal alignment path, the observation sequence O a ′ and O b ′ are aligned in the time dimension, even if they are of different lengths; for shorter sequences, their time points may be aligned with time points in the longer sequence multiple times, thereby normalizing them in the time dimension.

[0149] Furthermore, S3. establishes a mapping relationship:

[0150] A mapping relationship table is established to map the touch styles to corresponding operation commands, including one-to-one mapping association between touch styles (such as sliding, pinch-to-zoom, clicking, etc.) and specific operation commands (such as page turning, zooming, confirming).

[0151] Specifically, the mapping relationship is as follows:

[0152] The "two-finger slide" gesture is mapped to the "page move" command;

[0153] The "two-finger zoom" gesture is mapped to the "zoom" command to adjust the scale of the displayed content;

[0154] The "long press" gesture is mapped to the "select" command.

[0155] The "click left or right edge of the page" gesture is mapped to the "turn page left or right" command.

[0156] The "swipe on screen" gesture is mapped to the "exit" command.

[0157] Once the command is determined, the system needs to execute the corresponding operation in real time.

[0158] Furthermore, the operating system further includes a pressure sensing unit, which is used to realize adaptive adjustment of the thickness of handwriting according to the pressure sensing data of the stylus. The specific implementation steps include:

[0159] Step 1) obtaining the real-time pressure data of the stylus from the touch screen hardware system; the data reflects the pressure value of the stylus of the user at the current time t.

[0160] Step 2) De-noising the collected pressure data to reduce the noise caused by the user's hand shaking or other factors; the pressure data is smoothed using the exponential smoothing method, and the formula is as follows:

[0161] Ps(t)=α·P(t)+(1-α)·Ps(t-1);

[0162] Where: Ps(t) is the smoothed pressure value at time t;

[0163] P(t) is the measured pressure value at time t;

[0164] α is the smoothing factor (ranging from 0 to 1), which controls the smoothness;

[0165] After smoothing, the generated smoothed pressure value ensures the stability of pressure changes and reduces fluctuations caused by noise.

[0166] Step 3) Set the linear relationship between the handwriting thickness and the smoothing pressure value, expressed as a function:

[0167] W(t)=k·Ps(t)+b

[0168] Where W(t) is the handwriting thickness value, k and b are constants; through this linear function, the pressure value of the stylus directly affects the handwriting thickness. The greater the pressure, the thicker the handwriting, and vice versa.

[0169] Step 4) continuously acquiring the real-time pressure data of the stylus, smoothing it, and dynamically updating the smoothed pressure value;

[0170] Based on the linear relationship between pressure value and handwriting thickness, the handwriting thickness value is calculated and updated in real time to ensure that the handwriting thickness is adjusted instantly as the pressure changes during the handwriting process.

[0171] Furthermore, the pressure sensing unit further includes step 5) abnormality processing: determining whether the current pressure data fluctuates abnormally; the system will suspend the adjustment of the handwriting thickness value and recalibrate the pressure sensing unit;

[0172] Monitor the smoothness of the handwriting thickness adjustment process. If there is any lag or delay, optimize the calculation process or adjust the smoothing parameters.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0174] In the several embodiments provided in the present application, it should be understood that the disclosed systems, units and processes can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0175] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0176] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

[0177] In the description of the specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0178] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. An all-in-one electronic whiteboard based on multi-touch technology, the all-in-one electronic whiteboard includes a hardware system and an operating system, and is characterized by: The hardware system is used to collect, process and store multi-touch data according to multi-touch technology; the hardware system includes a touch screen, a sensor, a processor and a memory; The operating system includes a touch recognition unit, which is used to process touch data and identify touch types and their corresponding touch patterns; The implementation method of the touch recognition unit includes the following process: S1. Data collection: Initialize the touch system and collect various touch feature data of all touch points; S2. Model construction and identification: A hierarchical HMM structure is used, and the touch recognition task is decomposed into two layers of step-by-step recognition; the first layer recognizes the touch type, including finger touch or pen touch; the second layer recognizes the specific touch style, including specific gestures or geometric strokes; S21. At the first level, a hidden Markov model is constructed to combine various touch feature data at multiple moments into a feature vector, expressed as: EITHER t =[x t ,and t ,to t ,p t ]; In the formula, O t : The observation vector generated at time point t; x t ,y t : coordinates of the touch point; a t : Touch area; p t : Touch pressure; S22. Then, during the touch control at multiple moments, the observation sequence of the touch control features is expressed as: O=[O1,O2,...,O t ]; Wherein, O represents an observation sequence of a touch type, including touch feature data at multiple moments; S23. After the model is initialized, the observation sequence O is input into the model as training data of the hidden Markov model; S24. Use the Baum-Welch algorithm to train the model to maximize the probability of the observation sequence; use the Viterbi algorithm to find the hidden state with the maximum probability, and the hidden state with the maximum matching probability is the touch type of the observation sequence; S25. At the second level, a semi-hidden Markov model is constructed, and the observation vector O generated at time point t t The duration and touch type of the touch feature are introduced to form a new observation vector, which is expressed as: O t ’=[x t ,y t ,a t ,p t ,Q t ,T t ]; In the formula, O t ': The new observation vector generated at time point t; Q t : touch type; T t : The duration of the touch feature; S26. Then, during the touch control at multiple moments, the observation sequence of the touch control pattern is expressed as: O'=[O1',O2',...,O t ']; Wherein, O' is an observation sequence of a touch style, including the data of the touch features and duration of the touch type; S27. Input the new observation sequence O' into the model as training data of the semi-hidden Markov model; S28. Use the Baum-Welch algorithm to train the model to maximize the probability of the observation sequence; use the Viterbi algorithm to find the hidden state with the maximum probability. The hidden state with the maximum matching probability is the touch pattern of the time observation sequence.

2. The electronic whiteboard integrated machine based on multi-touch technology according to claim 1, characterized in that: In process S1, the collected touch feature data is preprocessed, and the preprocessing includes data denoising and normalization processing.

3. The electronic whiteboard integrated machine based on multi-touch technology according to claim 1, characterized in that: In process S1, the touch feature data includes touch coordinates of a touch point, touch pressure, touch area, duration of a touch feature, or combination data of two or more of touch movement speed.

4. The electronic whiteboard integrated machine based on multi-touch technology according to claim 1, characterized in that: In process S2, the finger touch includes specific gestures, specifically single-finger click, two-finger slide, two-finger zoom or long press.

5. The electronic whiteboard integrated machine based on multi-touch technology according to claim 1, characterized in that: In process S2, the pen touch includes geometric strokes, specifically drawing a straight line, drawing a polyline, drawing a circle, drawing a rectangle or drawing a triangle.

6. The electronic whiteboard integrated machine based on multi-touch technology according to claim 1, characterized in that: It also includes, before process S27, performing dynamic time warping on the new observation sequence; and aligning sequences of different lengths using the DTW method.

7. The electronic whiteboard integrated machine based on multi-touch technology according to claim 1, characterized in that: Also includes S3. Establish mapping relationship: Mapping the touch style to the corresponding operation command, including mapping and associating the touch style with the specific operation command one by one.

8. The electronic whiteboard integrated machine based on multi-touch technology according to claim 1, characterized in that: The operating system further includes a pressure sensing unit, which is used to realize adaptive adjustment of the thickness of handwriting according to the pressure sensing data of the stylus. The specific implementation steps include: Step 1) obtaining real-time pressure data of the stylus from the touch screen hardware system; Step 2) De-noise the collected pressure data; use exponential smoothing method to smooth the pressure data, the formula is as follows: Ps(t)=α·P(t)+(1-α)·Ps(t-1); Where: Ps(t) is the smoothed pressure value at time t; P(t) is the measured pressure value at time t; α is the smoothing factor, ranging from 0 to 1; Step 3) Set the linear relationship between the handwriting thickness and the smoothing pressure value, expressed as a function: W(t)=k·Ps(t)+b Where W(t) is the handwriting thickness value, k and b are constants; Step 4) According to the linear relationship between the pressure value and the handwriting thickness, the handwriting thickness value is calculated and updated in real time.

9. The electronic whiteboard integrated machine based on multi-touch technology according to claim 8, characterized in that: The pressure sensing unit further includes step 5) abnormality handling: Determine whether the current pressure data fluctuates abnormally; the system will suspend the adjustment of the handwriting thickness value and recalibrate the pressure sensing unit; Monitor the smoothness of the handwriting thickness adjustment process. If there is any lag or delay, optimize the calculation process or adjust the smoothing parameters.

Images