A human-computer interaction method and interactive tablet and storage medium
By using infrared and vibration detection devices in the interactive tablet, the speakers and vibration signals are processed, and the precise detection of touch objects is achieved, which solves the problem of zero writing height of the optical touch system when writing, and improves the touch operation Accuracy and writing quality.
Patent Information
- Application Number
- CN202110518817.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-12
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-05-12
AI Technical Summary
Optical touch systems have poor performance in interactive tablets and cannot solve the problem of zero writing height, resulting in unclear handwriting and frequent continuous writing during writing.
An infrared detection device and vibration detection device are used to send audio signals through speakers. The vibration detection device collects and processes vibration signals, removes speaker vibration interference, and determines whether the touch object touches the display screen to achieve more accurate touch response.
It improves the flexibility and accuracy of touch operation, solves the problem of zero writing height, and makes the handwriting more neat when writing, reducing the phenomenon of continuous writing.
Smart Images

Figure CN113608661B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of touch control, and in particular to a human-computer interaction method and an interactive tablet and storage medium thereof. Background Art
[0002] The optical touch system is one of the touch methods used by interactive tablets. Figure 1A As shown, in an optical touch system, the optical touch sensors 120 are generally arranged on both sides of the surface of the display screen 110. The light signal transmitted by the optical touch sensor 120 propagates on the surface of the display screen 110. By analyzing the situation where the touch object (for example, a hand or a pen) blocks the light signal, the position of the touch object on the surface of the display screen 110 is calculated, and touch point data is generated. The touch data is then further processed by the interactive tablet. These optical touch methods have poor performance when applied to business. For example, when applied to writing business, the problem of zero writing height cannot be solved. Summary of the invention
[0003] The embodiment of the present invention provides a human-computer interaction method and an interactive tablet and a storage medium thereof to solve the problem of poor performance of an optical touch system in an interactive tablet.
[0004] In a first aspect, an embodiment of the present invention provides a human-computer interaction method, which is applied to an interactive tablet, wherein the interactive tablet includes an infrared detection device, a speaker, and a vibration detection device, and the method includes:
[0005] The processor of the interactive tablet sends an audio signal to the speaker, so that the speaker is in a vibrating state;
[0006] The processor of the vibration detection device synchronously collects the audio signal received by the speaker;
[0007] The display screen of the interactive flat panel receives touch of a touch object;
[0008] The sensor of the vibration detection device detects a first vibration signal transmitted from the display screen of the interactive flat panel, wherein the first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration generated by the touch object touching the display screen;
[0009] The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, and obtains a second vibration signal;
[0010] Determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object;
[0011] If the conditions are met, the interactive tablet responds to the touch of the touch object.
[0012] In a second aspect, an embodiment of the present invention further provides an interactive tablet, the interactive tablet comprising an infrared detection device, a speaker and a vibration detection device;
[0013] The processor of the interactive tablet is used to send an audio signal to the speaker, so that the speaker is in a vibrating state;
[0014] The processor of the vibration detection device is used to synchronously collect the audio signal received by the speaker;
[0015] The display screen of the interactive tablet is used to receive touches from a touch object;
[0016] The sensor of the vibration detection device is used to detect a first vibration signal transmitted from the display screen of the interactive flat panel, wherein the first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration generated by the touch object touching the display screen;
[0017] The processor of the vibration detection device is further used to process the first vibration signal, remove the vibration signal generated by the vibration of the speaker, and obtain a second vibration signal;
[0018] Determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object;
[0019] The interactive tablet is used to respond to the touch of the touch object when the condition is met.
[0020] In a third aspect, an embodiment of the present invention further provides an interactive tablet, the interactive tablet comprising:
[0021] one or more processors;
[0022] a memory for storing one or more programs,
[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the human-computer interaction method as described in the first aspect.
[0024] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the human-computer interaction method as described in the first aspect is implemented.
[0025] In this embodiment, the processor of the interactive tablet sends an audio signal to the speaker, so that the speaker is in a vibrating state. The processor of the vibration detection device synchronously collects the audio signal received by the speaker. The display screen of the interactive tablet receives the touch of the touch object. The sensor of the vibration detection device detects a first vibration signal transmitted from the display screen of the interactive tablet. The first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration caused by the touch object touching the display screen. The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, obtains a second vibration signal, and determines whether the second vibration signal meets the condition of triggering the interactive tablet to respond to the touch of the touch object. If so, the interactive tablet responds to the touch of the touch object. When the touch object triggers the touch operation, the touch operation can be given a more dimensional feature to enrich the meaning of the touch operation. In some scenarios, if distinguishing whether the touch object touches the surface of the display screen has a more positive meaning, then this embodiment distinguishes the way of responding to the touch operation on the basis of detecting whether the touch object touches the surface of the display screen, adapts to the needs of the scene, and thus improves the performance of the touch operation in the scene. In the writing scene, the problem of zero-height writing can be solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1A This is an example diagram of an existing touch signal and its reporting point;
[0027] Figure 1B The following is an example diagram of an existing handwriting;
[0028] Figure 2A A flowchart of a human-computer interaction method for an interactive tablet provided in Embodiment 1 of the present invention;
[0029] Figure 2B A schematic diagram of a touch detection scenario provided in the first embodiment of the present invention;
[0030] FIG. 3A to FIG. 3D An example diagram of handwriting provided in the first embodiment of the present invention;
[0031] Figure 4 is a flow chart of a human-computer interaction method of an interactive tablet provided in Embodiment 2 of the present invention;
[0032] Figure 5 is a flow chart of a human-computer interaction method of an interactive tablet provided by Embodiment 3 of the present invention;
[0033] Figure 6 A schematic diagram of the structure of an interactive tablet provided in Embodiment 4 of the present invention;
[0034] Figure 7 This is a schematic diagram of the structure of an interactive tablet provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION
[0035] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.
[0036] There has been a long-standing technical problem in the industry for realizing the touch function of interactive tablets through optical touch sensors, namely the problem of zero writing height. The so-called zero writing height means that when a touch object (such as a hand or a pen) writes on the display screen, the writing handwriting is generated only when the touch object touches the surface of the display screen, and the distance between the touch object and the surface of the display screen when writing is zero.
[0037] To better understand zero writing height, combined with Figure 1A For further explanation, the optical touch sensor 120 generally protrudes from the surface of the display screen 110, and there is a certain height range when emitting or receiving light signals, thereby forming a touch area in the vertical direction, and its height is H. When the touch object is pressed or lifted, there may be a situation where the touch object is located in the touch area but does not touch the surface of the display screen 110. At this time, the touch point data will still be reported and the corresponding operation will be performed, such as displaying the handwriting of the touch object.
[0038] From state 131, state 132 to state 133, the stages of the touch object being pressed are described. In state 131, the touch object is above the touch area, and no touch signal is generated and no touch point data is reported. In state 132, the touch object is in the touch area and close to the surface of the display screen 110 but does not touch the surface of the display screen 110, and a touch signal is generated and touch point data is reported. In state 133, the touch object is in the touch area and has touched the display screen 110, and a touch signal is generated and touch point data is reported.
[0039] State 133, state 134 to state 135 describe the stage of the touch object being lifted. In state 134, the touch object is in the touch area, close to the surface of the display screen 110 but does not touch the display screen 110, and a touch signal is generated and touch point data is reported. In state 135, the touch object is above the touch area, and no touch signal is generated and no touch point data is reported.
[0040] That is, when the touch function of the interactive tablet is realized through the optical touch sensor, there will be a writing height, and zero writing height cannot be achieved.
[0041] Since most of the touch devices commonly used by users in daily life are mobile phones and tablets, and these touch devices mostly use resistive or capacitive touch operations, users are more accustomed to using resistive or capacitive touch operations, that is, the touch point data is reported and corresponding operations are performed only when the display screen is touched.
[0042] When the user triggers an optical touch operation while maintaining the habit of using capacitive touch operations, the performance is poor in some business scenarios.
[0043] Taking handwriting as an example, when the user writes handwriting on the display screen, when the finger is within the touch area of the optical touch sensor but does not touch the display screen, the handwriting is displayed. However, at this time, the user may be starting or ending a stroke, not intending to write. Displaying the starting and ending stroke handwriting will cause connected strokes, and the written font will appear rather scribbled.
[0044] For example, as Figure 1B shown, if the user writes the character "正", there are continuous starting and ending strokes 141 between writing the first horizontal stroke and the first vertical stroke, a starting stroke 142 at the end of writing the second horizontal stroke, and continuous starting and ending strokes 143 between writing the second vertical stroke and the third horizontal stroke, all of which have relatively obvious connected strokes.
[0045] Currently, there are mainly two solutions to this technical problem:
[0046] First, reduce the writing height as much as possible. By reducing the height of the optical touch sensor protruding from the surface of the display screen, the writing height can be reduced.
[0047] Second, by setting a piezoelectric sensor in the touch pen, when the touch pen touches the surface of the display screen, the touch pen will generate pressure data, thus achieving zero writing height.
[0048] However, for the first method, even if the position of the optical touch sensor is extremely low, it must still protrude from the surface of the display screen, so there must be a height of the light signal, and thus zero writing height still cannot be achieved. Moreover, this method requires the optical touch sensor to be buried in the interactive flat panel, and strict control is required for the structure of the interactive flat panel (such as the cover glass and back panel of the display screen) to avoid the display screen and the cover glass protruding outward to block the light of the optical touch sensor, resulting in relatively high costs and control difficulties.
[0049] For the second method, when the touch object is not a touch pen, such as when the touch object is a hand, zero writing height still cannot be achieved. Moreover, the connection between the touch pen and the interactive flat panel occupies the limited port resources of the interactive flat panel, affecting operations such as screen sharing and screen mirroring.
[0050] Embodiment 1
[0051] Figure 2AThis is a flowchart of a human-computer interaction method provided in Embodiment 1 of the present invention. This embodiment can be applied to a situation where a touch object touches a display screen according to a vibration signal in an interactive tablet, and a response is made. The method can be executed by an interactive tablet, and the interactive tablet can be implemented by software and / or hardware, including components such as an infrared detection device, a speaker, and a vibration detection device, and specifically includes the following steps:
[0052] Step 201: The processor of the interactive tablet sends an audio signal to the speaker, so that the speaker is in a vibrating state.
[0053] The interactive tablet is equipped with a processor, and the processor of the interactive tablet is a high-performance computing module.
[0054] For example, the processor of the interactive tablet can be an Android module, which means that the Android system can be installed and configured with components such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), RAM (random access memory) and ROM (Read-Only Memory). For example, for Android 7.0 version, the CPU is dual-core A72 and quad-core A53, the GPU is Mali T860, the RAM is 4GB, the ROM is 32GB, and so on.
[0055] For another example, the processor of the interactive tablet can be a PC (personal computer) module, which is configured with components such as CPU, GPU, memory, hard disk, etc. For example, for a plug-in Intel Core series modular computer, the CPU is Intel Core i5 / i7, the GPU is Intel HD Graphics, the memory is DDR4 8G / 16G, and the hard disk is 128G / 256G.
[0056] like Figure 2B As shown, in an interactive tablet, audio processing components such as a microphone and a speaker 201 can usually be configured. The scientific name of a microphone is a microphone, also known as a microphone or a micro-phone, which is an energy conversion device that converts sound signals into electrical signals. The speaker 201 is also called a horn, which is a transducer device that converts electrical signals into sound signals.
[0057] In some scenarios, the interactive tablet communicates with other devices based on voice, such as telephone conferences, video conferences, voice chats, video chats, etc. At this time, the processor of the interactive tablet receives the audio signal sent by the device at the other end and plays it through the speaker 201, and the processor of the interactive tablet receives the audio signal through the microphone and transmits it to the device at the other end.
[0058] In some scenarios, the interactive tablet plays a file with voice, for example, when playing courseware, a movie clip or a song clip is played. At this time, the processor of the interactive tablet can call the speaker 201 to play the audio signal in the file.
[0059] Generally, a speaker is provided with a magnet with multiple coils wrapped around it. When different electron energies are transmitted to the coils, the coils generate an energy that interacts with the magnetic field of the magnets. This interaction causes the paper plate of the speaker to vibrate. As the electron energy changes at any time, the coil of the speaker will move forward or backward, so the speaker of the speaker will move with it. These actions change the density of the air and produce sound. Therefore, when the speaker plays an audio signal, it usually produces a certain degree of vibration. At this time, the speaker is in a vibrating state.
[0060] Step 202: The processor of the vibration detection device synchronously collects the audio signal received by the speaker.
[0061] When the loudspeaker plays the audio signal, when the loudspeaker plays each frame of the audio signal, the processor of the vibration detection device can synchronously collect the frame of the audio signal.
[0062] Step 203: The display screen of the interactive flat panel receives a touch from a touch object.
[0063] The interactive tablet is equipped with a display screen, for example, an LED (Light Emitting Diode) display screen, an OLED (Organic Light-Emitting Diode) display screen, an LCD (Liquid Crystal Display) display screen, etc. By arranging optical touch sensors on both sides of the display screen surface, a touch display screen can be formed.
[0064] like Figure 2B As shown, the user uses the touch object 203 to touch the surface of the display screen to trigger a touch operation, for example, clicking a control (such as a control for turning pages, a control for pausing playback, etc.), writing handwriting, and so on.
[0065] It is understandable that in order to protect the display screen from being scratched by a touching object, a glass cover plate is provided in the display screen. Therefore, in this embodiment, the surface of the display screen may refer to the surface of the glass cover plate of the display screen.
[0066] Step 204: The sensor of the vibration detection device detects a first vibration signal transmitted from the display screen of the interactive flat panel.
[0067] like Figure 2B As shown, at a location in the interactive tablet where vibration can be transmitted, such as a display screen, the back of the interactive tablet, etc., one or more sensors 202 are configured for the vibration detection device. The sensor 202 of the vibration detection device can be used to detect vibration, such as a piezoelectric sensor, a bone conduction sensor, etc.
[0068] In addition to the speaker generating a certain degree of vibration when playing an audio signal, a user usually generates a certain degree of vibration when using the touch object 203 to touch the surface of the display screen and trigger a touch operation. Therefore, the sensor of the vibration detection device can detect the vibration generated on the surface of the interactive tablet display screen, thereby generating a first vibration signal, which includes the vibration of the speaker and the vibration signal generated by the touch object touching the display screen.
[0069] Step 205: The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, and obtains a second vibration signal.
[0070] The vibration detection device is configured with a processor. The processor of the vibration detection device may be the same processor as the processor of the interactive tablet, or may be another processor independent of the processor of the interactive tablet, which is not limited in this embodiment.
[0071] The vibration generated when the speaker plays the audio signal can also be transmitted to the sensor of the vibration detection device, thereby interfering with the vibration generated when the touch object touches the display screen and triggers the touch operation.
[0072] Since the characteristics of vibration generated when a speaker plays an audio signal are different from the characteristics of vibration generated when a touch object touches a display screen and triggers a touch operation, this embodiment can remove the interference of the speaker from the first vibration signal based on the difference in these characteristics. For ease of distinction, the first vibration signal without the interference of the speaker can be called the second vibration signal.
[0073] In one embodiment of the present invention, step 205 includes the following steps:
[0074] Step 2051: Determine multiple interference paths.
[0075] In this embodiment, the vibration generated when the speaker plays an audio signal can be called a noise signal. After being attenuated by various components in the interactive tablet, the noise signal can reach the sensor of the vibration detection device. In this case, the noise signal can also be called an echo signal.
[0076] Generally speaking, when the loudspeaker and the sensor of the vibration detection device are fixed, there is a relatively fixed attenuation gain between the echo signal and the noise signal.
[0077] Therefore, the path for transmitting the echo between the loudspeaker and the sensor of the vibration detection device can be determined to obtain the interference path, that is, the interference path is the path when the noise signal when the loudspeaker vibrates is propagated as the echo signal.
[0078] The audio signal played by the speaker changes in real time, so the noise signal generated when the speaker plays the audio signal also changes in real time. However, the amplitude of the change is small. Therefore, in some cases, the interference path can be considered to be a fixed value. In some cases, the interference path can be updated in real time.
[0079] In a method of real-time updating of the interference path, it is considered that the noise signal generated when the speaker plays the audio signal has a certain correlation with the audio signal. For example, the higher the volume of the audio signal, the greater the amplitude of the speaker vibration, the higher the pitch of the audio signal, the higher the frequency of the speaker vibration, and so on. Therefore, before the speaker plays the audio signal, the audio signal to be played is collected through the internal circuit. The audio signal can be used as a reference signal to calculate the interference path. For scenarios such as video conferencing, the audio signal is a voice signal sent by the other end to the interactive tablet.
[0080] like Figure 2B As shown, the speaker 201 plays the audio signal, which is a continuous behavior. Continuously playing multiple frames of audio signals, and the user uses the touch object 203 to trigger the touch operation, which is also a continuous behavior. The surface of the display screen is continuously touched. These operations can form different vibration signals. After these vibration signals propagate and attenuate through multiple original paths in the space, they can be superimposed at the sensor 202 of the vibration detection device to form a first vibration signal. If multiple frames of audio signals are propagated and attenuated through multiple original paths and then superimposed to form a first vibration signal, the original path can be reversely calculated based on the first vibration signal and the audio signal.
[0081] The original path is the path when the first vibration signal is propagated as the echo signal. The first vibration signal may include vibrations generated when a touch object triggers a touch operation, or may include vibrations generated when a speaker plays an audio signal.
[0082] The first vibration signal of the nth frame (n is a positive integer) is decomposed into the convolution between the original path of the 0th frame to the K-1th frame (K is a positive integer) and the audio signals of the nth frame to the n-K+1th frame. When the first vibration signal of the current frame and multiple frames of audio signals in the second time period are known, inverse convolution can be performed based on the first vibration signal of the nth frame and the audio signals of the nth frame to the n-K+1th frame to calculate the original path.
[0083] The so-called convolution may refer to the sum of the multiplication of the original path and the audio signal. For the multiplied original path and the audio signal, the sum of the sequence number of the original path and the sequence number of the audio signal is equal to the sequence number of the first vibration signal of the current frame, which is expressed as follows:
[0084]
[0085] Wherein, y is the first vibration signal, w is the original path, and X is the audio signal.
[0086] In a short period of time, the original path changes little and can be considered to be the same. Then, the original path can be solved by the above-mentioned method of decomposing the original path, especially the original path of the initial one or more frames.
[0087] For the same time, there is a difference between the original path and the interference path, but the difference is small. Therefore, the original path can be filtered to obtain the initial interference path.
[0088] Exemplarily, a linear adaptive filter may be determined, and the original path may be input into the adaptive filter for processing to obtain the interference path.
[0089] Among them, the adaptive filter refers to a filter that uses an adaptive algorithm to change the parameters and structure of the filter according to changes in the environment.
[0090] Linear adaptive filtering is to solve fe=f(fs), establish an echo signal model, perform echo cancellation, and continuously adjust the parameters to make the estimated value closer to the real echo signal. The closer the estimated value is to the real echo signal, the better the echo cancellation effect.
[0091] Among them, fs = far-end signal (far-end voice signal), fe = far-end echo (far-end echo signal).
[0092] Of course, in addition to the adaptive filter, other filters may be used to filter the original path, and this embodiment does not limit this.
[0093] Step 2052: After the multiple frames of audio signals are propagated and attenuated through multiple interference paths, they are superimposed to form a noise signal when the speaker vibrates.
[0094] like Figure 2B As shown, the speaker 201 plays the audio signal which is a continuous behavior. It plays multiple frames of audio signals continuously. The operation of playing the audio signal can form different vibration signals. After these vibration signals propagate and attenuate through multiple interference paths in the space, they can be superimposed at the sensor 202 of the vibration detection device to form a noise signal.
[0095] Therefore, the convolution between the interference path from frame 0 to frame K-1 and the audio signal from frame n to frame n-K+1 is calculated to obtain the noise signal of frame n when the speaker vibrates, which is expressed as follows:
[0096]
[0097] Among them, echo is the noise signal, w' is the interference path, and X is the audio signal.
[0098] The so-called convolution may refer to summing the multiplied interference path and the audio signal. For the multiplied original path and the audio signal, the sum of the sequence number of the interference path and the sequence number of the audio signal is equal to the sequence number of the current frame noise signal.
[0099] Step 2053: Subtract the current frame signal of the noise signal from the current frame signal of the first vibration signal to obtain the current frame signal of the second vibration signal.
[0100] By subtracting the current frame noise signal from the current frame first vibration signal, the current frame second vibration signal can be obtained, which is expressed as follows:
[0101] e(n)=y(n)-echo(n)
[0102] Among them, e is the second vibration signal, y is the first vibration signal, and echo is the noise signal.
[0103] In one embodiment of the present invention, considering that the voice signal played by the speaker changes frame by frame and the touch operation of the touch object also changes frame by frame, the interference path can be adjusted frame by frame according to the changes in the voice signal played by the speaker and the touch operation of the touch object, so that the estimated interference path is closer to the actual interference path, thereby improving the accuracy of the interference path. The closer the estimated interference path is to the actual interference path, the better the denoising effect will be.
[0104] Specifically, after determining the second vibration signal of the current frame, the interference path of the previous frame can be extracted, and the update information can be calculated based on the second vibration signal of the current frame and the audio signal of the current frame, wherein the update information includes the update direction and the update amplitude, thereby updating the interference path of the previous frame according to the update information to obtain the interference path of the current frame.
[0105] Exemplarily, the product of the second vibration signal of the current frame, the audio signal of the current frame and the preset weight is calculated as the update information. At this time, the update of the interference path is expressed as follows:
[0106] w′(n)=w′(n-1)+μe(n)X(n)
[0107] Wherein, w' is the interference path, e is the second vibration signal, X is the audio signal, and μ is the weight. Usually, 0≤μ≤2.
[0108] Initially, the interference path is obtained by filtering the original path. Then, the interference path can be updated subsequently by the real-time current frame second vibration signal and the current frame audio signal to maintain the accuracy of the interference path. In addition, the updated interference path can be used as the interference path of the first time period to calculate the noise signal, thereby denoising the first vibration signal and obtaining an accurate second vibration signal.
[0109] Step 206 , determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object; if so, executing step 207 .
[0110] Step 207: The interactive tablet responds to the touch of the touch object.
[0111] The vibration characteristics of the touch object are different depending on whether it touches the surface of the display screen. Therefore, one or more conditions for the touch of the touch object can be set in advance. After denoising, the characteristics of the second vibration signal can be analyzed to determine whether the second vibration signal meets the condition, thereby determining whether the touch object touches the surface of the display screen.
[0112] In some scenarios, distinguishing whether the touch object touches the surface of the display screen has a positive significance for the touch operation application of the interactive tablet. For example, a touch touch is defined when the touch object touches the surface of the display screen, and a floating touch is defined when the touch object does not touch the surface of the display screen. At this time, the meaning of the touch operation can be enriched, and the touch operation will give the touch screen more expressiveness, thereby improving the flexibility of the touch operation.
[0113] Furthermore, for a writing scenario, when the touch object touches the surface of the display screen, handwriting can be written, and when the touch object does not touch the surface of the display screen, handwriting is not written, thereby solving the problem of zero-height writing.
[0114] Taking handwriting as an example, when a user writes on the display screen of an interactive tablet, the handwriting will be displayed when the finger is within the touch area of the infrared detection device but does not touch the display screen. However, the user may be starting or putting down the pen at this time and does not intend to write. Displaying the handwriting of starting and putting down the pen will result in connected strokes, and the handwriting will appear rather sloppy.
[0115] As Figure 3A shown, if the user writes the character "正", there are continuous starting and ending strokes 311 between the first horizontal stroke and the first vertical stroke, a starting stroke 312 at the end of the second horizontal stroke, and continuous starting and ending strokes 313 between the second vertical stroke and the third horizontal stroke, all of which have relatively obvious connected strokes.
[0116] Applying the method of this embodiment, as Figure 3B shown, if the user writes the character "正", the surface of the display screen touched by the touch object (shown as a solid line) and the surface of the display screen not touched by the touch object (shown as a dotted line) are distinguished. It can be seen that there are continuous starting and ending strokes 311 between the first horizontal stroke and the first vertical stroke, a starting stroke 312 at the end of the second horizontal stroke, and continuous starting and ending strokes 313 between the second vertical stroke and the third horizontal stroke, all of which have relatively obvious connected strokes, and these connected strokes are mostly generated when the touch object does not touch the surface of the display screen.
[0117] As Figure 3C shown, when the touch object touches the surface of the display screen, handwriting can be written. When the touch object does not touch the surface of the display screen, no handwriting is written. At this time, for the continuous starting and ending strokes 311 between the first horizontal stroke and the first vertical stroke, the starting stroke 312 at the end of the second horizontal stroke, and the continuous starting and ending strokes 313 between the second vertical stroke and the third horizontal stroke, relatively obvious connected strokes can be eliminated, making the character "正" more regular.
[0118] Furthermore, as Figure 3D shown, in area 321, the method of distinguishing whether the touch object touches the surface of the display screen to determine whether to write handwriting is applied to write multiple characters "正". In area 322, the prior art is applied to directly respond to the touch object to draw corresponding handwriting and write multiple characters "正".
[0119] By comparison, it can be seen that for the multiple characters "正" in area 321, the situation of connected strokes is greatly reduced, and the whole appears more regular. For the multiple characters "正" in area 322, the situation of connected strokes is more obvious, and the whole appears more scribbled.
[0120] In specific implementation, a threshold value γ can be preset through experiments, and the second vibration signal is compared with the preset threshold value γ to determine whether the touch object touches the surface of the display screen.
[0121] If the second vibration signal e is greater than the threshold value γ, indicating that the second vibration signal e is relatively large, it can be determined that one or more touch objects touch the surface of the display screen, and it is determined that the second vibration signal meets the condition for the interactive flat panel to respond to the touch of the touch object.
[0122] If the second vibration signal e is less than or equal to the threshold value γ, it means that the second vibration signal e is small, and it can be determined that the touch object does not touch the surface of the display screen, and the second vibration signal does not meet the condition of triggering the interactive tablet to respond to the touch of the touch object.
[0123] In this embodiment, the processor of the interactive tablet sends an audio signal to the speaker, so that the speaker is in a vibrating state. The processor of the vibration detection device synchronously collects the audio signal received by the speaker. The display screen of the interactive tablet receives the touch of the touch object. The sensor of the vibration detection device detects a first vibration signal transmitted from the display screen of the interactive tablet. The first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration caused by the touch object touching the display screen. The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, obtains a second vibration signal, and determines whether the second vibration signal meets the condition of triggering the interactive tablet to respond to the touch of the touch object. If so, the interactive tablet responds to the touch of the touch object. When the touch object triggers the touch operation, the touch operation can be given a more dimensional feature to enrich the meaning of the touch operation. In some scenarios, if distinguishing whether the touch object touches the surface of the display screen has a more positive meaning, then this embodiment distinguishes the way of responding to the touch operation on the basis of detecting whether the touch object touches the surface of the display screen, adapts to the needs of the scene, and thus improves the performance of the touch operation in the scene. In the writing scene, the problem of zero-height writing can be solved.
[0124] Secondly, in scenarios such as writing, users can continue to use resistive or capacitive touch operations without having to learn new touch operation skills for interactive tablets, thereby reducing the cost of using touch operations on interactive tablets.
[0125] Furthermore, the optical touch sensor can maintain a protruding setting, which reduces or avoids the need to bury the optical touch sensor in the interactive tablet, reduces or avoids the need for recessed control of the structure of the interactive tablet (such as the glass and back panel of the display), reduces costs and control difficulties, and because the optical touch sensor can maintain the touch area, it can maintain the scanned information and maintain the accuracy of touch detection.
[0126] Embodiment 2
[0127] Figure 4 This is a flow chart of a human-computer interaction method for an interactive tablet provided in Embodiment 2 of the present invention. This embodiment is based on the previous embodiment and further adds an infrared touch operation. The method specifically includes the following steps:
[0128] Step 401: The processor of the interactive tablet sends an audio signal to the speaker, so that the speaker is in a vibrating state.
[0129] Step 402: The processor of the vibration detection device synchronously collects the audio signal received by the speaker.
[0130] Step 403: Start the infrared detection device to detect the touch of the touch object and generate a detection data packet.
[0131] The infrared detection device is a sensor that uses infrared signals to scan touch objects. In this embodiment, the infrared detection device can be used to detect touch objects, such as user's fingers, stylus, etc., on the surface of the display screen using infrared signals, and generate a detection data packet when the touch of the touch object is detected.
[0132] From the perspective of technical principles, they can include infrared touch technology, CCD (charge coupled device) / CMOS (Complementary Metal Oxide Semiconductor) touch technology, non-total internal reflection touch technology (Frustrated Total Internal Reflection, FITR), laser plane touch technology (Laser Light Plane, LLP), light emitting diode plane multi-point touch technology (Light emitting Diode Light Plane, LED-LP), and so on.
[0133] Taking infrared touch technology as an example, the infrared detection device includes an infrared transmitter and an infrared receiver. The infrared transmitter is used to transmit infrared signals, and the infrared receiver is used to receive infrared signals. The infrared signals densely distributed in different directions are used to form a light beam grid to locate the touch point.
[0134] The display screen is installed with a frame with a circuit board, the function of which is to arrange infrared transmitters and infrared receivers around the display screen to form a horizontal and vertical cross beam grid.
[0135] When the touch object blocks the infrared signal, it will cause the light measurement value at the corresponding infrared receiver to decrease, so the position of the touch point on the screen can be determined.
[0136] In addition, in one example, the touch point data included in the touch control data includes at least one of the following items:
[0137] Status, Touch ID, X coordinate, Y coordinate, width, height.
[0138] In this example, the data format of the touch data packet is as follows:
[0139]
[0140] The status may include at least one of the following data:
[0141] Credibility,retention parameters.
[0142] Credibility is an optional parameter;
[0143] Reserved parameters are reserved fields, and you can add required parameters based on the needs of the business scenario.
[0144] In this example, the status data format is as follows:
[0145]
[0146] Step 404: The display screen of the interactive tablet receives a touch from a touch object.
[0147] Step 405: The sensor of the vibration detection device detects a first vibration signal transmitted from the display screen of the interactive flat panel.
[0148] The first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration caused by the touch object touching the display screen.
[0149] Step 406: The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, and obtains a second vibration signal.
[0150] Step 407 , determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object; if so, executing step 408 .
[0151] Step 408: The interactive tablet responds to the touch of the touch object according to the detection data packet.
[0152] In this embodiment, the interactive tablet can analyze the content of the detection data packet, thereby responding to the touch of the touch object and realizing the touch operation.
[0153] In one example, when the touch of the touch object is a writing operation, the infrared detection device sends a detection data packet to the processor of the interactive tablet, and the processor of the interactive tablet generates touch coordinate data of the touch object according to the detection data packet, that is, the processor of the interactive tablet filters out the detection data packet recorded when the touch object does not touch the surface of the display screen (the second vibration signal does not meet the condition of triggering the interactive tablet to respond to the touch of the touch object), and the remaining detection data packet recorded when the touch object touches the surface of the display screen (the second vibration signal meets the condition of triggering the interactive tablet to respond to the touch of the touch object), and obtains the touch coordinate data of the touch object from the X coordinate and Y coordinate in the detection data packet.
[0154] If the touch coordinate data is obtained through analysis, the handwriting of the touch object can be displayed on the display screen along the touch coordinate data, and zero-height writing can be achieved.
[0155] In another example, when the touch of the touch object is a writing operation, the detection data packet generated based on the writing operation and the generation time of the detection data packet (i.e., the time recorded when the detection data packet is generated) are continuously obtained, the generation time of the second vibration signal that meets the above conditions obtained by the processor of the vibration detection device is determined (i.e., the time recorded when the second vibration signal that meets the above conditions is generated), and the generation time of the detection data packet is compared with the generation time of the second vibration signal that meets the above conditions. If the generation time is before the generation time, the touch object does not touch the surface of the display screen during the generation time; if the generation time is after the generation time, the touch object touches the surface of the display screen during the generation time. Therefore, the detection data packet whose generation time is after the generation time is responded to, that is, the writing handwriting of the touch object is displayed along the coordinate data (X coordinate, Y coordinate) in the detection data packet whose generation time is after the generation time, so that zero-height writing can be achieved.
[0156] Embodiment 3
[0157] Figure 5 This is a flowchart of a human-computer interaction method for an interactive tablet provided in Embodiment 3 of the present invention. This embodiment is based on the previous embodiment and further adds an operation of identifying the type of the touch object. The method specifically includes the following steps:
[0158] Step 501: The processor of the interactive tablet sends an audio signal to the speaker, so that the speaker is in a vibrating state.
[0159] Step 502: The processor of the vibration detection device synchronously collects the audio signal received by the speaker.
[0160] Step 503: The display screen of the interactive tablet receives a touch from a touch object.
[0161] Step 504: The sensor of the vibration detection device detects a first vibration signal transmitted from the display screen of the interactive flat panel.
[0162] The first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration caused by the touch object touching the display screen.
[0163] Step 505: The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, and obtains a second vibration signal.
[0164] Step 506 , determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object; if so, executing step 507 .
[0165] Step 507: Search for a sample vibration signal.
[0166] In this embodiment, different types of touch objects can be configured for the interactive tablet. The materials of different types of touch objects are different, for example, wood, plastic, rubber, etc. These touch objects can be configured by the manufacturer of the interactive tablet or by the user. The same touch object can include a single material or multiple materials at the same time. For example, one end of the touch pen is made of plastic and the other end of the touch pen is made of rubber, etc. This embodiment does not limit this.
[0167] The characteristics of vibrations generated when different touch objects touch the surface of the display screen are different. Sample vibration signals can be recorded in advance for these touch objects under the condition that the interactive tablet calls the speaker without playing an audio signal. That is, the sample vibration signal is a vibration signal detected by the sensor of the vibration detection device when the speaker does not play an audio signal, which indicates that the specified type of touch object touches the surface of the interactive tablet when the interactive tablet calls the speaker without playing an audio signal.
[0168] These sample vibration signals can be stored in a local database of the interactive tablet, and when a touch object triggers a touch operation, the sample vibration signal can be read from the database.
[0169] Step 508: Match the second vibration signal with the sample vibration signal.
[0170] Step 509: If the match is successful, it is determined that the touch object belongs to the type corresponding to the sample vibration signal.
[0171] Step 510: respond to the touch of the touch object based on the type.
[0172] Different types of touch objects are made of different materials, so that different types of touch objects have different characteristics when vibrating. Therefore, features can be extracted from each sample vibration signal in advance, and features of the second vibration signal can be extracted, and the features of the second vibration signal can be matched with the features of each sample vibration signal in turn. If the features of the second vibration signal successfully match the features of a sample vibration signal, the type of touch object to which the sample vibration signal belongs can be extracted, and the type is assigned to the touch object currently triggering the touch operation on the interactive tablet, thereby triggering a response pre-defined for the touch object of this type, for example, writing handwriting in a first style (such as red), writing handwriting in a second style (such as green), erasing handwriting, and so on.
[0173] Exemplarily, on the one hand, the second vibration signal can be converted into a spectrum diagram by means of Discrete Fourier Transform (DFT), short-time Fourier transform (or, short-term Fourier transform, STFT), etc. As a feature of the second vibration signal, for ease of distinction, the spectrum diagram can be called a target spectrum signal, wherein the Fourier transform can reflect the mean of the frequency in the second vibration signal, but cannot reflect the dynamic characteristics of the frequency changing with time, and the short-time Fourier transform overcomes this weakness by adding a window to the second vibration signal, and can reflect both the frequency intensity of the second vibration signal and the change of the frequency intensity with time.
[0174] On the other hand, the sample vibration signal is converted into a spectrum diagram in advance through Fourier transform, short-time Fourier transform, etc. As a feature of the sample vibration signal, for the convenience of distinction, the spectrum diagram can be called a sample spectrum signal. The sample spectrum signal can replace the sample vibration signal and be stored in a database to characterize the sample vibration signal. Therefore, when the sample vibration signal is to be matched, the sample spectrum signal converted from the sample vibration signal can be searched in the database.
[0175] The similarity between the target spectrum signal and the sample spectrum signal is calculated. If the similarity meets a preset matching condition, it can be determined that the second vibration signal matches the sample vibration signal successfully.
[0176] The matching condition may include at least one of the following:
[0177] The similarity is the highest and the similarity is greater than the threshold.
[0178] In an embodiment of the present invention, a sample vibration signal is searched, the sample vibration signal indicating that a specified type of touch object touches the surface of the interactive flat panel under the condition that the interactive flat panel calls the speaker but does not play an audio signal, the second vibration signal is matched with the sample vibration signal, and if the match is successful, it is determined that the touch object belongs to the type corresponding to the sample vibration signal, thereby responding to the touch of the touch object based on the type. In some scenarios, if distinguishing the types of touch objects has a more positive meaning, then this embodiment distinguishes different types of touch objects based on the vibration characteristics detected on the surface of the display screen of the interactive flat panel, and can distinguish different ways of responding to touch operations based on different types of touch objects, adapt to the needs of the scenario, and thus improve the performance of the touch operation in the scenario.
[0179] For example, in a writing scenario, the color and thickness of the handwriting can be distinguished for different types of touch objects, so that the content written by different users can be distinguished without the need for different users to frequently switch writing configurations when writing alternately.
[0180] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0181] Embodiment 4
[0182] Figure 6 This is a structural block diagram of an interactive tablet provided in Embodiment 4 of the present invention. The interactive tablet 600 includes an infrared detection device 610, a speaker 620 and a vibration detection device 630;
[0183] The processor 601 of the interactive tablet 600 is used to send an audio signal to the speaker 620, so that the speaker 620 is in a vibrating state;
[0184] The processor 631 of the vibration detection device 630 is used to synchronously collect the audio signal received by the speaker 620;
[0185] The display screen 602 of the interactive tablet 600 is used to receive touches from a touch object;
[0186] The sensor 632 of the vibration detection device 630 is used to detect a first vibration signal transmitted from the display screen 602 of the interactive tablet 600, wherein the first vibration signal includes a vibration signal generated by the vibration of the speaker 620 and the vibration generated by the touch object touching the display screen 602;
[0187] The processor 631 of the vibration detection device 630 is further used to process the first vibration signal, remove the vibration signal generated by the vibration of the speaker 620, and obtain a second vibration signal;
[0188] Determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object;
[0189] The interactive tablet 600 is used to respond to the touch of the touch object when the condition is met.
[0190] In one embodiment of the present invention, the infrared detection device 610 is used to detect the touch of the touch object and generate a detection data packet.
[0191] In one embodiment of the present invention, the infrared detection device 610 is further used to send the detection data packet to the processor 601 of the interactive tablet 600 when the touch of the touch object is a writing operation;
[0192] The processor 601 of the interactive tablet 600 is further configured to generate touch coordinate data of the touch object according to the detection data packet, and display the handwriting of the touch object on the display screen.
[0193] In one embodiment of the present invention, the interactive tablet 600 is also used for:
[0194] When the touch of the touch object is a writing operation, continuously obtaining a detection data packet generated based on the writing operation and a generation time of the detection data packet;
[0195] Determining a generation time of the second vibration signal satisfying the condition obtained by the processor of the vibration detection device;
[0196] A response is made to a detection data packet whose generation time is after the generation time.
[0197] In one embodiment of the present invention, the processor 631 of the vibration detection device 630 includes:
[0198] An interference path determination module, used to determine a plurality of interference paths, wherein the interference paths are paths along which the noise signal when the speaker vibrates is propagated as the echo signal;
[0199] A noise signal superposition module, used for superimposing multiple frames of the audio signal after propagation and attenuation through multiple interference paths to form a noise signal when the speaker vibrates;
[0200] The signal subtraction module is used to subtract the current frame signal of the noise signal from the current frame signal of the first vibration signal to obtain the current frame signal of the second vibration signal.
[0201] In one embodiment of the present invention, the noise signal superposition module includes:
[0202] The convolution calculation module is used to calculate the convolution between the interference path of the 0th frame to the K-1th frame and the audio signal of the nth frame to the n-K+1th frame to obtain the noise signal of the nth frame when the speaker vibrates.
[0203] In one embodiment of the present invention, the vibration signal denoising module 503 further includes:
[0204] A signal decomposition module, configured to calculate the original path based on the first vibration signal and the audio signal if multiple frames of the audio signal are propagated and attenuated through multiple original paths and then superimposed to form the first vibration signal, wherein the original path is a path when the first vibration signal is propagated as an echo signal;
[0205] The filtering processing module is used to filter the original path to obtain an interference path.
[0206] In one embodiment of the present invention, the signal decomposition module includes:
[0207] A first vibration signal decomposition module, used for decomposing the first vibration signal of the nth frame into a convolution between the original path of the 0th frame to the K-1th frame and the audio signal of the nth frame to the n-K+1th frame;
[0208] A reverse calculation module is used to calculate the original path based on the first vibration signal of the nth frame and the audio signals of the nth frame to the (n-K+1)th frame.
[0209] In one embodiment of the present invention, the filtering processing module includes:
[0210] An adaptive filter determination module, used for determining a linear adaptive filter;
[0211] The linear filtering module is used to input the original path into the adaptive filter for processing to obtain the interference path.
[0212] In one embodiment of the present invention, the processor 631 of the vibration detection device 630 further includes:
[0213] An interference path extraction module, used to extract the interference path of the previous frame;
[0214] An update information calculation module, used to calculate update information based on the second vibration signal of the current frame and the audio signal of the current frame, wherein the update information includes an update direction and an update amplitude;
[0215] The interference path updating module is used to update the interference path of the previous frame according to the update information to obtain the interference path of the current frame.
[0216] In one embodiment of the present invention, the update information calculation module includes:
[0217] The product calculation module is used to calculate the product between the second vibration signal of the current frame, the audio signal of the current frame and a preset weight as update information.
[0218] In one embodiment of the present invention, the processor 631 of the vibration detection device 630 includes:
[0219] A threshold value comparison module, used for comparing the second vibration signal with a preset threshold value;
[0220] The threshold value determines the threshold, and is used to determine that the second vibration signal meets the condition of triggering the interactive tablet to respond to the touch of the touch object if the second vibration signal is greater than the threshold value.
[0221] In one embodiment of the present invention, the interactive tablet 600 includes:
[0222] A sample vibration signal search module, used to search for a sample vibration signal, wherein the sample vibration signal indicates that a specified type of touch object touches the surface of the interactive flat panel under the condition that the interactive flat panel calls the speaker but does not play an audio signal;
[0223] a sample vibration signal matching module, configured to match the second vibration signal with the sample vibration signal;
[0224] a type determination module, configured to determine, if the match is successful, that the touch object belongs to the type corresponding to the sample vibration signal;
[0225] The type response module is used to respond to the touch of the touch object based on the type.
[0226] In one embodiment of the present invention, the sample vibration signal matching module includes:
[0227] A target spectrum signal conversion module, used for converting the second vibration signal into a target spectrum signal;
[0228] A sample spectrum signal search module, used for searching a sample spectrum signal converted from the sample vibration signal;
[0229] A similarity calculation module, used to calculate the similarity between the target spectrum signal and the sample spectrum signal;
[0230] The matching condition determination module is used to determine whether the second vibration signal matches the sample vibration signal successfully if the similarity satisfies a preset matching condition.
[0231] The interactive tablet provided in the embodiment of the present invention can execute the human-computer interaction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0232] Embodiment 5
[0233] Figure 7 This is a schematic diagram of the structure of an interactive tablet provided in Embodiment 4 of the present invention. Figure 7A block diagram of an exemplary interactive tablet 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7 The interactive tablet 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0234] like Figure 7 As shown, the interactive tablet 12 is in the form of a general-purpose computing device. The components of the interactive tablet 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
[0235] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
[0236] The interactive tablet 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the interactive tablet 12, including volatile and non-volatile media, removable and non-removable media.
[0237] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The interactive tablet 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.
[0238] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.
[0239] The interactive tablet 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the interactive tablet 12, and / or any device that enables the interactive tablet 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the interactive tablet 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the interactive tablet 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the interactive tablet 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0240] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the human-computer interaction method of the interactive tablet provided in the embodiment of the present invention.
[0241] Embodiment 6
[0242] Embodiment 6 of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the human-computer interaction method of the above-mentioned interactive tablet are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0243] Among them, computer-readable storage media may include, for example, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.
[0244] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A human-computer interaction method, characterized in that: Applied to an interactive tablet, the interactive tablet includes an infrared detection device, a speaker and a vibration detection device, and the method includes: The processor of the interactive tablet sends an audio signal to the speaker, so that the speaker is in a vibrating state; The processor of the vibration detection device synchronously collects the audio signal received by the speaker; The display screen of the interactive flat panel receives touch of a touch object; The sensor of the vibration detection device detects a first vibration signal transmitted from the display screen of the interactive flat panel, wherein the first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration generated by the touch object touching the display screen; The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, and obtains a second vibration signal; Determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object; If the conditions are met, the interactive tablet responds to the touch of the touch object; The processor of the vibration detection device processes the first vibration signal, removes the vibration signal generated by the vibration of the speaker, and obtains the second vibration signal, including: Determine a plurality of interference paths, wherein the interference paths are paths along which noise signals when the loudspeaker vibrates are propagated as echo signals; After the multiple frames of the audio signal are propagated and attenuated through the multiple interference paths, they are superimposed to form a noise signal when the speaker vibrates; subtracting the current frame signal of the noise signal from the current frame signal of the first vibration signal to obtain a current frame signal of the second vibration signal; Extracting the interference path of the previous frame; Calculate update information based on the second vibration signal of the current frame and the audio signal of the current frame, wherein the update information includes an update direction and an update amplitude; The interference path of the previous frame is updated according to the update information to obtain the interference path of the current frame.
2. The human-computer interaction method according to claim 1, characterized in that: Before the display screen of the interactive flat panel receives a touch of a touch object, the method further includes: The infrared detection device is started to detect the touch of the touch object and generate a detection data packet.
3. The human-computer interaction method according to claim 2, characterized in that: The interactive flat panel responds to the touch of the touch object, including: When the touch of the touch object is a writing operation, the infrared detection device sends the detection data packet to the processor of the interactive tablet; The processor of the interactive tablet generates touch coordinate data of the touch object according to the detection data packet, and displays the handwriting of the touch object on the display screen.
4. The human-computer interaction method according to claim 2, characterized in that: The interactive flat panel responds to the touch of the touch object, including: When the touch of the touch object is a writing operation, continuously obtaining a detection data packet generated based on the writing operation and a generation time of the detection data packet; Determining a generation time of the second vibration signal satisfying the condition obtained by the processor of the vibration detection device; A response is made to a detection data packet whose generation time is after the generation time.
5. The method according to claim 1, characterized in that The step of superimposing the multiple frames of the audio signal after propagation and attenuation through the multiple interference paths to form a noise signal when the speaker vibrates includes: The convolution between the interference path from the 0th frame to the K-1th frame and the audio signal from the nth frame to the n-K+1th frame is calculated to obtain the noise signal of the nth frame when the speaker vibrates.
6. The method according to claim 1, characterized in that The processing of the first vibration signal to remove the vibration signal generated by the vibration of the speaker to obtain a second vibration signal also includes: If multiple frames of the audio signal are propagated and attenuated through multiple original paths and then superimposed to form the first vibration signal, the original path is calculated based on the first vibration signal and the audio signal, and the original path is the path when the first vibration signal is propagated as an echo signal; The original path is filtered to obtain an interference path.
7. The method according to claim 6, characterized in that The calculating the original path based on the first vibration signal and the audio signal comprises: Decomposing the first vibration signal of the nth frame into the convolution between the original path of the 0th frame to the K-1th frame and the audio signal of the nth frame to the n-K+1th frame; The original path is calculated based on the first vibration signal of the nth frame and the audio signals of the nth frame to the (n-K+1)th frame.
8. The method according to claim 6, characterized in that The filtering the original path to obtain the interference path includes: Determine the linear adaptive filter; The original path is input into the adaptive filter for processing to obtain an interference path.
9. The method according to claim 1, characterized in that: The calculating update information based on the second vibration signal of the current frame and the audio signal of the current frame includes: The product of the second vibration signal of the current frame, the audio signal of the current frame and a preset weight is calculated as update information.
10. The method according to any one of claims 1 to 9, characterized in that The determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object includes: comparing the second vibration signal with a preset threshold value; If the second vibration signal is greater than the threshold value, it is determined that the second vibration signal meets the condition of triggering the interactive tablet to respond to the touch of the touch object.
11. The method according to any one of claims 1 to 9, characterized in that The interactive flat panel responds to the touch of the touch object, including: searching for a sample vibration signal, where the sample vibration signal is a vibration signal generated when a sensor of the vibration detection device detects that a touch object of a specified type touches the display screen under the condition that the speaker does not play an audio signal; matching the second vibration signal with the sample vibration signal; If the match is successful, it is determined that the touch object belongs to the type corresponding to the sample vibration signal; The touch of the touch object is responded to based on the type.
12. The method according to claim 11, characterized in that The matching the second vibration signal with the sample vibration signal comprises: converting the second vibration signal into a target frequency spectrum signal; Finding a sample frequency spectrum signal converted from the sample vibration signal; Calculating the similarity between the target spectrum signal and the sample spectrum signal; If the similarity satisfies a preset matching condition, it is determined that the second vibration signal matches the sample vibration signal successfully.
13. An interactive tablet, characterized in that: The interactive tablet includes an infrared detection device, a speaker and a vibration detection device; The processor of the interactive tablet is used to send an audio signal to the speaker, so that the speaker is in a vibrating state; The processor of the vibration detection device is used to synchronously collect the audio signal received by the speaker; The display screen of the interactive tablet is used to receive touches from a touch object; The sensor of the vibration detection device is used to detect a first vibration signal transmitted from the display screen of the interactive flat panel, wherein the first vibration signal includes a vibration signal generated by the vibration of the speaker and the vibration generated by the touch object touching the display screen; The processor of the vibration detection device is further used to process the first vibration signal, remove the vibration signal generated by the vibration of the speaker, and obtain a second vibration signal; Determining whether the second vibration signal satisfies a condition for triggering the interactive tablet to respond to the touch of the touch object; The interactive tablet is used to respond to the touch of the touch object when the condition is met; Wherein, the processor of the vibration detection device includes: An interference path determination module, used to determine a plurality of interference paths, wherein the interference paths are paths along which the noise signal when the speaker vibrates is propagated as the echo signal; A noise signal superposition module, used for superimposing multiple frames of the audio signal after propagation and attenuation through multiple interference paths to form a noise signal when the speaker vibrates; a signal subtraction module, configured to subtract the current frame signal of the noise signal from the current frame signal of the first vibration signal to obtain a current frame signal of the second vibration signal; An interference path extraction module, used to extract the interference path of the previous frame; An update information calculation module, used to calculate update information based on the second vibration signal of the current frame and the audio signal of the current frame, wherein the update information includes an update direction and an update amplitude; The interference path updating module is used to update the interference path of the previous frame according to the update information to obtain the interference path of the current frame.
14. An interactive tablet, characterized in that: The interactive tablet comprises: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the human-computer interaction method of the interactive tablet according to any one of claims 1 to 12.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the human-computer interaction method of the interactive tablet according to any one of claims 1 to 12 is implemented.
