Touch operation type detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a touch operation type detection method and device, electronic equipment and a storage medium, according to different follow-up changes of a vibration signal generated when normal writing operation starts and a vibration signal corresponding to an abnormal pulse, namely, the vibration signal generated by the normal writing operation is gradually reduced; the attenuation of the vibration signal corresponding to the abnormal pulse is suddenly changed, after the preliminary confirmation of the writing operation is completed, continuing to cache a section of subsequent vibration signal as a verification vibration signal, carrying out intensity change statistics on the verification vibration signal, and when the result of the intensity change statistics reaches a preset writing intensity threshold value, carrying out verification on the verification vibration signal; and confirming that the writing operation is detected. Therefore, the writing medium type which needs to be confirmed when the writing operation really occurs in the vibration signals generated in the writing operation process can be accurately distinguished, false detection of the writing medium type is eliminated, display mutation of handwriting attributes in the writing process is correspondingly avoided, and stability and accuracy of handwriting display in the writing process are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of interactive technologies, and in particular to a touch operation type detection method, device, electronic device, and storage medium. Background Art

[0002] With the continuous development and popularization of information technology, various types of electronic devices are constantly emerging, and the ways of connecting these devices are also becoming more and more diverse. For example, a conference room built on information technology is usually equipped with interactive tablets, microphones, speakers, mice, keyboards, and other electronic devices. These electronic devices are centered around the interactive tablet, and other electronic devices serve as peripherals. These devices are interconnected to realize the collection, transmission, and presentation of information during the meeting communication process.

[0003] In the various interactive designs of existing interactive tablets, users mainly operate the interactive tablets through various touch objects such as writing pens and fingers. In order to improve the interactive experience, related technologies use various sensors to detect the specific contact status of writing pens and fingers, so as to quickly and accurately complete the corresponding responses to various operations.

[0004] When the inventors used the writing operation function in the interactive tablet, they found that when responding to the writing operation, the medium type of the touch object can be identified based on the vibration signal generated when the touch object contacts the interactive tablet, and different handwriting attributes are displayed for touch objects of different medium types during the writing operation. However, the existing solution may mistakenly detect multiple writing medium types during a single writing operation, which may cause a sudden change in the display of handwriting attributes during the writing process, and the handwriting display during the writing process is not stable and accurate enough. Summary of the Invention

[0005] The present invention provides a touch operation type detection method, device, electronic device and storage medium to solve the technical problems that handwriting attributes may change suddenly during the writing process and the handwriting display during the writing process is not stable and accurate enough.

[0006] In a first aspect, an embodiment of the present invention provides a touch operation type detection method, the touch operation type detection method comprising:

[0007] Continuously acquiring vibration signals detected by an elastic wave detection device installed on the touch screen, and performing operation type detection on the acquired current vibration signals;

[0008] When the operation type of the current vibration signal is confirmed to be a pen-down operation, a preset number of vibration signals subsequent to the current vibration signal are cached as verification vibration signals;

[0009] Perform intensity change statistics on the verification vibration signal according to the fluctuation of the verification vibration signal;

[0010] When the result of the intensity change statistics reaches a preset writing intensity threshold, it is confirmed that the writing operation is detected.

[0011] As described above, based on the difference in subsequent changes between the vibration signal generated at the start of a normal writing operation and the vibration signal corresponding to the abnormal pulse, i.e., the attenuation of the vibration signal generated at the start of a normal writing operation is gradually reduced, while the attenuation of the vibration signal corresponding to the abnormal pulse is abrupt, after completing the initial confirmation of the writing operation, a subsequent vibration signal is cached as a verification vibration signal, and the intensity change statistics of the verification vibration signal are performed. When the result of the intensity change statistics reaches a preset writing intensity threshold, the writing operation is confirmed to have been detected. This allows the accurate distinction of the writing medium type that needs to be confirmed when the writing operation actually occurs in the vibration signal generated during the writing operation, eliminates false detection of the writing medium type, and accordingly avoids sudden changes in the display of handwriting attributes during the writing process, thereby improving the stability and accuracy of the handwriting display during the writing process.

[0012] The intensity change statistics of the verification vibration signal are performed according to the fluctuation of the verification vibration signal, including:

[0013] Confirm and verify the average value of the signal strength value of the vibration signal;

[0014] The sum of the absolute values ​​of the differences between the signal strength value of each verification vibration signal and the average value is counted as the result of the strength change statistics.

[0015] In the above, by calculating the difference between the signal strength value and the average value, the statistical judgment is completed quickly, thereby avoiding the delay of waiting for the display attribute during the user's writing process.

[0016] The vibration signal is acquired in frames, and each frame contains multiple vibration signals.

[0017] As described above, acquiring and processing vibration signals using frames as processing units can effectively improve the orderliness of vibration processing and avoid the situation where too little data makes accurate judgment impossible or too much data takes too long to process.

[0018] The number of vibration signals contained in each frame is the same as the number of verification vibration signals.

[0019] As described above, the verification of the writing operation is completed directly through a single-frame vibration signal, ensuring processing efficiency.

[0020] Each frame includes 600 vibration signals.

[0021] As described above, 600 vibration signals can control the balance between processing accuracy and processing delay as much as possible.

[0022] When the result of the intensity change statistics reaches a preset writing intensity threshold, after confirming that a writing operation is detected, the method further includes:

[0023] The type of writing medium corresponding to the writing operation is identified according to the current vibration signal.

[0024] As described above, by identifying the type of writing medium through the current vibration signal, the handwriting produced by different touch objects can be distinguished during the writing process, thereby improving the interaction effect.

[0025] The method of identifying the type of writing medium corresponding to the writing operation according to the current vibration signal includes:

[0026] Perform fast Fourier transform on the current vibration signal to obtain frequency domain information;

[0027] The frequency domain information is input into the pre-trained medium recognition model for recognition, and the writing medium type is output.

[0028] As described above, intelligent recognition based on frequency domain information extraction can quickly identify commonly used writing media and improve recognition efficiency and accuracy.

[0029] In a second aspect, an embodiment of the present invention provides a touch operation type detection device, the touch operation type detection device comprising:

[0030] An acquisition detection unit, configured to continuously acquire a vibration signal detected by an elastic wave detection device installed on the touch screen, and perform operation type detection on the acquired current vibration signal;

[0031] a signal buffer unit, configured to, upon confirming that the operation type of the current vibration signal is detected as a pen-down operation, buffer a preset number of vibration signals subsequent to the current vibration signal as verification vibration signals;

[0032] a signal statistics unit, configured to perform intensity change statistics on the verification vibration signal according to fluctuations of the verification vibration signal;

[0033] The writing confirmation unit is used to confirm that a writing operation is detected when the result of the intensity change statistics reaches a preset writing intensity threshold.

[0034] As described above, based on the difference in subsequent changes between the vibration signal generated at the start of a normal writing operation and the vibration signal corresponding to the abnormal pulse, i.e., the attenuation of the vibration signal generated at the start of a normal writing operation is gradually reduced, while the attenuation of the vibration signal corresponding to the abnormal pulse is abrupt, after completing the initial confirmation of the writing operation, a subsequent vibration signal is cached as a verification vibration signal, and the intensity change statistics of the verification vibration signal are performed. When the result of the intensity change statistics reaches a preset writing intensity threshold, the writing operation is confirmed to have been detected. This allows the accurate distinction of the writing medium type that needs to be confirmed when the writing operation actually occurs in the vibration signal generated during the writing operation, eliminates false detection of the writing medium type, and accordingly avoids sudden changes in the display of handwriting attributes during the writing process, thereby improving the stability and accuracy of the handwriting display during the writing process.

[0035] The signal statistics unit includes:

[0036] An average value confirmation module is used to confirm and verify the average value of the signal strength value of the vibration signal;

[0037] The difference statistics module is used to count the sum of the absolute values ​​of the differences between the signal strength value of each verification vibration signal and the average value as the result of the strength change statistics.

[0038] In the above, by calculating the difference between the signal strength value and the average value, the statistical judgment is completed quickly, thereby avoiding the delay of waiting for the display attribute during the user's writing process.

[0039] The vibration signal is acquired in frames, and each frame contains multiple vibration signals.

[0040] As described above, acquiring and processing vibration signals using frames as processing units can effectively improve the orderliness of vibration processing and avoid the situation where too little data makes accurate judgment impossible or too much data takes too long to process.

[0041] The number of vibration signals contained in each frame is the same as the number of verification vibration signals.

[0042] As described above, the verification of the writing operation is completed directly through a single-frame vibration signal, ensuring processing efficiency.

[0043] Each frame includes 600 vibration signals.

[0044] As described above, 600 vibration signals can control the balance between processing accuracy and processing delay as much as possible.

[0045] The touch operation type detection device further includes:

[0046] The medium identification unit is used to identify the type of writing medium corresponding to the writing operation according to the current vibration signal.

[0047] As described above, by identifying the type of writing medium through the current vibration signal, the handwriting produced by different touch objects can be distinguished during the writing process, thereby improving the interaction effect.

[0048] The medium identification unit includes:

[0049] Fourier transform module, used to perform fast Fourier transform on the current vibration signal to obtain frequency domain information;

[0050] The pattern recognition module is used to input frequency domain information into a pre-trained medium recognition model for recognition and output the writing medium type.

[0051] As described above, intelligent recognition based on frequency domain information extraction can quickly identify commonly used writing media and improve recognition efficiency and accuracy.

[0052] In a third aspect, an embodiment of the present invention provides an electronic device, comprising:

[0053] one or more processors;

[0054] a memory for storing one or more computer programs;

[0055] When one or more computer programs are executed by one or more processors, the electronic device implements the touch operation type detection method of the first aspect.

[0056] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the touch operation type detection method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 A flow chart of a method for detecting a touch operation type provided in an embodiment of the present application;

[0059] Figure 2 A schematic diagram of a writing state of an interactive tablet provided as an example in an embodiment of the present application;

[0060] Figure 3 Schematic diagram of an ideal vibration signal when a touch object comes into contact with the interactive flat panel;

[0061] Figure 4 A schematic diagram of possible vibration signals when a touch object comes into contact with the interactive flat panel;

[0062] Figure 5 A schematic structural diagram of a touch operation type detection device provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of this application more apparent, embodiments of the present application will be further described in detail below with reference to the accompanying drawings. It will be understood that the specific embodiments described herein are intended to illustrate the present invention, not to limit it. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all of the components.

[0065] It should be noted that due to space limitations, this application specification does not enumerate all optional implementation methods. After reading this application specification, those skilled in the art should be able to understand that as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method.

[0066] Each embodiment is described in detail below.

[0067] Interactive tablets are electronic devices designed and developed for multi-person communication scenarios. Multi-person communication scenarios are defined based on the number of participants and the type of behavior. Interpersonal communication itself is a relatively complex and changing behavior, and multi-person communication scenarios have different specific scenario details. For example, conference scenarios and teaching scenarios are both multi-person communication scenarios, but conference scenarios and teaching scenarios have significantly different communication characteristics. Multi-person communication scenarios can be classified based on the different role positioning methods of the members participating in the multi-person communication. They can be divided into one-way multi-person communication in which a single member (or multiple members in sequence) outputs information and other users receive the information, and multi-directional communication in which multiple members speak freely and cross-output and receive information. In one-way multi-person communication, depending on the characteristics of the communication topic, information may be presented in language or a combination of language and images.

[0068] The requirements for interactive tablets in different multi-person communication scenarios typically vary, but there may also be common needs. For example, in a multi-person communication setting like a visit, an interactive tablet may only need to present the topic of the conversation. In teaching scenarios, especially those that require concrete presentation of teaching content, the ability to input and display notes on the blackboard is a key requirement. In all multi-person communication scenarios, there may be a need to record the communication process.

[0069] To enhance the interactive experience, existing interactive tablets use various sensors to detect the specific contact state between a pen and a finger, enabling quick and accurate responses to various operations. For example, when responding to a writing operation, the vibration signal generated by the touch object in contact with the interactive tablet can be used to identify the medium type of the touch object. Different handwriting attributes can be displayed for different medium types during the writing operation. However, existing solutions can mistakenly detect multiple writing medium types during a single writing operation, resulting in sudden changes in the displayed handwriting attributes, making the handwriting display less stable and accurate.

[0070] To solve the above technical problems, the inventors analyzed the existing writing operation process and found that the existing confirmation vibration signal is the vibration signal when the writing operation occurs. The method uses a preset detection algorithm to determine whether the vibration signal is the same as the pre-recorded vibration signal when the writing operation occurs. However, in the actual collection of vibration signals, it is possible that a pulse will be generated in the subsequent vibration signal after the touch object contacts the display screen of the interactive flat panel for a period of time. The vibration signal containing the pulse is very similar to the vibration signal when the writing operation occurs, and may be determined by the detection algorithm to be a writing operation. In response to the above technical problems and the reasons for the technical problems identified by the analysis of the technical problems, the embodiment of the present application proposes a touch operation type detection method. According to the difference in subsequent changes between the vibration signal generated at the beginning of a normal writing operation and the vibration signal corresponding to the abnormal pulse, that is, the attenuation of the vibration signal generated at the beginning of a normal writing operation is gradually reduced, while the attenuation of the vibration signal corresponding to the abnormal pulse is a sudden change, after completing the preliminary confirmation of the writing operation, a subsequent vibration signal is buffered as a verification vibration signal, and the intensity change statistics of the verification vibration signal are performed. When the result of the intensity change statistics reaches a preset writing intensity threshold, the writing operation is confirmed to be detected. This makes it possible to accurately distinguish the type of writing medium that needs to be confirmed when the writing operation actually occurs from the vibration signal generated during the writing operation, eliminate false detection of the writing medium type, and accordingly avoid sudden changes in the display of handwriting attributes during the writing process, thereby improving the stability and accuracy of the handwriting display during the writing process.

[0071] The interactive tablet used in the embodiments of the present application may be an all-in-one device that uses touch technology to control content displayed on a display screen and implement human-computer interaction. It may integrate one or more functions such as a projector, electronic whiteboard, screen, audio, television, and video conferencing terminal. Of course, the interactive tablet does not include any restrictions on the surface characteristics of the display surface. For example, the surface characteristics of the interactive tablet may be flat, curved, or a combination of multiple flat surfaces.

[0072] Typically, an interactive tablet is installed with at least one operating system, including but not limited to Android, Linux, Windows, and Huawei's Hongmeng OS. This operating system is used to control and coordinate the interactive tablet and peripheral devices, allowing the various independent hardware components in the interactive tablet to work together as a stable whole. The interactive tablet includes at least one display screen. For example, the interactive tablet is equipped with a touch-enabled display screen, i.e., a touch-sensitive display screen. The touch-sensitive display screen can be an infrared display screen, a capacitive display screen, a resistive display screen, an electromagnetic display screen, or a pressure-sensitive display screen. On a touch-enabled display screen, a user can perform touch operations by touching the display screen with a finger or a stylus. Accordingly, the interactive tablet detects the touch location and responds accordingly to implement the touch function. Different touch sensor modules are used on touch-enabled displays, and the original touch signals collected by the touch sensor modules are different, and the converted touch signals are also not completely the same. In interactive tablets, to quickly access commonly used functions without exiting the current page, sidebars are usually set on both sides of the display screen. The sidebars display multiple function controls, such as a camera control for launching the camera and a writing control for quickly starting writing.

[0073] For infrared displays, the touch sensor module is an infrared touch frame. The touch signal it collects may include a signal indicating that infrared rays are blocked. The converted touch signal may include a position touch signal, which may include the X and Y coordinates of the touch location. For capacitive displays, the touch sensor module is a capacitive touch pad. The touch signal it collects may include the current flowing through the electrodes of the touch screen. The converted touch signal may include a position touch signal, which may include the X and Y coordinates of the touch location. For resistive displays, the touch sensor module is a resistive touch pad. The touch signal it collects may include the voltage at the touch location. The converted touch signal may include a position touch signal, which may include the X and Y coordinates of the touch location. For electromagnetic displays, the touch sensor module is an electromagnetic pad. The touch signal it collects may include the change in magnetic flux and the frequency of the received electromagnetic signal. The converted touch signal may include a position touch signal corresponding to the change in magnetic flux and a pressure signal corresponding to the frequency. The position touch signal may include the X and Y coordinates of the touch location; the pressure signal may include a pressure value. For pressure-sensitive displays, the touch sensor module is a pressure sensor. The touch signals it collects may include pressure signals, and the converted touch signals may include position touch signals. The position touch signals may include the X and Y coordinates of the touch location. In the embodiments of the present application, based on the user experience of the integrated arrangement of the touch sensor module and the display, the entire device is defined as a touch-sensitive display, also known as a touch screen, which can implement both touch detection and screen display.

[0074] The interactive tablet described in the embodiments of the present application may also be called an all-in-one device, a smart tablet, a conference tablet, a conference board, etc. It should be understood that electronic devices used in multi-person communication scenarios that support information, presentation, recording, and sharing functions during multi-person communication are all within the scope of the interactive tablet.

[0075] Figure 1 This is a flow chart of a method for detecting a touch operation type provided by an embodiment of the present application. Figure 1 As shown, the touch operation type detection method includes but is not limited to steps S110 to S140. The touch operation type detection method is described exemplarily by taking the operation process of a user using a touch object on an interactive tablet.

[0076] Step S110: continuously acquiring vibration signals detected by the elastic wave detection device installed on the touch screen, and performing operation type detection on the acquired current vibration signals.

[0077] The interactive tablet applicable to the touch operation type provided in the embodiment of the present application includes, in addition to the touch screen described above, an elastic wave detection device.

[0078] In the embodiments of this application, the interactive tablet detects touch operations not only by detecting touch position information but also by detecting vibration signals generated by the contact between the touch object and the physical surface (e.g., touch screen) used to present the interactive tablet's display interface, i.e., acquiring elastic wave signals. Mechanical vibrations below audible frequency, sounds in the audible range, and ultrasonic waves above audible frequency are all wave phenomena in media such as gases, liquids, and solids. Compared to light and electromagnetic waves, these wave phenomena are called elastic waves.

[0079] During the implementation of this solution, elastic wave signals are detected using elastic wave sensors (e.g., piezoelectric ceramic sensors) within the elastic wave detection device. These sensors are specifically installed in locations where they can transmit vibrations generated by the touch screen, thereby detecting when a touch object on the touch screen touches the touch screen. The elastic wave sensors do not necessarily need to be installed on the touch screen, as long as they are able to detect vibrations generated when the touch object contacts the touch screen during a touch operation. The number of elastic wave sensors deployed can be tailored to the size of the touch screen and the accuracy of detection. Generally speaking, the larger the touch screen, the higher the required accuracy, and the more elastic wave sensors are required. The elastic wave sensors can be installed directly on the surface of the touch screen, for example, on the top or bottom surface of the touch screen, to receive vibrations transmitted by the touch screen and improve the accuracy of touch detection. The elastic wave sensors can also be installed within the touch screen's frame to minimize impact on the internal structure and reduce common-mode noise interference from the touch screen. Of course, the elastic wave sensors can also be installed on other components in contact with the touch screen, receiving vibrations generated on the touch screen through transmission from other components and generating corresponding vibration signals.

[0080] Elastic wave sensors can all passively detect elastic waves, or one or more of them can actively excite elastic waves outward. The excited elastic waves can be detected by all elastic wave sensors. When there is an external touch on the touch screen, additional elastic waves will be generated and detected by all elastic wave sensors. The system can determine the medium type corresponding to the external touch based on the signal generated by the combined effect of multiple elastic waves.

[0081] When an object touches the surface of the touch screen (including point touch and slide), a characteristic elastic wave is generated. The elastic wave starts from the contact point and propagates around the touch screen or into the inside of the touch screen. The elastic wave sensor located at the screen frame or inside the touch screen can convert the vibration signal into a sensing signal. The sensing signal is transmitted to a processor with temperature compensation for amplification and conversion into a digital elastic wave signal. Here, the sensing signal includes a voltage signal, a current signal or a magnetic flux signal. In the process of detecting the sensing signal, the elastic wave sensor determines whether it is a valid touch signal based on the strength of the sensing signal and filters out the noise signal.

[0082] Taking voltage signals as an example, when an object contacts a touchscreen, the voltage signal obtained by the elastic wave sensor is scanned at a certain frequency f. When the voltage value exceeds a first voltage threshold V1, it is recorded as a valid piezoelectric value above noise; otherwise, it is recorded as zero voltage. In the specific judgment process, the first voltage threshold V1 can vary with temperature because circuit noise varies with temperature, and the coupling coefficient of the piezoelectric material also changes with temperature. K consecutively recorded voltage signals constitute an elastic wave signal segment, with a duration of T0 = K / f. Within this segment, if none of the K signals in the segment exceeds the first voltage threshold V2, the signal strength does not reach a valid touch strength and is judged as a noise signal and discarded. Conversely, if one of the K voltage signals exceeds the first voltage threshold V2, the signal strength reaches a valid touch strength and is recorded as a valid touch signal.

[0083] Of course, depending on the different detection methods of the elastic wave sensor, the signal type converted from the elastic wave is different. For example, it can also be a change in magnetic flux, and then the corresponding digital information is generated according to the change in magnetic flux to realize the detection of the elastic wave.

[0084] According to the sensing signal detected by the elastic wave sensor, it can be known that at time t i At the moment of touch, the center position and contact shape of the touch object. Combined with the temperature information, the speed of different elastic waves propagating in the touch screen can be obtained. For example, the speed of the surface wave is v a (T), which is a function of temperature T. The wave velocity may be anisotropic (for example, the anisotropy of the touch screen material, or the anisotropy caused by different internal stresses). Based on the position information, it can be concluded that the distance between the touch object and an elastic wave sensor is S k , then the elastic wave signal of the touching object is i +S k / v a ) at the same time; and the intensity of the elastic wave will decay with distance. The specific attenuation rate is also related to the temperature of the touch screen and the frequency of the elastic wave itself. Generally speaking, the higher the temperature of the touch screen, the faster the attenuation rate; the higher the frequency, the faster the attenuation rate. By the same token, it can be concluded that the signal reception time and signal strength of other elastic wave sensors are related. In the specific implementation process, the touching object directly contacts the cover of the touch screen. The characteristics of the elastic wave generated when the touching object contacts the touch screen are mainly affected by the cover. When confirming the type of medium based on the elastic wave, the information or parameters of various aspects of the cover are mainly considered.

[0085] All elastic wave signals obtained by the elastic wave sensor can be filtered out based on their time, frequency, and other characteristic parameters. First, there are vibration signals that originate outside of touch behavior. These signals, such as those from external vibrations, internal speakers, or the frame, all generate some vibration. With the help of the signals detected by the elastic wave sensor, the time period during which valid signals occur can be determined. Elastic wave signals in other time periods are considered non-touch signals. Removing non-valid touch signals from the piezoelectric sensor in different time periods facilitates subsequent object identification and judgment. Secondly, there are signals that originate from touch behavior but for which the characteristics of the touched object have not been previously recorded. For interactive touch gestures or other object touches, there is a known characteristic frequency distribution. For example, when a finger touches or slides with a stylus or eraser, as supported by the system, the characteristic frequency is between 1kHz and 20kHz, while signals at other frequencies are considered noise or unsupported signals. Therefore, a fast Fourier transform can be performed on the voltage signal to filter out the unsupported frequency ranges and remove the signal range.

[0086] It should be noted that the elastic wave sensor in the same location can be composed of one or more polarization sensors. For example, with two polarization directions in the + / -Z direction, a differential circuit can be used to calculate the difference between the two piezoelectric signals, amplifying the signal and reducing changes in the piezoelectric coefficient caused by temperature and other factors. For another example, one or more piezoelectric sensors with polarization directions in the XY plane can sense elastic wave signals in an additional dimension, detect the propagation of different elastic waves, increase the amount of information (facilitating multi-point identification and judgment), and add waterproofing (taking advantage of the low shear modulus of water to detect the characteristic signals of tangential waves) or other stain detection functions.

[0087] At this point, the elastic wave sensor can output a valid elastic wave signal, which is recorded in the form of a corresponding voltage signal, in which information such as amplitude, frequency, and phase can be confirmed.

[0088] Elastic waves can be generated by various noise-generating components within the interactive tablet (such as motors, speakers, and fans) as well as external noise-generating devices (such as air conditioners and televisions). These components can cause the interactive tablet to generate elastic waves unintended by the user during use. The type of component can be used to identify how it generates noise. For example, the noise generated by a speaker primarily comes from elastic waves generated within the interactive tablet's overall dielectric medium when it emits sound. Elastic wave sensors can readily detect various elastic wave signals, but many of these signals are not derived from touch operations themselves.

[0089] For these components or devices, the elastic wave signals generated by the components or devices under different driving voltages within a preset time period can be pre-collected, and the electrical signals corresponding to the elastic wave signals can be converted into noise signals. After filtering out all the signals detected by the elastic wave sensor and eliminating noise interference, an effective elastic wave signal can be obtained. In the process of filtering out noise, considering that the interference generated by the noise signal is not the same, the signal generated by the touch operation itself can be adaptively amplified or reduced with reference to the noise signal to highlight the difference between the noise signal and the signal generated by the touch operation itself.

[0090] Considering that when the noise generated by a noise source such as a speaker is relatively mild, the elastic wave signal generated by the touch object when it is touched will be significantly higher than the elastic wave signal (i.e., the noise signal) generated by the speaker and other noise sources. In this case, the difference between the two is not obvious. By setting a voltage range to limit the voltage threshold, it is easier to identify the elastic wave band generated by the noise and remove it, thereby reducing the amount of calculation, effectively improving the efficiency of filtering out the noise, and quickly obtaining the signal corresponding to the touch operation itself, i.e., the elastic wave signal. Specifically, within the range formed by the voltage threshold, all detected elastic wave signals and noise signals are phase-matched, and signal data that is consistent with the phase of the noise signal is removed, reducing unnecessary calculations during actual calculations. For example, when the noise generated by a noise source such as a speaker is relatively mild, the touch signal generated by the user's touch operation will be significantly higher than the noise signal generated by the speaker and other noise sources. In this case, when comparing the two, the difference is not obvious. By limiting the voltage threshold, it is easier to identify the band generated by the noise and remove it, thereby effectively improving the efficiency of noise removal. During the filtering process, all elastic wave signals and noise signals can be compared in phase. If the elastic wave signal is consistent with the noise signal in phase, the signal data in the elastic wave signal that is consistent with the noise signal in phase will be deleted. If the elastic wave signal is opposite to the noise signal in phase, the signal data in the elastic wave signal that is opposite to the noise signal in phase will be retained. This avoids the error caused by direct subtraction when the band of the noise signal is opposite to the band of the elastic wave signal, and retains the signal in the elastic wave signal that is actually derived from the touch operation. The elastic wave itself is a type of vibration. In the scenario targeted by the embodiment of the present application, the vibration actually exists in the form of an elastic wave, so the definition of the elastic wave signal is equivalent to a vibration signal.

[0091] Based on the above basic hardware design and signal processing of elastic wave detection, such as Figure 2 As shown, when a touch object 20 made of different materials is operated on the interactive tablet 10, not only the touch position can be detected by the touch sensor module, but also the elastic wave detection device can detect the elastic wave signals corresponding to different materials, thereby identifying the type of medium made of the touch object 20.

[0092] During the specific processing process, vibration signals are acquired in frames, and each frame contains multiple vibration signals. Acquiring and processing vibration signals in frames can effectively improve the orderliness of vibration processing, avoiding the situation where there is too little data to make accurate judgments or too much data to process, which takes too long. The number of vibration signals contained in each frame is the same as the number of verification vibration signals. During subsequent verification, the verification of the writing operation is completed directly through the single-frame vibration signal to ensure processing efficiency. Each frame includes 600 vibration signals. 600 vibration signals can control the balance between processing accuracy and processing delay as much as possible. Of course, other numbers of vibration signals can also achieve basic detection and verification, such as 557 vibration signals, 680 vibration signals, etc.

[0093] Step S120 : When it is confirmed that the operation type of the current vibration signal is detected as a pen-down operation, a preset number of vibration signals subsequent to the current vibration signal are cached as verification vibration signals.

[0094] The current vibration signal is compared with a pre-set reference using a preset detection algorithm to determine whether it represents a pen-down operation. A pen-down operation refers to the initial contact between the touch object and the touchscreen during writing. At this point, the vibration signal changes significantly, as the state between the touch object and the touchscreen changes from non-contact to contact, making it easier to determine whether a pen-down operation has occurred. The specific detection method is already implemented in related technologies and will not be explained here.

[0095] After an in-depth analysis of the vibration signals during the entire writing process, it was found that under normal conditions, such as Figure 3 As shown in Figure 2, the vibration signal has a relatively obvious initial fluctuation sequence 21 when writing, and then gradually decays until it remains basically stable. However, in the actual writing process, it may also be as follows Figure 4 As shown, at the beginning of writing, there is a relatively obvious initial wave sequence 21 and gradual decay. After maintaining stability for a certain period of time, an intermediate wave sequence 22 appears again and quickly decays to a stable state. The false detection in the prior art is due to the appearance of the intermediate wave sequence 22, and the intermediate wave sequence 22 may be identified as another type of touch object to start a writing operation, thereby causing the user's actual writing operation to change the color of the handwriting to be displayed. Based on the above findings, in order to eliminate false detections in writing operations and maintain the stability and accuracy of handwriting display during the writing process, a preset number of vibration signals subsequent to the current vibration signal are further cached as verification vibration signals, and the verification vibration signals are used to further verify the current vibration signal. It should be understood that the current vibration signal is not the vibration signal cached when the verification vibration signal is received, but refers to the vibration signal corresponding to the initial confirmation of the pen-down operation, and subsequent judgments are all processed with this sequence of vibration signals as the center. In addition, Figure 3 and Figure 4It is only used to illustrate the overall change trend of the vibration signal and does not represent the actual vibration signal in the process of processing. Figure 3 as well as Figure 4 The vibration signals shown are identical.

[0096] Step S130: performing intensity change statistics on the verification vibration signal according to the fluctuation of the verification vibration signal.

[0097] contrast Figure 3 and Figure 4 It can be found that the vibration signal after the initial wave sequence 21 gradually decays until it remains basically stable. Compared with the stable state, the amplitude of the vibration signal immediately after the initial wave sequence 21 is still relatively large, while the intermediate wave sequence 22 quickly remains stable without a gradual decay transition. The vibration signal immediately after the intermediate wave sequence 22 can be directly regarded as a vibration signal in a stable state. This fluctuation amplitude compared to the stable state is defined as an intensity change. The overall evaluation of the fluctuation of the verification wave signal is the intensity change statistic. In the embodiment of the present application, any current vibration signal that may be identified as a pen-down operation is subsequently judged to further confirm whether the "pen-down operation" is derived from the pen-down behavior of a real writing operation.

[0098] During the specific statistical process, the vibration signal before the writing operation begins, i.e., before the pen is confirmed to have been put down, can be used as a reference for intensity change statistics. Alternatively, the average signal strength value of the verification vibration signal can be determined; then, the absolute value of the difference between the signal strength value of each verification vibration signal and the average value can be calculated as the result of the intensity change statistics. Regardless of the method used, the basic principle of the statistics is to accumulate the absolute fluctuation of each vibration signal, i.e., the absolute value of the difference between the vibration signal and the reference value, as a representation of the overall fluctuation of the verification vibration signal.

[0099] Step S140: When the result of the intensity change statistics reaches a preset writing intensity threshold, it is confirmed that the writing operation is detected.

[0100] By judging whether the result of the intensity change statistics is greater than the preset writing intensity threshold, in the case that each frame includes 600 vibration signals, through multiple acquisition calculations and verification, the preset writing intensity threshold is set to 90000 as an example here. Of course, in the actual processing process, it can be adjusted according to the number of vibration signals or other physical factors, and the writing intensity threshold can be used to judge whether the pen-down operation is a pen-down operation corresponding to the real writing behavior, or a false detection caused by the pulse signal.

[0101] If the intensity change statistics reach a preset writing intensity threshold, after confirming detection of a writing operation, the system further includes identifying the type of writing medium corresponding to the writing operation based on the current vibration signal. Identifying the writing medium type based on the current vibration signal allows for distinguishing handwriting produced by different touch objects during the writing process, improving interaction quality.

[0102] The specific recognition process can involve comparing the current vibration signal with pre-stored media classification information to determine the corresponding media type of the touched object. Alternatively, the current vibration signal can be subjected to a Fast Fourier Transform to obtain frequency domain information. This frequency domain information is then input into a pre-trained media recognition model for identification, which then outputs the writing medium type. Intelligent recognition based on frequency domain information extraction can quickly identify commonly used writing media, improving recognition efficiency and accuracy.

[0103] Comparison with pre-stored media classification information can be performed directly against one or more set attribute information, such as amplitude, frequency, and phase. When the matching degree of the compared attribute information reaches a set threshold, the vibration signal is confirmed to be the corresponding media type. The comparison method for one vibration signal and multiple vibration signals is the same, that is, each is compared with the pre-stored media classification information one by one until a match is successful or no match is found after all comparisons. Alternatively, multiple media types and reference vibration signals corresponding to various media types can be pre-stored to compare and confirm the similarity between the current vibration signal and each reference vibration signal. The media type corresponding to the reference vibration signal that has the highest similarity to the current vibration signal and reaches the preset similarity threshold is confirmed as the media type corresponding to the touch object.

[0104] For classification using a media identification model, after Fourier transforming to obtain frequency domain information, the frequency domain information is used as input to a machine learning or / and deep learning media identification model, which outputs a classification function value of 0, 1, 2, 3, 4..., where 0 represents an unsupported object, 1 represents the pad of a finger, 2 represents pen A, 3 represents pen B, 4 represents an eraser, and so on. The specific format or value of the classification function value can be set arbitrarily based on the actual model. For example, in this embodiment, 0, 1, 2, 3, 4... can also be set to the format of 000, 001, 010, 011, 100..., or to other values ​​such as 10, 11, 12, 13, 14... In addition to outputting a clear classification result, the media identification model can also further indicate the probability of each classification. For example, if the elastic wave signal detected during a touch operation is used to determine the media type, the probability of pen A is 85%, the probability of pen B is 13%, and the probability of a finger pad is 2%. Based on the probability, the media type can be determined to be pen A.

[0105] In the above-mentioned touch operation type detection method, the vibration signal detected by the elastic wave detection device installed on the touch screen is continuously obtained, and the operation type detection is performed on the current vibration signal obtained; when it is confirmed that the operation type of the current vibration signal is detected as a pen-down operation, a preset number of vibration signals subsequent to the current vibration signal are cached as verification vibration signals; the intensity change statistics of the verification vibration signal are performed based on the fluctuation of the verification vibration signal; when the result of the intensity change statistics reaches the preset writing intensity threshold, it is confirmed that the writing operation is detected. According to the difference between the subsequent changes of the vibration signal generated at the beginning of a normal writing operation and the vibration signal corresponding to the abnormal pulse, that is, the attenuation of the vibration signal generated at the beginning of a normal writing operation is gradually reduced, and the attenuation of the vibration signal corresponding to the abnormal pulse is a sudden change, after completing the preliminary confirmation of the writing operation, a subsequent vibration signal is continuously cached as a verification vibration signal, and the intensity change statistics of the verification vibration signal are performed. When the result of the intensity change statistics reaches the preset writing intensity threshold, it is confirmed that the writing operation is detected. This makes it possible to accurately distinguish the type of writing medium that needs to be confirmed when the writing operation actually occurs from the vibration signal generated during the writing operation, eliminate false detection of the writing medium type, and accordingly avoid sudden changes in the display of handwriting attributes during the writing process, thereby improving the stability and accuracy of the handwriting display during the writing process.

[0106] Figure 5 This is a structural diagram of a touch operation type detection device provided in an embodiment of the present application. Figure 5 As shown, the touch operation type detection device includes an acquisition detection unit 310 , a signal buffer unit 320 , a signal statistics unit 330 and a writing confirmation unit 340 .

[0107] Among them, the acquisition detection unit 310 is used to continuously acquire the vibration signal detected by the elastic wave detection device installed on the touch screen, and perform operation type detection on the current vibration signal acquired; the signal cache unit 320 is used to cache a preset number of vibration signals subsequent to the current vibration signal as verification vibration signals when it is confirmed that the operation type of the current vibration signal is detected as a pen-down operation; the signal statistics unit 330 is used to perform intensity change statistics on the verification vibration signal according to the fluctuation of the verification vibration signal; the writing confirmation unit 340 is used to confirm that a writing operation is detected when the result of the intensity change statistics reaches a preset writing intensity threshold.

[0108] Based on the above embodiment, the signal statistics unit 330 includes:

[0109] An average value confirmation module is used to confirm and verify the average value of the signal strength value of the vibration signal;

[0110] The difference statistics module is used to count the sum of the absolute values ​​of the differences between the signal strength value of each verification vibration signal and the average value as the result of the strength change statistics.

[0111] Based on the above embodiment, the vibration signal is acquired in frames, and each frame contains multiple vibration signals.

[0112] Based on the above embodiment, the number of vibration signals included in each frame is the same as the number of verification vibration signals.

[0113] Based on the above embodiment, each frame includes 600 vibration signals.

[0114] Based on the above embodiment, the touch operation type detection device further includes:

[0115] The medium identification unit is used to identify the type of writing medium corresponding to the writing operation according to the current vibration signal.

[0116] Based on the above embodiment, the medium identification unit includes:

[0117] Fourier transform module, used to perform fast Fourier transform on the current vibration signal to obtain frequency domain information;

[0118] The pattern recognition module is used to input frequency domain information into a pre-trained medium recognition model for recognition and output the writing medium type.

[0119] The touch operation type detection device provided in the embodiment of the present application is included in an electronic device and can be used to execute the corresponding audio and video decoding and output method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0120] It is worth noting that in the embodiment of the above-mentioned touch operation type detection device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0121] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device includes a processor 410 and a memory 420. In an optional product form of the electronic device, it may further include an input device 430, an output device 440, and a communication device 450. The number of processors 410 in the electronic device may be one or more. Figure 6 In the figure, a processor 410 is used as an example; the processor 410, the memory 420, the input device 430, the output device 440 and the communication device 450 in the electronic device can be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0122] Memory 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the touch operation type detection method in the embodiments of the present application. Processor 410 executes the software programs, instructions, and modules stored in memory 420 to execute various functional applications and data processing of the electronic device, thereby implementing the aforementioned touch operation type detection method.

[0123] The memory 420 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 420 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 420 may further include a memory remotely located relative to the processor 410, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0124] The input device 430 may be used to receive network configuration information, user operations, etc. The output device 440 may include a display device such as a display screen.

[0125] The electronic device described above can be used to execute any touch operation type detection method and has corresponding functions and beneficial effects.

[0126] An embodiment of the present invention also provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are used to perform relevant operations in the touch operation type detection method provided in any embodiment of the present application, and have corresponding functions and beneficial effects.

[0127] Those skilled in the art should understand that the embodiments of the present application may provide methods, systems, or computer program products.

[0128] Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0129] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-permanent storage in a computer-readable medium, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0130] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of other identical elements in the process, method, commodity, or apparatus comprising the element.

[0132] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A touch operation type detection method, characterized in that: include: Continuously acquiring vibration signals detected by an elastic wave detection device installed on the touch screen, and performing operation type detection on the acquired current vibration signals; If the operation type of the current vibration signal is determined to be a pen-down operation, buffering a preset number of vibration signals subsequent to the current vibration signal as verification vibration signals; Performing intensity change statistics on the verification vibration signal according to the fluctuation of the verification vibration signal; When the result of the intensity change statistics reaches a preset writing intensity threshold, it is confirmed that the writing operation is detected.

2. The touch operation type detection method according to claim 1, wherein: The performing intensity change statistics on the verification vibration signal according to the fluctuation of the verification vibration signal includes: Determining an average value of the signal strength values ​​of the verification vibration signal; The sum of the absolute values ​​of the differences between the signal strength value of each verification vibration signal and the average value is counted as the result of the intensity change statistics.

3. The touch operation type detection method according to claim 1 or 2, characterized in that: The vibration signal is acquired in frames, and each frame contains multiple vibration signals.

4. The touch operation type detection method according to claim 3, wherein: The number of vibration signals contained in each frame is the same as the number of the verification vibration signals.

5. The touch operation type detection method according to claim 3, wherein: Each frame includes 600 vibration signals.

6. The touch operation type detection method according to claim 1 or 2, characterized in that: When the result of the intensity change statistics reaches a preset writing intensity threshold, after confirming that a writing operation is detected, the method further includes: A writing medium type corresponding to the writing operation is identified according to the current vibration signal.

7. The touch operation type detection method according to claim 6, characterized in that: The identifying the type of writing medium corresponding to the writing operation according to the current vibration signal includes: Performing a fast Fourier transform on the current vibration signal to obtain frequency domain information; The frequency domain information is input into a pre-trained medium recognition model for recognition, and the writing medium type is output.

8. A touch operation type detection device, characterized in that: include: An acquisition detection unit, configured to continuously acquire a vibration signal detected by an elastic wave detection device installed on the touch screen, and perform operation type detection on the acquired current vibration signal; a signal buffer unit, configured to, upon confirming that the operation type of the current vibration signal is detected as a pen-down operation, buffer a preset number of vibration signals subsequent to the current vibration signal as verification vibration signals; a signal statistics unit, configured to perform intensity change statistics on the verification vibration signal according to fluctuations of the verification vibration signal; The writing confirmation unit is configured to confirm that a writing operation is detected when the result of the intensity change statistics reaches a preset writing intensity threshold.

9. An electronic device, characterized in that include: one or more processors; a memory for storing one or more computer programs; When the one or more computer programs are executed by the one or more processors, the electronic device implements the touch operation type detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the touch operation type detection method according to any one of claims 1 to 7 is implemented.

Images