A touch operation processing method, device, interactive tablet and storage medium

Abstract

The embodiment of the present application discloses a touch operation processing method, device, interactive tablet and storage medium, which includes: receiving touch sensing information corresponding to the writing operation input by the touch pen on the electronic whiteboard, extracting the touch track information detected by the touch sensing module and the elastic wave characteristic information detected by the elastic wave sensor in the touch sensing information; determining the material information of the touch pen according to the elastic wave characteristic information, and generating the corresponding writing handwriting according to the touch track information; comparing the material information of the touch pen with the preset target material information to determine whether the two material information match; if the two material information match, performing text recognition on the writing handwriting, obtaining the text content corresponding to the writing handwriting, and displaying the text content on the electronic whiteboard; if the two material information do not match, displaying the writing handwriting on the electronic whiteboard. The above technical means are adopted to solve the technical problem of high reliability and low cost of touch recognition in the prior art.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of electronic whiteboards, and in particular to a touch operation processing method, device, interactive tablet, and storage medium. Background Art

[0002] With the continuous advancement of science and technology, many electronic devices with handwriting input systems have begun to enter the market, such as smart terminals such as mobile phones, e-books and tablet computers with handwriting functions. In order to enrich the application of handwriting functions in smart terminals, electronic whiteboard applications have come into being. Electronic whiteboards are an application that displays corresponding text or pictures according to the touch operations input by users on the screen. In the prior art, in order to improve the convenience of electronic whiteboard operation, different touch pens are set to trigger different operations, and it is necessary to identify the touch pen corresponding to the touch operation. The traditional touch pen recognition method is to use a touch pen that can sense, and the touch pen is provided with a receiving head or embedded receiving hardware, and report information to the operating system through short-distance communication, such as Bluetooth, so that the operating system can identify which touch pen triggers the current touch operation based on the reported information, and then distinguish the touch pen of the touch operation, so as to achieve different operations.

[0003] The inventors found that the traditional touch pen recognition method relies more on the communication device installed on the touch pen, which has high requirements for the touch pen and high cost. Once the touch pen runs out of power or the communication device is damaged, the electronic whiteboard cannot be operated. The reliability is low and the user experience is poor. Summary of the invention

[0004] The embodiments of the present application provide a touch operation processing method, device, interactive tablet and storage medium, which solve the technical problem of high reliability and low cost of touch pen recognition methods in the prior art and improve user experience.

[0005] In a first aspect, an embodiment of the present application provides a touch operation processing method, including:

[0006] Receiving touch sensing information corresponding to a writing operation input by a touch pen on an electronic whiteboard, and extracting touch track information detected by a touch sensing module and elastic wave feature information detected by an elastic wave sensor from the touch sensing information;

[0007] Determine the material information of the touch pen according to the elastic wave characteristic information, and generate corresponding writing handwriting according to the touch track information;

[0008] Compare the touch pen material information with preset target material information to determine whether the two material information match;

[0009] If the two types of material information match, performing text recognition on the handwriting, obtaining text content corresponding to the handwriting, and displaying the text content on the electronic whiteboard;

[0010] If the two types of material information do not match, the handwriting is displayed on the electronic whiteboard.

[0011] In a second aspect, an embodiment of the present application provides a touch operation processing device, including:

[0012] A receiving module is configured to receive touch sensing information corresponding to a writing operation input by a touch pen on the electronic whiteboard, and extract touch track information detected by the touch sensing module and elastic wave feature information detected by the elastic wave sensor from the touch sensing information;

[0013] An information determination module, configured to determine the touch pen material information according to the elastic wave characteristic information, and generate corresponding writing handwriting according to the touch track information;

[0014] A comparison module, configured to compare the touch pen material information with preset target material information to determine whether the two material information match;

[0015] A text display module is configured to perform text recognition on the handwriting if the two material information match, obtain text content corresponding to the handwriting, and display the text content on the electronic whiteboard;

[0016] The handwriting display module is configured to display the written handwriting on the electronic whiteboard if the two material information do not match.

[0017] In a third aspect, an embodiment of the present application provides an interactive tablet, including:

[0018] one or more processors;

[0019] A memory for storing one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the touch operation processing method as described in the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the touch operation processing method as described in the first aspect.

[0022] The above-mentioned touch operation processing method, device, interactive tablet and storage medium receive touch sensing information corresponding to the writing operation input by the touch pen on the electronic whiteboard, extract the touch trajectory information detected by the touch sensing module and the elastic wave characteristic information detected by the elastic wave sensor in the touch sensing information; determine the material information of the touch pen according to the elastic wave characteristic information, and generate the corresponding writing handwriting according to the touch trajectory information; compare the material information of the touch pen with the preset target material information to determine whether the two material information match; if the two material information match, perform text recognition on the writing handwriting, obtain the text content corresponding to the writing handwriting, and display the text content on the electronic whiteboard; if the two material information do not match, display the writing handwriting on the electronic whiteboard. By adopting the above technical means, based on the different elastic wave characteristic information generated by touch pens of different materials, the material information of the touch pen can be identified through the elastic wave characteristic information of the touch pen. The user can pre-set the target material information. If the material information of the touch pen matches the target material information, the handwriting of the touch pen is converted into text content and displayed on the electronic whiteboard. If it does not match, the handwriting is displayed on the electronic whiteboard. That is, the matching operations of touch pens of different materials are pre-set, and the material information is determined by the elastic wave signal of the touch pen, thereby determining the corresponding operation of the touch pen, so as to realize different touch pens triggering different operations. In addition, the touch pen does not need to install communication equipment, has high reliability and low cost, and can effectively improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a touch operation processing method provided by an embodiment of the present application;

[0024] Figure 2 is a first schematic diagram of an electronic whiteboard provided in an embodiment of the present application;

[0025] Figure 3 is a second schematic diagram of the electronic whiteboard provided in an embodiment of the present application;

[0026] Figure 4 is a third schematic diagram of an electronic whiteboard provided in an embodiment of the present application;

[0027] Figure 5 is a fourth schematic diagram of an electronic whiteboard provided in an embodiment of the present application;

[0028] Figure 6 is a flow chart of another touch operation processing method provided by an embodiment of the present application;

[0029] Figure 7 is a flow chart of another touch operation processing method provided by an embodiment of the present application;

[0030] Figure 8is a structural schematic diagram of a touch operation processing device provided by an embodiment of the present application;

[0031] Fig. 9 It is a structural schematic diagram of an interactive tablet provided by an embodiment of the present application. DETAILED DESCRIPTION

[0032] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0033] It should be noted that, in the present application, relational terms such as first and second are only used to distinguish one entity or operation or object from another entity or operation or object, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations or objects. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0034] The touch operation processing method provided in the embodiment of the present application can be executed by an interactive tablet with a touch function. The interactive tablet can operate the content displayed on the display screen and realize human-computer interaction through touch technology. It integrates one or more functions such as a projector, an electronic whiteboard, a screen, an audio system, a television, and a video conferencing terminal. In actual applications, the hardware part of the interactive tablet is composed of parts such as a display screen and an intelligent processing system, which are combined together by integral structural parts and are also supported by a dedicated software system, wherein the display screen has a touch function. In the embodiment, the display screen displays an electronic whiteboard, and the user can touch the display screen with a finger or a touch pen. The intelligent processing system generates a touch operation according to the touch operation input by the user, generates handwriting or text content corresponding to the handwriting according to the touch operation, and displays the handwriting or text content on the electronic whiteboard.

[0035] Typically, the smart interactive tablet is installed with at least one type of operating system, wherein the operating system includes but is not limited to the Android system, the Linux system and the Windows system. The touch operation received through the display screen is processed by the operating system. Furthermore, the smart interactive tablet can install at least one application based on the operating system. In the embodiment, the electronic whiteboard application is used as an example description. For example, an electronic whiteboard application is installed in the smart interactive tablet. Among them, the application can be an application that comes with the operating system, or it can be an application downloaded from a third-party device or server. Optionally, under the electronic whiteboard function, functions such as writing, inserting tables, inserting pictures, inserting multimedia, inserting files (such as PPT, etc.), playing multimedia, inserting graphics, drawing tables, etc. can be realized.

[0036] Figure 1 is a flow chart of a touch operation processing method provided by an embodiment of the present application. Figure 1 , the touch operation processing method includes:

[0037] S110, receiving touch sensing information corresponding to the writing operation input by the touch pen on the electronic whiteboard, and extracting touch track information detected by the touch sensing module and elastic wave feature information detected by the elastic wave sensor from the touch sensing information.

[0038] For example, when a user uses a touch pen to input a writing operation on the touch interface of a display screen, the touch sensing module and elastic wave sensor installed on the display screen detect the writing operation input by the user to generate touch sensing information, and send the touch sensing information to the processor to inform the processor to process the writing operation.

[0039] Among them, the touch sensing information includes elastic wave characteristic information, and the elastic wave characteristic information is measured by a piezoelectric elastic wave sensor installed at the screen frame or on the inner side of the screen cover. Exemplarily, when the touch pen touches the screen cover, a characteristic elastic wave signal is generated, and the elastic wave signal starts from the contact point and propagates around along the screen cover, or propagates to the inner side of the screen cover. The piezoelectric elastic wave sensor located at the screen frame or on the inner side of the cover converts the elastic wave signal into a voltage signal, and then transmits the voltage signal to an IC chip with a temperature compensation circuit for amplification processing, and converts the amplified voltage signal into a digital voltage signal through an analog-to-digital conversion circuit, and uses the digital voltage signal as the elastic wave characteristic information. The processor analyzes and processes the elastic wave characteristic information to determine whether the elastic wave characteristic information corresponds to the touch pen material information.

[0040] Further, the touch sensing information also includes touch trajectory information, which is measured by the optical touch sensors installed on both sides of the display screen surface about the trajectory information of the user moving the touch pen on the display screen, which specifically includes the contact area between the touch pen and the touch interface, the moving speed of the touch pen on the touch interface, and the contact position when the touch pen contacts the touch interface. Exemplarily, the optical touch sensors are arranged on both sides of the display screen surface, and the optical touch sensors include an infrared transmitter and an infrared receiver, the infrared transmitter is used to transmit infrared light, and the infrared receiver is used to receive infrared light, and the light beam grid formed by the dense infrared light in different directions is used to locate the contact position of the touch pen contacting the screen cover. M infrared transmitters are arranged on one side of the display screen in the horizontal direction, and N infrared transmitters are arranged on one side of the vertical direction. Correspondingly, M infrared receivers are arranged on the other side in the horizontal direction, and N infrared receivers are arranged on the other side in the vertical direction. The infrared transmitter transmits infrared light at a certain frequency, and the corresponding infrared receiver receives infrared light at a certain frequency. When the touch pen touches the screen cover, the touch pen will completely or partially block one or more vertical and horizontal infrared lights, and then obtain an M*N infrared light intensity pixel map. First, find the position greater than the first light intensity threshold on the infrared light intensity pixel map. The first light intensity threshold represents a real and effective touch, rather than noise or the light intensity measured when the infrared light is half-blocked when an object is close but not completely touched. The real touch position is the contact position when the touch pen touches the screen cover. Further, find the position greater than the second light intensity threshold in the vicinity of the real touch position. The second light intensity threshold is greater than the noise value, which can represent the extension of the real touch area. The position greater than the second light intensity threshold in the vicinity of the real touch position is marked as a valid touch area. The area of ​​the touch area is the contact area between the touch pen and the screen cover. When the touch pen moves on the screen cover, the infrared light intensity pixel map with multiple frames of time continuous records the contact position and contact area of ​​the touch pen on the screen cover. The moving speed of the touch pen on the screen cover can be obtained through the contact position and contact area recorded in the continuous frames. When the touch pen leaves the screen cover, the touch pen's obstruction of infrared light becomes smaller. If the contact position in the previous frame of the infrared light intensity pixel map, the relative position in the current infrared light intensity pixel map, and the adjacent area do not have pixels exceeding the third light intensity threshold, then it is determined that the relationship between the touch pen and the screen cover at this time is out of touch, and the third light intensity threshold represents the maximum light intensity when the touch is out of touch.

[0041] S120: Determine the material information of the touch pen according to the elastic wave characteristic information, and generate corresponding writing handwriting according to the touch trajectory information.

[0042] It can be understood that in order to achieve the effect of different touch pens triggering different operations, it is necessary to distinguish the touch pens corresponding to the touch operation to determine the operation triggered by the touch pen. In this regard, in order to distinguish the touch pens corresponding to the touch operation, the traditional touch pen identification method is to identify the touch pen corresponding to the touch operation by reporting touch information through the communication device installed on the touch pen. As can be seen from the above content, this embodiment can detect the elastic wave signal generated when the touch pen triggers the corresponding touch operation through the piezoelectric elastic wave sensor. Based on the different characteristics of the elastic force of the interaction force between the particles in the elastic medium, it can be concluded that the elastic wave signals generated when the touch pens of different materials touch the screen cover are different. Therefore, based on the different elastic wave signals generated by the touch pens of different materials, the corresponding elastic wave feature information is also different, and the touch pens of different materials can be distinguished by the elastic wave feature information. Further, the elastic wave feature information is analyzed and processed to obtain the amplitude and phase information of the elastic wave signal. The amplitude and phase information of the elastic wave signal reflect the elastic force of the interaction force between the particles in the touch pen, so the amplitude and phase information of the elastic wave signal are used to characterize the material information of the touch pen.

[0043] After obtaining the contact area between the touch pen and the touch interface, the moving speed of the touch pen on the touch interface, and the contact position when the touch pen contacts the touch interface, the corresponding writing handwriting is generated based on this information. Exemplarily, the coordinates of the touch pen touching the screen cover are determined based on the contact position, and the coordinates of the track point corresponding to the writing handwriting on the electronic whiteboard are determined based on the coordinates, and the rendering area of ​​the corresponding track point on the electronic whiteboard is determined based on the contact area corresponding to the contact position. According to the moving speed of the touch pen between two adjacent contact positions, the connection track between the track points corresponding to the two contact positions is determined. According to the track point coordinates, the track point rendering area and the connection track, the corresponding writing handwriting is generated.

[0044] S130, comparing the touch pen material information with the preset target material information to determine whether the two material information match. If the two material information match, execute step S140; if the two material information do not match, execute step S150.

[0045] Exemplarily, in order to realize that different touch pens trigger different operations, the operations corresponding to the triggering of each touch pen can be preset. Whenever the user inputs a certain touch operation, the touch pen corresponding to the touch operation is matched with the preset operation according to the preset correspondence between the touch pen and the operation to determine the operation triggered by the touch pen, so as to execute the operation. In this regard, in order to realize the operation of triggering the recognition of writing handwriting into text content by a touch pen of a specific material, the target material information corresponding to the touch pen is preset. Once a touch operation is generated, the material information of the touch pen corresponding to the touch operation is compared with the preset target material information, and whether the writing handwriting generated by the touch operation needs to be converted into corresponding text content is determined based on whether the two material information match. Accordingly, the steps specifically include S131-S132:

[0046] S131. Obtain a signal threshold range, compare the elastic wave signal of the touch pen with the signal threshold range, and determine whether the elastic wave signal of the touch pen is within the signal threshold range.

[0047] Exemplarily, it can be known from the above content that the material information of the touch pen is the amplitude and phase information of the elastic wave signal, and correspondingly, the target material information is the amplitude and phase information of the elastic wave signal of the target touch pen that can trigger the handwriting recognition into text content operation. For the acquisition of the target material information, the corresponding target touch pen can be used to touch the screen cover in advance to obtain the elastic wave signal generated by the target touch pen touching the screen cover, and the signal threshold range of the elastic wave signal of the target touch pen is determined by collecting the elastic wave signal of the target touch pen multiple times. The signal threshold range is used to characterize that the elastic wave signals within the threshold range are all elastic wave signals generated by the target touch pen, that is, if the elastic wave signal generated by the touch pen is within the signal threshold range, then the material of the touch pen is the same as that of the target touch pen.

[0048] Among them, the signal threshold range of the target touch pen includes an amplitude threshold range and a phase threshold range. The amplitude and phase of the elastic wave signal of the touch pen are compared with the amplitude threshold range and the phase threshold range respectively to determine whether the amplitude is within the amplitude threshold range and whether the phase is within the phase threshold range. If both the amplitude and the phase are within the corresponding threshold range, it is confirmed that the elastic wave signal of the touch pen is within the signal threshold range, otherwise it is confirmed that the elastic wave signal of the touch pen is not within the signal threshold range.

[0049] S132: If the elastic wave signal of the touch pen is within the elastic wave signal threshold range, it is determined that the two material information matches; otherwise, it is determined that the two material information does not match.

[0050] Exemplarily, if the elastic wave signal of the touch pen is within the elastic wave signal threshold range, it indicates that the material of the touch pen is the same as the material of the target touch pen that triggers the operation of recognizing the written handwriting as text content, so it can be determined that the touch pen can trigger the operation of recognizing the written handwriting as text content. If the elastic wave signal of the touch pen is not within the elastic wave signal threshold range, it indicates that the material of the touch pen is different from the material of the target touch pen that triggers the operation of recognizing the written handwriting as text content, so it can be determined that the touch pen cannot trigger the operation of recognizing the written handwriting as text content.

[0051] S140: Perform text recognition on the handwriting, obtain text content corresponding to the handwriting, and display the text content on the electronic whiteboard.

[0052] Exemplarily, if the material information of the touch pen matches the material information of the target touch pen, it indicates that the material of the touch pen is the same as that of the target touch pen, so it can be determined that the touch pen can trigger the operation of recognizing handwriting as text content. Further, the handwriting is recognized as text content, and the text content is displayed on the electronic whiteboard, so as to realize the operation of triggering the recognition of handwriting as text content by a touch pen of the same material as the target touch pen. It can be understood that when a user uses a touch pen of the same material as the target touch pen to write sloppy handwriting on the display screen, the electronic whiteboard automatically converts the sloppy handwriting into corresponding neat text content for the user to view.

[0053] S150, displaying the written handwriting on the electronic whiteboard.

[0054] For example, if the material information of the touch pen does not match the material information of the target touch pen, it indicates that the material of the touch pen is different from that of the target touch pen, so it can be determined that the touch pen cannot trigger the operation of recognizing the writing handwriting as text content, and the writing handwriting is directly displayed on the electronic whiteboard. It can be understood that when the user wants to obtain a real writing effect, he uses a touch pen with a different material from the target touch pen to write on the electronic whiteboard, and the corresponding writing handwriting is directly displayed on the electronic whiteboard.

[0055] The technical solution provided by the embodiment is described exemplarily below:

[0056] The user opens the electronic whiteboard and creates a new whiteboard file. The display screen is configured with a first touch pen and a second touch pen. The user uses the first touch pen to write the four words "influencing factors" on the electronic whiteboard. The piezoelectric elastic wave sensor installed at the screen frame or the inner side of the screen cover measures the elastic wave signal generated when the first touch pen touches the screen cover, and converts the elastic wave signal into digital elastic wave characteristic information. The optical touch sensors installed on both sides of the display surface measure the writing information of the four words "influencing factors" written by the first touch pen on the electronic whiteboard. The display packages the elastic wave characteristic information and writing information generated by the two touch sensors to generate a touch operation, and sends it to the processor. The processor receives the touch operation and obtains the elastic wave characteristic information and writing information from the touch operation. According to the writing information, the handwriting corresponding to the "influencing factors" is generated, and the first material information of the first touch pen is determined according to the elastic wave characteristic information. Figure 2 is a first schematic diagram of an electronic whiteboard provided in an embodiment of the present application. Figure 2 The user uses the first touch pen 12 to write the four words “influencing factors” on the electronic whiteboard 11 , and the first writing handwriting 13 of “influencing factors” is displayed on the corresponding electronic whiteboard 11 .

[0057] Further, the first material information of the first touch pen is compared with the preset target material information. Assuming that the currently set target material information is the first material information, it is confirmed that the first material information of the first touch pen matches the target material information, that is, it is confirmed that the first touch pen can trigger the operation of recognizing the handwriting as text content. At this time, the processor performs text recognition on the handwriting of the "influencing factor", obtains its corresponding text content, and replaces the handwriting on the electronic whiteboard with its corresponding text content. Figure 3 is a second schematic diagram of the electronic whiteboard provided in the embodiment of the present application. Figure 3 , the first text content 14 of "influencing factors" is displayed on the electronic whiteboard 11. The font size of the first text content 14 is consistent with the font size of the first handwriting 13, and the position coordinates are also consistent.

[0058] Figure 4 3 is a third schematic diagram of an electronic whiteboard provided in an embodiment of the present application. Figure 4 , the user uses the second touch pen 15 to write "weather" on the electronic whiteboard, and the second handwriting 16 of "weather" is displayed on the electronic whiteboard 12. At this time, the target material information set is the first material information. The processor compares the first material information with the target material information, and confirms that the second material information of the second touch pen does not match the target material information, and then confirms that the second touch pen cannot trigger the operation of recognizing the handwriting as text content, and the electronic whiteboard remains the same. Figure 4 shown.

[0059] on the other hand, Figure 5 4 is a fourth schematic diagram of an electronic whiteboard provided in an embodiment of the present application. Figure 5 , a writing box 17 and a display box 18 are provided on the electronic whiteboard 11. The user uses the first touch pen 12 to write "Influencing factors:" in the writing box 12, and the third writing handwriting 19 of "Influencing factors:" is correspondingly displayed in the writing box 12. Since the first material information of the first touch pen matches the target material information, the third writing handwriting is recognized as the corresponding third text content 20, and the third text content 20 is displayed on the display box 18, wherein the size of the third text content 20 is determined by the font size set in the display box 18. Through this embodiment, a writing area of ​​appropriate size can be set for the user to write, and a display area can be set to organize the handwriting of the user into neat text content for the user to watch, thereby improving the user experience.

[0060] Figure 6 1 is a flowchart of another touch operation processing method provided by an embodiment of the present application. This embodiment is specifically described based on the above embodiment. Figure 6 , the touch operation processing method provided in this embodiment includes:

[0061] S210, receiving touch sensing information corresponding to the writing operation input by the touch pen on the electronic whiteboard, and extracting touch track information detected by the touch sensing module and elastic wave feature information detected by the elastic wave sensor from the touch sensing information.

[0062] For example, refer to Figure 2 , the user uses the first touch pen 12 to write "influencing factors" on the electronic whiteboard 11, and the piezoelectric elastic wave sensor measures the elastic wave signal generated when the first touch pen touches the screen cover, and converts the elastic wave signal into elastic wave characteristic information. The touch sensing module measures the touch trajectory information of the four words "influencing factors" written by the first touch pen on the electronic whiteboard. The display packages the elastic wave characteristic information and touch trajectory information generated by the two touch sensors to generate touch sensing information, and sends the touch sensing information to the processor, which responds to the corresponding touch operation according to the touch sensing information.

[0063] S220 , determining the material information of the touch pen according to the elastic wave characteristic information, and generating corresponding writing handwriting according to the touch trajectory information.

[0064] Exemplarily, the elastic wave feature information is used as an input signal, and the material type of the touch pen is output through the machine learning model, and the material information of the touch pen is represented by the material type of the touch pen. It can be understood that the elastic wave signals generated when the touch pens of the same material touch the screen cover are roughly the same. The machine learning model is trained in advance with a large number of training samples to identify the touch pen material corresponding to the elastic wave signal through the machine learning model, and the training samples include the elastic wave signal and the corresponding touch pen material.

[0065] For example, the elastic wave characteristic information measured by the piezoelectric elastic wave sensor when the first touch pen writes "influencing factors" is input into the machine learning model to obtain material No. 1 output by the machine learning model. Further, the first writing handwriting 13 corresponding to the "influencing factors" is drawn on the electronic whiteboard 11 according to the touch track information.

[0066] S230, comparing the touch pen material information with the preset target material information to determine whether the two material information match. If the two material information match, execute step S240; if the two material information do not match, execute step S270.

[0067] Exemplarily, the material type of the touch pen also belongs to the material information of the touch pen. In order to realize the operation of triggering the recognition of written handwriting as text content through the target touch pen, the material type of the target touch pen can be pre-set. Once a touch operation is generated, the material type of the touch pen corresponding to the touch operation is compared with the pre-set target material type, and whether the written handwriting generated by the touch operation needs to be converted into the corresponding text content is determined based on whether the two material information are the same. It can be understood that if the two material types are the same, it indicates that the touch pen is the target touch pen for triggering the operation of recognizing written handwriting as text content, that is, it is necessary to continue to perform the operation of recognizing written handwriting as text content. If the material types of the two are different, it indicates that the touch pen is not the target touch pen for triggering the operation of recognizing written handwriting as text content, that is, it is not necessary to perform the operation of recognizing written handwriting as text content, but the written handwriting continues to be displayed on the electronic whiteboard.

[0068] For example, the currently set target material type is material No. 1, and the material type of the first touch pen output by the machine learning model is also material No. 1. It can be determined that the material type of the first touch pen is the same as the target material type, then the operation of recognizing the handwriting as text content continues.

[0069] S240: Perform text recognition on the handwriting to obtain text content corresponding to the handwriting.

[0070] Exemplarily, handwriting information of the written handwriting is obtained, and the handwriting information is input into a pre-trained text recognition model to obtain text content output by the text recognition model.

[0071] S250: Determine the font size, color and font of the text content according to the size and color of the handwriting and the font set on the electronic whiteboard.

[0072] Exemplarily, in order to obtain text content that matches the handwriting, the font size and color of the text content are determined according to the size and color of the handwriting, and the font of the text content is determined according to the font set on the electronic whiteboard. For example, if the font set on the electronic whiteboard is Songti, the font of the text content is also Songti accordingly.

[0073] S260: Display the text content on the electronic whiteboard according to the position information of the handwriting on the electronic whiteboard.

[0074] For example, refer to Figure 2 and Figure 3 After determining that the target material type is the same as the material type of the first touch pen 11, the first handwriting 13 is input into the pre-trained text recognition model, and the first text content 14 corresponding to the first handwriting 13 output by the text recognition model is obtained. The first text content 14 is adjusted to the same size as the first handwriting 13, the first text content 14 is displayed in the same position, and the original first handwriting 13 is deleted, that is, the electronic whiteboard is removed from Figure 2 Convert to Figure 3 .

[0075] S270, displaying the written handwriting on the electronic whiteboard.

[0076] For example, if the currently set target material type is not material No. 1, it is determined that the material type of the first touch pen is different from the target material type, and the first writing handwriting 13 continues to be displayed on the electronic whiteboard 11, that is, the electronic whiteboard 11 is still the same as Figure 2 shown.

[0077] Figure 7 1 is a flowchart of another touch operation processing method provided by an embodiment of the present application. This embodiment is specifically described based on the above embodiment. Figure 7 , the touch operation processing method provided in this embodiment includes:

[0078] S310, receiving touch sensing information corresponding to the writing operation input by the touch pen in the writing box of the electronic whiteboard, and extracting touch track information detected by the touch sensing module and elastic wave feature information detected by the elastic wave sensor from the touch sensing information.

[0079] For example, refer to Figure 5, the user uses the first touch pen 12 to write "influencing factors" in the writing box 17 of the electronic whiteboard 11, and the piezoelectric elastic wave sensor measures the elastic wave signal generated when the first touch pen touches the screen cover, and converts the elastic wave signal into elastic wave characteristic information. The touch sensing module measures the touch trajectory information of the four words "influencing factors" written by the first touch pen on the electronic whiteboard. The display packages the elastic wave characteristic information and touch trajectory information generated by the two touch sensors to generate touch sensing information, and sends the touch sensing information to the processor, which responds to the corresponding touch operation according to the touch sensing information.

[0080] S320: Determine the material information of the touch pen according to the elastic wave characteristic information, and generate corresponding writing handwriting according to the touch trajectory information.

[0081] For example, the elastic wave characteristic information measured by the piezoelectric elastic wave sensor when the first touch pen writes "influencing factors" is input into the machine learning model to obtain material No. 1 output by the machine learning model. Further, the third writing handwriting 19 corresponding to the "influencing factors" is drawn in the writing box 17 of the electronic whiteboard 11 according to the touch track information.

[0082] S330, comparing the touch pen material information with the preset target material information to determine whether the two material information match. If the two material information match, execute step S340; if the two material information do not match, execute step S370.

[0083] For example, the currently set target material type is material No. 1, and the material type of the first touch pen output by the machine learning model is also material No. 1. It can be determined that the material type of the first touch pen is the same as the target material type, then the operation of recognizing the third writing handwriting 19 in the writing box 17 as the corresponding text content is continued.

[0084] S340: Perform text recognition on the handwriting to obtain text content corresponding to the handwriting.

[0085] For example, the handwriting information of the third writing handwriting 19 is obtained, and the handwriting information is input into a pre-trained text recognition model to obtain the text content output by the text recognition model.

[0086] S350: Display the text content on the display frame according to the font and font size set in the display frame of the electronic whiteboard.

[0087] For example, assuming that the font currently set in the display frame is Songti and the font size is size 4, the text content of the third handwriting 19 is converted into text content 20 in size 4 Songti and displayed in the display frame 19 .

[0088] S360: receiving a save operation on the display box, and in response to the save operation, saving the text content in the display box in a text format.

[0089] Exemplarily, the user can save the text content in the display box through the save control of the display box.

[0090] S370, displaying the written handwriting on the electronic whiteboard.

[0091] The above-mentioned touch operation processing method receives a touch operation, generates corresponding writing handwriting according to the writing information in the touch operation, determines the material information of the touch pen that triggers the touch operation according to the elastic wave characteristic information in the touch operation, and the elastic wave characteristic information is the elastic wave signal when the touch pen touches the screen cover; compares the touch pen material information with the preset target material information to determine whether the two material information match; if the two material information match, performs text recognition on the writing handwriting, obtains the text content corresponding to the writing handwriting, and displays the text content on the electronic whiteboard; if the two material information do not match, displays the writing handwriting on the electronic whiteboard. By adopting the above technical means, based on the different elastic wave characteristic information generated by touch pens of different materials, the material information of the touch pen can be identified through the elastic wave characteristic information of the touch pen. The user can pre-set the target material information. If the material information of the touch pen matches the target material information, the handwriting of the touch pen is converted into text content and displayed on the electronic whiteboard. If it does not match, the handwriting is displayed on the electronic whiteboard. That is, the matching operations of touch pens of different materials are pre-set, and the material information is determined by the elastic wave signal of the touch pen, thereby determining the corresponding operation of the touch pen, so as to realize different touch pens triggering different operations. In addition, the touch pen does not need to install communication equipment, has high reliability and low cost, and can effectively improve the user experience.

[0092] Figure 8 is a schematic diagram of a touch operation processing device provided by an embodiment of the present application. Figure 8 The touch operation processing device includes: a receiving module 401, an information determination module 402, a comparison module 403, a text display module 404 and a handwriting display module 405.

[0093] The receiving module 401 is configured to receive touch sensing information corresponding to the writing operation input by the touch pen on the electronic whiteboard, and extract touch track information detected by the touch sensing module and elastic wave feature information detected by the elastic wave sensor from the touch sensing information;

[0094] The information determination module 402 is configured to determine the material information of the touch pen according to the elastic wave characteristic information, and generate corresponding writing handwriting according to the touch track information;

[0095] The comparison module 403 is configured to compare the touch pen material information with the preset target material information to determine whether the two material information match;

[0096] The text display module 404 is configured to perform text recognition on the handwriting if the two material information match, obtain the text content corresponding to the handwriting, and display the text content on the electronic whiteboard;

[0097] The handwriting display module 405 is configured to display the written handwriting on the electronic whiteboard if the two material information do not match.

[0098] On the basis of the above-mentioned embodiment, the touch track information includes the contact area, movement speed, movement acceleration and / or contact position when the touch pen touches the electronic whiteboard.

[0099] Based on the above embodiment, the text display module includes: a format adjustment unit configured to determine the font size, color and font of the text content according to the size and color of the handwriting and the font set on the electronic whiteboard.

[0100] On the basis of the above embodiment, the text display module further includes: a first display unit configured to display the text content on the electronic whiteboard according to the position information of the handwriting on the electronic whiteboard.

[0101] On the basis of the above embodiment, the receiving module includes: a writing box receiving unit configured to receive touch sensing information corresponding to the writing operation input by the touch pen in the writing box of the electronic whiteboard.

[0102] On the basis of the above embodiment, the text display module further includes: a second display unit configured to display the text content on the display frame according to the font and font size set in the display frame of the electronic whiteboard.

[0103] On the basis of the above embodiment, the touch operation processing device further includes: a saving module configured to receive a saving operation performed on the display frame, and in response to the saving operation, save the text content in the display frame into a text format.

[0104] The above-mentioned touch operation processing device receives a touch operation, generates corresponding writing handwriting according to the writing information in the touch operation, determines the material information of the touch pen that triggers the touch operation according to the elastic wave characteristic information in the touch operation, and the elastic wave characteristic information is the elastic wave signal when the touch pen touches the screen cover; compares the touch pen material information with the preset target material information to determine whether the two material information match; if the two material information match, performs text recognition on the writing handwriting, obtains the text content corresponding to the writing handwriting, and displays the text content on the electronic whiteboard; if the two material information do not match, displays the writing handwriting on the electronic whiteboard. By adopting the above technical means, based on the different elastic wave characteristic information generated by touch pens of different materials, the material information of the touch pen can be identified through the elastic wave characteristic information of the touch pen. The user can pre-set the target material information. If the material information of the touch pen matches the target material information, the handwriting of the touch pen is converted into text content and displayed on the electronic whiteboard. If it does not match, the handwriting is displayed on the electronic whiteboard. That is, the matching operations of touch pens of different materials are pre-set, and the material information is determined by the elastic wave signal of the touch pen, thereby determining the corresponding operation of the touch pen, so as to realize different touch pens triggering different operations. In addition, the touch pen does not need to install communication equipment, has high reliability and low cost, and can effectively improve the user experience.

[0105] It is worth noting that in the above-mentioned embodiment of the touch operation processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0106] The touch operation processing device provided in the embodiment of the present application is included in an interactive tablet and can be used to execute the touch operation processing method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0107] Fig. 9 Schematic diagram of an interactive tablet provided by an embodiment of the present application. Fig. 9 As shown, the interactive tablet includes a processor 50, a memory 51, an input device 52, an output device 53 and a display screen 54; the number of processors 50 in the interactive tablet can be one or more. Fig. 9 In the example, a processor 40 is used; the number of display screens 54 in the interactive tablet can be one or more, Fig. 9 A display screen 54 is taken as an example; the processor 50, the memory 51, the input device 52, the output device 53 and the display screen 54 in the interactive tablet can be connected by a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0108] The memory 51 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the touch operation processing method in the embodiment of the present application (for example, the receiving module 401, the information determination module 402, the comparison module 403, the text display module 404 and the handwriting display module 405 in the touch operation processing device). The processor 50 executes various functional applications and data processing of the interactive tablet by running the software programs, instructions and modules stored in the memory 51, that is, realizing the above-mentioned touch operation processing method.

[0109] The memory 51 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the interactive tablet, etc. In addition, the memory 51 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 51 may further include a memory remotely arranged relative to the processor 40, and these remote memories may be connected to the interactive tablet via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0110] The input device 52 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the interactive tablet. The output device 53 may include an audio output device such as a speaker. The display screen 54 is a display screen with a touch function, which can be a capacitive screen, an electromagnetic screen or an infrared screen. Optionally, when the display screen 54 is an infrared screen, it includes an optical touch sensor, which is arranged around the display screen 54, and can also be used to receive infrared signals and send the infrared signals to the processor 50 or other devices. The display screen 54 also includes a piezoelectric elastic wave sensor, which is used to measure the elastic wave characteristic information of the touch pen. The display screen 54 can display handwriting and text content on the electronic whiteboard according to the instructions of the processor.

[0111] The above-mentioned interactive tablet includes a touch operation processing device, which can be used to execute any touch operation processing method and has corresponding functions and beneficial effects.

[0112] An embodiment of the present application provides a computer-readable storage medium, and when the program is executed by a processor, the program is used to execute the touch operation processing method provided in the above embodiment.

[0113] Of course, the computer-readable storage medium provided in the embodiment of the present application, whose computer-executable instructions are not limited to the above method operations, can also execute related operations in the touch operation processing method provided in any embodiment of the present application.

[0114] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0115] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A touch operation processing method, characterized in that: include: Receiving touch sensing information corresponding to a writing operation input by a touch pen on an electronic whiteboard, and extracting touch track information detected by a touch sensing module and elastic wave feature information detected by an elastic wave sensor from the touch sensing information; Determine the material information of the touch pen according to the elastic wave characteristic information, and generate corresponding writing handwriting according to the touch track information; Compare the touch pen material information with preset target material information to determine whether the two material information match; If the two types of material information match, text recognition is performed on the handwriting to obtain text content corresponding to the handwriting, and the text content is displayed on the electronic whiteboard; If the two types of material information do not match, the handwriting is displayed on the electronic whiteboard.

2. The method according to claim 1, characterized in that The touch track information includes a contact area, a moving speed and / or a contact position when the touch pen touches the electronic whiteboard.

3. The method according to claim 1, characterized in that Before displaying the text content on the electronic whiteboard, the method further includes: The font size, color and font of the text content are determined accordingly according to the size and color of the handwriting and the font set on the electronic whiteboard.

4. The method according to claim 3, characterized in that: The displaying of the text content on the electronic whiteboard comprises: According to the position information of the handwriting on the electronic whiteboard, the text content is displayed on the electronic whiteboard accordingly.

5. The method according to claim 1, characterized in that The receiving of touch sensing information corresponding to the writing operation input by the touch pen on the electronic whiteboard includes: The touch sensing information corresponding to the writing operation input by the touch pen in the writing box of the electronic whiteboard is received.

6. The method according to claim 5, characterized in that The step of displaying the text content on the electronic whiteboard includes: The text content is displayed on the display frame according to the font and font size set in the display frame of the electronic whiteboard.

7. The method according to claim 6, characterized in that After displaying the text content on the display frame, the method further includes: A save operation on the display frame is received, and in response to the save operation, the text content in the display frame is saved in a text format.

8. A touch operation processing device, characterized in that: include: A receiving module is configured to receive touch sensing information corresponding to a writing operation input by a touch pen on the electronic whiteboard, and extract touch track information detected by the touch sensing module and elastic wave feature information detected by the elastic wave sensor from the touch sensing information; An information determination module, configured to determine the touch pen material information according to the elastic wave characteristic information, and generate corresponding writing handwriting according to the touch track information; A comparison module, configured to compare the touch pen material information with preset target material information to determine whether the two material information match; A text display module is configured to perform text recognition on the handwriting if the two material information match, obtain text content corresponding to the handwriting, and display the text content on the electronic whiteboard; The handwriting display module is configured to display the written handwriting on the electronic whiteboard if the two material information do not match.

9. An interactive tablet, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the touch operation processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the touch operation processing method as described in any one of claims 1 to 7 is implemented.

Images