Material identification method, interactive tablet, and storage medium

Abstract

The present application discloses a material identification method, an interactive tablet, and a storage medium for use in an interactive tablet. The interactive tablet includes an operation panel, an infrared touch sensor, and an elastic wave sensor. The infrared touch sensor is provided at at least one edge of the operation panel to form a touch detection area of the interactive tablet. The elastic wave sensor is used to detect vibrations of the operation panel and generate an electrical signal. The present application can improve the identification accuracy of the material of the touch object by combining the infrared blocking electrical signal and the original touch electrical signal, and solve the technical problem of the prior art that the interactive device cannot accurately identify the material of the touch object. Next, by setting a first period, when the first material information cannot be timely generated based on the infrared blocking electrical signal and the original touch electrical signal, the material type of the touch object can be confirmed by directly using the second material information generated based on the infrared blocking electrical signal, thereby avoiding an overly long calculation delay and improving the user experience.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of material identification, and more particularly to a material identification method, an interactive tablet, and a storage medium.

Background Art

[0002] Currently, interactive devices such as tablets or large displays mainly interact with users through an infrared touch screen or a capacitive touch screen provided on themselves. In the interaction process, the interactive device can sense the material type of the touch object only based on the touch area of the touch object. However, if the touch area is not accurately measured or cannot be measured, the interactive device cannot accurately determine the material type of the touch object.

[0003] In summary, how to accurately determine the material of the touch object has become a technical problem to be solved currently.

Summary of the Invention

[0004] Embodiments of the present application provide a material identification method, an interactive tablet, and a storage medium that can accurately identify the material of a touch object and solve the technical problem of the prior art that an interactive device cannot accurately identify the material of a touch object.

[0005] In a first aspect, embodiments of the present application provide a material identification method for use in an interactive tablet, the interactive tablet comprising an operation panel, an infrared touch sensor, and an elastic wave sensor, the infrared touch sensor being provided at at least one edge of the operation panel to form a touch detection area of the interactive tablet, the elastic wave sensor being used to detect vibrations of the operation panel and generate an electrical signal, the material identification method comprising When a touch object performs a touch operation in the touch detection area, the infrared touch sensor starts generating an infrared blocking electrical signal from a first point in time, calculating an energy average value of the original touch electrical signal in the current time window, calculating a ratio between the energy average value and a history energy average value, where the history energy average value is the energy average value of the original touch electrical signal in the immediately previous time window, determining whether the ratio is greater than a pre-set threshold, when it is greater than the threshold, obtaining the initial point in time of the current time window and using the initial point in time of the current time window as the start point of the valid touch electrical signal, obtaining valid touch electrical signals within a predetermined time from the start point as elastic wave data, and determining first material information of the touch object based on the infrared blocking electrical signal and the elastic wave data.

[0006] In a second aspect, an embodiment of the present application provides an interactive tablet, the interactive tablet comprising an operation panel, an infrared touch sensor, an elastic wave sensor, and at least one processing device, the infrared touch sensor being provided at at least one edge of the operation panel to form a touch detection area of the interactive tablet, the elastic wave sensor being used to detect vibrations of the operation panel and generate an electrical signal, and the at least one processing device when a touch object performs a touch operation in the touch detection area, the infrared touch sensor starts generating an infrared blocking electrical signal from a first point in time, from the first point in time, the elastic wave sensor starts generating an original touch electrical signal, calculating an energy average value of the original touch electrical signal in the current time window, calculating a ratio between the energy average value and a history energy average value, where the history energy average value is the energy average value of the original touch electrical signal in the immediately previous time window, Determine whether the ratio is greater than a preset threshold value, if it is greater than the threshold value, obtain the initial time point of the current time window, and use the initial time point of the current time window as the start time point of the valid touch electrical signal, obtain the valid touch electrical signal within a predetermined time from the start time point as elastic wave data, It is used to determine the first material information of the touch object based on the infrared blocking electrical signal and the elastic wave data.

[0007] In a third aspect, an embodiment of the present application provides a storage medium for storing computer-executable instructions, and when the computer-executable instructions are executed by a computer processor, they are used to execute the material identification method described in the first aspect.

[0008] As described above, the embodiments of the present application provide a material identification method, an interactive tablet, and a storage medium for use in an interactive tablet. The interactive tablet includes an operation panel, an infrared touch sensor, and an elastic wave sensor. The infrared touch sensor is provided at at least one edge of the operation panel to form a touch detection area of the interactive tablet. The elastic wave sensor is used to detect vibrations of the operation panel and generate an electrical signal. The material identification method includes: when a touch object performs a touch operation in the touch detection area, the infrared touch sensor starts generating an infrared blocking electrical signal from a first time point; calculating an energy average value of the original touch electrical signal in the current time window; calculating a ratio between the energy average value and a historical energy average value, where the historical energy average value is the energy average value of the original touch electrical signal in the immediately previous time window; determining whether the ratio is greater than a preset threshold; if it is greater than the threshold, obtaining an initial time point of the current time window and using the initial time point of the current time window as the start time point of the valid touch electrical signal; obtaining valid touch electrical signals within a predetermined time from the start time point as elastic wave data; and determining first material information of the touch object based on the infrared blocking electrical signal and the elastic wave data.

[0009] The embodiments of the present application can improve the identification accuracy of the material of the touch object by combining the infrared blocking electrical signal and the original touch electrical signal, thereby overcoming the technical problem of the prior art that the infrared blocking electrical signal has no material specificity and the original touch electrical signal is susceptible to vibration interference, resulting in a low material identification rate, and the interactive device cannot accurately identify the material of the touch object.

Brief Description of the Drawings

[0010]

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0011] The following description and the accompanying drawings fully disclose specific embodiments of the present application so that those skilled in the art can implement them. The examples represent only possible variations. Separate components and functions are selectable unless explicitly required, and the order of operations can vary. Parts and features of some embodiments may be included in or replaced by parts and features of other embodiments. The scope of the embodiments of the present application includes the entire scope of the claims and all obtainable equivalents of the claims. In this specification, each embodiment may be represented simply and for convenience, either alone or collectively, by the term "invention". However, when more than one invention is actually disclosed, it is not intended to automatically limit the scope of the present application to any single invention or inventive concept. In this specification, relational terms such as first and second are used only to distinguish one entity or operation from another entity and operation, and do not require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variation are intended to cover non-exclusive inclusion, so that a process, method or device comprising a series of elements includes not only those elements but also other elements not explicitly listed. Each example in this specification is described progressively, and the important explanatory parts of each example are all the differences from other examples. The similar parts between each example may be referred to each other. The structures, products, etc. disclosed in the examples are briefly described to correspond to the parts disclosed in the examples, and for related parts, reference may be made to the description of the method part.

[0012] An interactive device refers to a device that can exchange information with a user in a specific interactive manner. For example, devices such as mobile phones and tablets exchange information with users through gesture recognition. Currently, when identifying the type of a touch object, an interactive device generally uses an infrared touch screen or a capacitive touch screen provided on itself to identify the type of the touch object. A touch object refers to an object that comes into mutual contact with an interactive device, and may be, for example, a piece of chalk, an eraser, a marker, a meeting pen, a stylus pen, or a finger. An infrared touch screen refers to a touch screen provided with an infrared touch sensor. Exemplarily, as shown in FIG. 1A, the infrared touch screen 10 includes an operation panel 11 and an infrared touch sensor provided at at least one edge of the operation panel 11 to form a touch detection area. The operation panel 11 includes a first bezel 111, a second bezel 112, a third bezel 113, and a fourth bezel 114. Since the first bezel 111 is located at the top, it is also called the top side. Since the third bezel 113 is located at the bottom, it is also called the bottom side. The second bezel 112 and the fourth bezel 114 are located on opposite sides. Therefore, the second bezel 112 is also called the left side, and the fourth bezel 114 is also called the right side. The infrared touch sensor includes an infrared transmitter 121 and an infrared receiver 122. In one embodiment, in one set of infrared touch sensors, the infrared transmitter 121 is provided on the first bezel 111, and the infrared receiver 122 is provided on the third bezel 113. In another set of infrared touch sensors, the infrared transmitter 121 is provided on the second bezel 112, and the infrared receiver 122 is provided on the fourth bezel 114. Here, the infrared transmitter 121 is used to emit infrared rays, and the infrared receiver 122 is used to receive infrared rays. When the infrared rays emitted from the infrared transmitter 121 are blocked and the infrared receiver 122 cannot receive the infrared rays, it is determined that there is an obstacle between the infrared transmitter 121 and the infrared receiver 122.Therefore, based on the range covered by the infrared rays emitted from all infrared touch sensors (as shown by the straight lines in Fig. 1A), the detection range of the infrared touch sensors can be determined, and based on the detection range, the corresponding touch detection area on the infrared touch screen can be determined. Here, the touch detection range refers to the range within which a touch object can be detected when the touch object performs a touch operation on the operation panel. It can be understood that the size and position of the touch detection range are specifically determined by the number and position of the infrared touch sensors.

[0013] When the infrared touch screen operates, the infrared transmitter 121 of the infrared touch sensor emits infrared rays at a set frequency, and the infrared receiver 122 receives the infrared rays. This process is called the scanning process of the infrared touch sensor. In this embodiment, when a touch object enters the touch detection area, if the infrared rays emitted from the infrared touch sensor are blocked by the touch object, the electrical signal received by the infrared receiver 122 of the infrared touch sensor changes, thereby generating an infrared blocking electrical signal. By analyzing the infrared blocking electrical signal, information such as the position of the touch object and the area of the touch object can be further obtained. In one embodiment, inside the interactive tablet, there is an infrared processor called an infrared touch frame together with the infrared touch sensor. The infrared processor is an MCU (Microcontroller Unit) compatible with an infrared touch sensor such as an 811 SOC (System on Chip). When the infrared processor has the logical processing ability for the infrared blocking electrical signal, the infrared processor can provide a touch processing service to the upper-layer application based on the infrared blocking electrical signal. That is, the infrared processor can process the infrared blocking electrical signal scanned by the infrared touch sensor for the touch object. For example, it can calculate at least one piece of data such as the appearance coordinates (X coordinate, Y coordinate) of the touch object, the width and height of the touch object, the appearance time of the touch object, and the cross-sectional area of the touch object, and form a touch data packet (i.e., touch point data).

[0014] In one embodiment, a central processing unit is also provided inside the interactive tablet. Between the central processing unit and the infrared processor, USB components such as a USB HUB (hub), a USB switch, and a USB Redriver (signal repeater) are provided. The central processing unit may be used as a HID (Human Interface Device). The infrared processor and the central processor communicate via the USB components. When the infrared processor generates a touch data packet, it transmits the touch data packet to the central processor, and the central processor reports the touch data packet to the upper-layer application of the operating system.

[0015] It should be noted that in some models of interactive tablets, the central processor has the logical processing ability of the infrared blocking electrical signal. At this time, the central processing unit can provide touch processing services to the upper-layer application based on the infrared blocking electrical signal. That is, the infrared processor uploads the infrared blocking electrical signal to the central processing unit, and the central processing unit can process the infrared blocking electrical signal scanned by the infrared touch sensor for the touch object. For example, it can calculate at least one piece of data such as the appearance coordinates (X coordinate, Y coordinate) of the touch object, the width and height of the touch object, the appearance time of the touch object, and the cross-sectional area of the touch object, and form a touch data packet (i.e., touch point data).

[0016] Further to be explained, as shown in FIG. 1B, generally, since the infrared touch sensor 12 is higher than the surface of the operation panel 11 (also called the cover plate), the infrared touch sensor 12 transmits the light filtered through the filter strip (also called the filter plate). The filter strip is usually made by adding a colorant to the raw material and then through an injection molding or casting process. The filter strip can transmit infrared light and filter other ambient light, thereby improving the signal-to-noise ratio of the infrared blocking electrical signal. Then, the infrared light scanned by the infrared touch sensor 12 on the surface of the operation panel 11 has a certain height range. Therefore, the infrared touch sensor 12 forms a touch detection area in the vertical direction of the operation panel 11, and its height is H. The height H is high, generally greater than 2 mm (millimeters) and cannot be ignored.

[0017] The process by which the infrared touch screen realizes the touch function of the interactive tablet is as follows. When a touch object (such as a hand or a pen) writes on the operation panel, as a general process, first press the touch object, move when the touch object touches the surface of the operation panel, and finally lift the touch object. As shown in FIG. 1C, some of the processing operations of the infrared touch screen when pressing (Down), moving (Move), or lifting (Up) the touch object in one touch operation are described from the first touch state 131, the second touch state 132, the third touch state 133, the fourth touch state 134 to the fifth touch state 135.

[0018] In the first touch state 131, the touch object starts to be pressed and is above the touch detection area, that is, the distance between the touch object and the plane where the surface of the operation panel 11 is located is greater than H. Since the infrared light in the scanning process of the infrared touch sensor 12 is not blocked by the touch object, no infrared blocking electrical signal is generated. At this time, the infrared processor or the central processor does not report touch point data.

[0019] In the second touch state 132, the touch object continues to be pressed, is within the touch detection area, and does not touch the surface of the operation panel 11. That is, the distance between the touch object and the plane where the surface of the operation panel 12 is located is greater than 0 and less than H. Since the infrared rays in the scanning process of the infrared touch sensor 12 are blocked by the touch object, an infrared-blocking electrical signal is generated. At this time, the infrared processor or the central processor reports touch point data based on the infrared-blocking electrical signal.

[0020] In the third touch state 133, the touch object is within the touch detection area and touches the surface of the operation panel 11. The touch object can move on the surface of the operation panel 11. That is, the distance between the touch object and the plane where the surface of the operation panel 11 is located is 0 or less (the fact that it is less than 0 indicates the phenomenon that the surface of the operation panel 11 is recessed when receiving the acting force from the touch object). Obviously, the distance between the touch object and the plane where the surface of the operation panel 11 is located is less than H. At this time, since the infrared rays in the scanning process of the infrared touch sensor 12 are blocked by the touch object, an infrared-blocking electrical signal is generated, and the infrared processor or the central processor reports touch point data based on the infrared-blocking electrical signal.

[0021] In the fourth touch state 134, the touch object begins to be lifted, is within the touch detection area, and does not touch the surface of the operation panel 11. That is, the distance between the touch object and the plane where the surface of the operation panel 11 is located is greater than 0 and less than H. Since the infrared rays in the scanning process of the infrared touch sensor 12 are blocked by the touch object, an infrared-blocking electrical signal is generated. At this time, the infrared processor or the central processor reports touch point data based on the infrared-blocking electrical signal.

[0022] In the fifth touch state 135, the touch object continues to be lifted and is above the touch detection area. That is, the distance between the touch object and the plane where the surface of the operation panel 11 is located is greater than H, and since the infrared rays in the scanning process of the infrared touch sensor 12 are not blocked by the touch object, no infrared blocking electrical signal is generated. At this time, the infrared processor or the central processor does not report touch point data.

[0023] As a result, when the infrared touch sensor 12 scans the touch object for the first time, since the touch object does not contact the operation panel 11, the touch object needs to continue to move a height H toward the operation panel in a direction perpendicular to the operation panel in order to contact the operation panel 11. That is, when the infrared touch sensor 12 scans the touch object for the first time, a certain time interval is required until the touch object touches the operation panel 11. If the touch object does not touch the operation panel 11 for some time after the infrared touch sensor 12 scans the touch object for the first time, it indicates that there is a possibility that the touch object accidentally enters the touch detection area of the infrared touch 12 without the need to perform a touch operation on the operation panel 11. In this case, that is, when in the above-mentioned second touch state 132, when the infrared rays scanned by the infrared touch sensor 12 are blocked by the touch object, the infrared touch sensor 12 generates an infrared blocking electrical signal. At this time, the infrared processor or the central processing unit still reports touch point data. That is, when the touch object writes on the operation panel of the infrared screen, touch point data is generated when the touch object enters the touch detection area of the infrared touch sensor 12 but does not contact the operation panel.

[0024] In addition, the current method of determining the area of a touch object based on touch point data is applicable only when the touch object contacts the operation panel 11 vertically. However, when the touch object touches the operation panel obliquely, as shown in FIG. 2, the range of the infrared rays blocked by the touch object becomes larger. As a result, the area of the touch object determined based on the touch point data also becomes larger, and the type of the touch object is misidentified.

[0025] In other cases, when a capacitive touch screen is provided on the interactive device, the user usually performs a touch operation on the interactive device in accordance with a unified capacitive touch operation at present, that is, by pressing a touch object until it touches the surface of the screen. As shown in FIG. 3, FIG. 3 is a diagram showing the structure of the capacitive touch screen 14. When the touch object contacts the metal layer of the capacitive touch screen 14, as shown by the dark gray mesh in the contrast of FIG. 3, the capacitance of the contact point on the metal layer changes. Therefore, the frequency of the oscillator connected to the metal layer changes. By measuring the change in the frequency of the oscillator, the position of the contact point and the area of the touch object can be determined. Thereby, the interactive device can later determine the type of the touch object based on the area. However, when the touch object is an insulating touch object, the capacitive touch screen 14 cannot effectively identify the touch object and cannot identify the area of the touch object at the same time. As a result, the type of the touch object is easily misidentified. In another embodiment, when the touch object in contact with the capacitive touch screen is an active capacitive pen, the position detection circuit in the active capacitive pen collects the electrical signal on the capacitive touch screen to determine the position information of the active capacitive pen on the capacitive touch screen and transmits the position information to the interactive device. At the same time, the interactive device can also determine the type of the active capacitive pen based on the signal transmitted from the active capacitive pen. However, in this method, it is required that the touch object is an active device, and the touch object also needs to have the function of transmitting a signal to the interactive device. The implementation conditions are relatively strict and it cannot be applied to most usage scenarios.

[0026] Based on this, an embodiment of the present application provides a material identification method. As shown in FIG. 4, FIG. 4 is a flowchart of the material identification method according to the embodiment of the present application. The material identification method according to the embodiment of the present application may be executed by a material identification device. The material identification device may be implemented by software and / or hardware, and the material identification device may be composed of two or more physical entities or one physical entity. For example, the material identification device may be an interactive device such as a mobile phone, an interactive tablet, or a large display. The material identification method according to the embodiment of the present application is used in an interactive tablet. The interactive tablet includes an operation panel, an infrared touch sensor, and an elastic wave sensor. The infrared touch sensor is provided at at least one edge of the operation panel to form a touch detection area of the interactive tablet. The elastic wave sensor is used to detect the vibration of the operation panel and generate an electrical signal.

[0027] An interactive tablet refers to a tablet computer capable of interacting with users. For example, it is a tablet product that interacts with users through interactive touch technologies such as multi-point infrared or optical. Further to be explained, the structure of the interactive tablet according to this embodiment is shown in FIG. 5. The interactive tablet includes an operation panel 11, an infrared touch sensor 12, and an elastic wave sensor 15. The infrared touch sensor 12 is provided at at least one edge of the operation panel 11. Here, the operation panel 11 is a link for the interaction between the user and the interactive tablet. The interactive tablet can display different interfaces or contents through the operation panel 11. The user can send different commands to the interactive device through operations such as gestures or touches on the operation panel 11. For example, the operation panel 11 may be a touch screen or the like. In this embodiment, based on the range covered by the infrared rays emitted from the infrared touch sensors on different edges, the detection range of the infrared touch sensors can be determined, and thereby the touch detection area of the infrared touch sensors on the operation panel 11 of the interactive tablet can be determined. Also, the elastic wave sensor 15 provided in the interactive tablet is used to detect the elastic waves generated on the operation panel 11 by the contact of the touch object when the operation panel 11 vibrates, and generate an electrical signal based on the elastic waves. The electrical signal includes the waveform information of the elastic waves.

[0028] Furthermore, it should be further explained that when a touch object touches the surface of the operation panel, a deformation development occurs at the touched part of the surface of the operation panel, thereby generating an elastic wave that propagates into the operation panel. Here, the elastic wave is a type of stress wave, and the stress wave is a propagation form of disturbances of stress and strain, that is, the elastic wave is a transmission form of stress and strain caused by disturbances or external forces in an elastic medium. There are elastic forces that interact between the particles in the elastic medium. After a certain particle is displaced from the equilibrium position by a disturbance or an external force, the particle vibrates due to the elastic restoring force, causing displacement and vibration of the surrounding particles, and then the vibration propagates in the elastic medium while accompanying the transmission of energy. Each time a vibration occurs, the stress and strain change.

[0029] When a touch object touches the surface of the operation panel of an interactive tablet, the frequency of the elastic wave generated is determined by the two contact media, namely the touch object and the operation panel. In the touch process, a fundamental wave with a low frequency and higher harmonics are generated. The energy of the fundamental wave with a low frequency is generally much higher than that of the higher harmonics. Therefore, in this embodiment, in the case of the elastic wave generated when several commonly used touch objects touch the surface of the operation panel of an interactive tablet, the resonant frequency of the elastic wave sensor is designed to be consistent with the frequency of the fundamental wave, and the frequency of the fundamental wave is positioned as the operating frequency for material identification, thereby improving the signal-to-noise ratio of the sensor.

[0030] In one embodiment, at least one elastic wave sensor is attached to the operation panel of the interactive tablet. Here, the operating frequency of the elastic wave sensor basically coincides with the frequency for distinguishing different materials of the elastic wave generated when a touch object touches the surface of the operation panel. Under this condition, since the detection sensitivity of the elastic wave generated when the touch object writes on the operation panel can be increased, the elastic wave sensor detects the elastic wave generated when the touch object touches the surface of the operation panel, thereby generating a high-quality electrical signal and improving the detection accuracy of the material of the touch object.

[0031] Since elastic waves can be transmitted within the operation panel, in theory, the elastic wave sensor can be attached at any position where elastic waves are generated in the operation panel, and can detect elastic waves generated when a touch object touches the surface of the operation panel, regardless of whether it is at any position on the operation panel or at any position in direct or indirect contact with the operation panel.

[0032] In one embodiment, a piezoelectric sensor can be used as the elastic wave sensor. The piezoelectric sensor is a sensing component made by utilizing the piezoelectric effect that occurs after some dielectrics receive a force. Here, the piezoelectric effect refers to the phenomenon that when some dielectrics are deformed (including bending and stretching deformation) by an external force in a certain direction, charges are generated on their surfaces due to the change in the charge distribution within the material.

[0033] Furthermore, the piezoelectric sensor is affected by elastic waves and applies the positive piezoelectric effect to generate an electrical signal. Here, the positive piezoelectric effect may refer to the fact that when some dielectrics within the piezoelectric sensor are deformed (such as the deformation generated when a touch object touches the surface of the operation panel) by a force applied along a certain direction, a certain amount of charge is generated on the surface of the electrode, resulting in a charged state, and when the external force disappears, it returns to the normal uncharged state.

[0034] Generally, since the size of the operation panel of an interactive tablet is large and the degree of freedom of services such as an electronic whiteboard and annotation of documents provided by the interactive tablet is high, a user may touch the operation panel of the interactive tablet using a touch object at any coordinate on the surface of the operation panel of the interactive tablet, whereby a touch operation (such as clicking an icon or handwriting) is triggered. The mounting position of the elastic wave sensor on the operation panel is predetermined, and the attitude of the elastic wave sensor is also predetermined. The elastic wave sensor at the predetermined position and with the predetermined attitude has different detection sensitivities and accuracies for electrical signals generated at different coordinates on the operation panel of the interactive tablet. Generally, the closer the coordinate for triggering the touch operation is to the elastic wave sensor, the higher the accuracy of the elastic wave detected by the elastic wave sensor. Conversely, the farther the coordinate for triggering the touch operation is from the elastic wave sensor, the lower the accuracy of the elastic wave detected by the elastic wave sensor.

[0035] It should be noted that if the coordinate for triggering the touch operation is too close to the elastic wave sensor, the electrical signal generated by the elastic wave sensor will exceed the maximum voltage of the operational amplifier and distortion will occur. When maintaining a certain sensitivity and accuracy, it is considered that there is a detection range for the detection of elastic waves by the elastic wave sensor. Therefore, based on the relationship between the detection range and the operation panel of the interactive tablet, the number and position of the elastic wave sensors attached to the operation panel of the same interactive tablet can be determined.

[0036] Considering that a user may touch the surface of the operation panel of an interactive tablet using a touch object at any coordinate on the surface of the operation panel, since the portion in the invisible area of the operation panel (for example, the area blocked by the bezel) is limited, the area of the surface of the operation panel of the interactive tablet represents the area where the user touches the operation panel using the touch object. Generally, the number of elastic wave sensors has a positive correlation with the area of the surface of the operation panel of the interactive tablet, that is, the larger the area of the surface of the operation panel of the interactive tablet, the larger the number of elastic wave sensors, and conversely, the smaller the area of the surface of the operation panel of the interactive tablet, the smaller the number of elastic wave sensors. Of course, at least one elastic wave sensor is attached to the back surface of the operation panel of the interactive tablet, that is, the number of elastic wave sensors is at least one.

[0037] The material identification method provided by the embodiments of this application includes the following steps.

[0038] Step 101, when the touch object performs a touch operation in the touch detection area, the infrared touch sensor starts to generate an infrared blocking electrical signal from the first time point.

[0039] In this embodiment, when the touch object performs a touch operation in the touch detection area of the infrared touch sensor, the infrared rays emitted from the infrared touch sensor scan the touch object, and the infrared touch sensor starts to generate an infrared blocking electrical signal. Here, the first time point refers to the time point when the infrared rays of the infrared touch sensor first scan the touch object.

[0040] Step 102, from the first time point, the elastic wave sensor starts to generate the original touch electrical signal.

[0041] Meanwhile, starting from the first point in time, the elastic wave sensor begins to generate an original touch electrical signal in response to the vibration of the operation panel, and the original touch electrical signal is an electrical signal generated by the elastic wave sensor in response to the vibration of the operation panel after the first point in time.

[0042] It should be noted that starting from the first point in time, the fact that the elastic wave sensor begins to generate the original touch electrical signal means that at the first point in time, it does not generate the original touch electrical signal, but generates the original touch electrical signal after the first point in time. Based on the touch detection area with height H formed by the infrared touch sensor described in detail above, in the case of one complete touch operation, it can be understood that the touch object first enters the touch detection area with height H, and then contacts the operation panel at height 0. Corresponding to the movement process of the touch object, the infrared blocking signal is generated earlier than the touch electrical signal, that is, the touch electrical signal is not generated at the first point in time, but is generated with a delay after the first point in time. Although the absolute length of this delay may be very small, the relative order relationship cannot be excluded. In this scheme, to confirm the true start point of the touch electrical signal, that is, to confirm the exact contact time between the touch object and the operation panel, thereby confirming the infrared blocking signal generated when the touch object contacts the operation panel, and further to realize a touch response that accurately corresponds to the actual contact position and time based on the confirmed infrared blocking signal, the generation of the original touch electrical signal based on the first point in time and the first point in time is defined.

[0043] Step 103: Determine the first material information of the touch object based on the infrared blocking electrical signal and the original touch electrical signal.

[0044] After the infrared-blocking electrical signal and the original touch electrical signal are generated, the first material information of the touch object is determined based on the infrared-blocking electrical signal and the original touch electrical signal. Here, the first material information includes the material information of the touch object. For example, the first material information is a stylus pen or chalk, etc. In one embodiment, in the process of determining the first material information of the touch object, first, probabilities corresponding to different materials, such as a chalk probability of 65% and a stylus pen probability of 35%, can be generated based on the original touch electrical signal. Then, the probabilities are further calibrated using the infrared-blocking electrical signal to obtain the target probability. Exemplarily, the area of the touch object, such as 5 mm 2 or 10 mm 2 etc., is determined based on the infrared-blocking electrical signal, and then the probabilities corresponding to different materials are further adjusted based on the area to obtain the target probability. Finally, the first material information of the touch object can be generated based on the target probability.

[0045] Step 104, if it is determined that the first material information is obtained within the first period from the first time point, use the first material information as the material type of the touch object.

[0046] In this embodiment, if the first material information can be obtained within the first period from the first point in time, the first material information is used as the material type of the touch object. Further, the reason for setting the first period in this embodiment is that in the process of using the interactive tablet, there is a possibility that the touch object shields the infrared touch sensor but does not touch the operation panel, or a detection leakage of elastic wave data occurs. Therefore, by setting the first period, it is possible to determine whether the shielding of the touch object from the infrared touch sensor and the contact of the touch object with the interactive tablet are consistent operations. If the first material signal can be obtained within the first period from the first point in time, it can be shown that after the touch object shields the infrared touch sensor, it touches the operation panel in a very short time, that is, the touch operation is an effective touch operation. At this time, the first material information can be used as the material type of the touch object. Conversely, there may be a detection leakage of the original touch electrical signal or the touch object accidentally shields the infrared touch sensor. At this time, since the original touch electrical signal cannot be obtained to generate the first material information, the material information is not used as the material type of the touch object. It can be understood that the first period in this embodiment can be set according to actual needs. Exemplarily, considering the delay that the user may experience in the use process and the time required to determine the material type of the touch object, the first period can be set to 20 to 50 milliseconds. In one embodiment, the first period can be set to 32 milliseconds.

[0047] Step 105, if it is determined that the first material information is not obtained within the first period from the first point in time, the second material information is obtained as the material type of the touch object, and the second material information is generated based on the infrared blocking electrical signal.

[0048] If the first material information cannot be obtained within the first period from the first time point, in order to avoid an overly long calculation delay or an overly long waiting time for the original touch electrical signal due to detection leakage of elastic waves, second material information is generated based on the infrared blocking electrical signal, and the second material information is used as the material type of the touch object. The second material information also includes the material information of the touch object. In one embodiment, in the process of generating the second material information based on the infrared blocking electrical signal, the area of the touch object is determined based on the infrared blocking electrical signal, and then the second material information of the object can be generated based on the area of the touch object. Exemplarily, the touch area ranges corresponding to different touch objects are stored in advance, and the second material information of the touch object can be generated based on the area of the touch object. For example, the touch area range of a stylus pen is 3 - 6 mm 2 and the touch area range of chalk is 8 - 12 mm 2 If the area of the touch object is 5 mm 2 at this time, the second material information generated is a stylus pen. If the area of the touch object is 10 mm 2 at this time, the second material information generated is chalk.

[0049] As described above, in the embodiments of the present application, when a touch object performs a touch operation within the touch detection area, the infrared interruption electrical signal and the original touch electrical signal transmitted from the infrared touch sensor and the elastic wave sensor are respectively received, and if the first material information can be generated based on the infrared interruption electrical signal and the original touch electrical signal within the first period from the first time point when the generation of the infrared interruption electrical signal starts, the material type of the touch object is determined based on the first material information. Conversely, the material type of the touch object is determined based on the second material information generated by the infrared interruption electrical signal. The embodiments of the present application can improve the identification accuracy of the material by combining the infrared interruption electrical signal and the original touch electrical signal to identify the material of the touch object, thereby overcoming the technical problem of the prior art that the infrared interruption electrical signal has no material specificity and the original touch electrical signal is susceptible to vibration interference, resulting in a low material identification rate, and solving the technical problem that the interactive device cannot accurately identify the material of the touch object.

[0050] In the embodiments of the present application, by providing an elastic wave sensor, when a touch object operates on the operation panel, the original touch electrical signal generated by the elastic wave sensor can be collected, and thereby the material type of the touch object can be obtained by combining the original touch electrical signals. In the material identification process, how to position the start point of the valid touch electrical signal among the original touch electrical signals is a problem related to the identification efficiency and accuracy. One solution in the prior art (hereinafter referred to as Scheme 1) is to use the time point when the infrared touch sensor starts to generate an infrared interruption electrical signal as the start point of the valid touch electrical signal, and use the original touch electrical signals after the start point as the valid touch electrical signals, and calculate the material information of the touch object based on the valid touch electrical signals. However, as described above, when the touch object writes on the operation panel, even if it enters the touch detection area of the infrared touch sensor but does not contact the operation panel, the infrared touch sensor will generate an infrared interruption electrical signal, but at this time the touch object does not actually contact the operation panel. Therefore, there is a time difference between the start point of the valid touch electrical signal determined in this way and the actual start point of the valid touch electrical signal, and the signals within this time difference are actually noise signals, which have an adverse effect on the identification accuracy and speed of the material of the touch object.

[0051] To solve the above problems, the inventor creatively proposed a solution mechanism (hereinafter referred to as Scheme 2) that determines the starting point according to the change situation of the signal-to-noise ratio of the original touch electrical signal and sets a predetermined threshold value, and extracts the effective touch electrical signal from the original touch electrical signal according to whether the change situation of the signal-to-noise ratio of the original touch electrical signal is greater than the predetermined threshold value. Specifically, the signal-to-noise ratio refers to the ratio of the signal to the noise in an electronic device or an electronic system. In one embodiment, the signal-to-noise ratio refers to the ratio of the original touch electrical signal to the noise. When the signal-to-noise ratio is greater than a certain value (predetermined threshold value), it indicates that the value of the original touch electrical signal at that time is large, and the original touch electrical signal is not generated by noise but by the mutual touch between the touch object and the operation panel. Therefore, the starting point corresponding to the effective touch electrical signal can be determined. Specifically, the signal-to-noise ratio may be obtained by calculating the ratio of the energy average value of the original touch electrical signal in the current time window to the historical energy average value, and the historical energy average value is the energy average value of the original touch electrical signal in the immediately previous time window. When the ratio is greater than the predetermined threshold value, it indicates that the amplitude of the original touch electrical signal detected in the current time window is larger than that detected in the immediately previous time window. Since the change in the amplitude is not likely to be caused by the environment, it is considered that the change in the amplitude occurs when the touch object and the operation panel come into contact with each other. Therefore, the initial time point of the current window can be used as the starting point of the original touch electrical signal, that is, the time point when the touch object first touches the operation panel.

[0052] Specifically, as shown in FIG. 6, FIG. 6 is a material identification method according to Scheme 2 of the embodiment of the present application, and the material identification method is a specific implementation of the above material identification method. Referring to FIG. 6, the material identification method includes the following steps.

[0053] Step 201: When a touch object performs a touch operation in the touch detection area, the infrared touch sensor starts generating an infrared interruption electrical signal from the first point in time.

[0054] Step 202: From the first point in time, the elastic wave sensor starts generating an original touch electrical signal.

[0055] Step 203: Based on the original touch electrical signal, combine a predetermined threshold to determine the start point of the valid touch electrical signal.

[0056] In this embodiment, after the original touch electrical signal is acquired, further based on the original touch electrical signal, combine a predetermined threshold to determine the start point of the valid touch electrical signal. Since the original touch electrical signal generated by the elastic wave sensor may include noise signals, interference signals, etc., it is necessary to further confirm the valid touch electrical signal included therein. In the embodiment of this application, the original touch electrical signals collected by the elastic wave sensor are combined and calculated to obtain the start point of the valid touch electrical signal included therein, so that the valid touch electrical signal can be accurately positioned. Here, the start point is the point in time when the touch object first touches the operation panel.

[0057] Based on the above embodiment, the step of determining the start point of the valid touch electrical signal by combining a predetermined threshold based on the original touch electrical signal in step 203 includes the following steps.

[0058] Step 2031: Calculate the energy average value of the original touch electrical signal in the current time window.

[0059] A time window refers to a fixed time length, and the end point of the previous time window is the start point of the new time window. The time length of the time window may be set to 5 milliseconds or 10 milliseconds, etc., according to actual needs. In this embodiment, the specific time length of the time window is not limited. In this embodiment, in the current time window, the energy of the original touch electrical signal detected by the elastic wave sensor is measured in real time, and the average value of the energy is calculated. Here, the energy of the original touch electrical signal refers to the sum of the kinetic energy and potential energy of the elastic medium. When elastic waves propagate in the elastic medium, each particle vibrates near its equilibrium position. Therefore, the elastic medium has kinetic energy due to vibration. Also, due to vibration, the distance between each particle changes and the medium deforms. Therefore, the elastic medium has potential energy due to deformation. The sum of the kinetic energy due to vibration and the potential energy due to deformation is called the energy of elastic waves in this medium.

[0060] Step 2032: Calculate the ratio of the energy average value to the historical energy average value. The historical energy average value is the energy average value of the original touch electrical signal in the previous time window.

[0061] Calculate the energy average value within the current time window, and further calculate the ratio of the corresponding energy average value within the current time window to the corresponding historical energy average value within the previous time window. Here, the historical energy average value is the energy average value of the original touch electrical signal detected in the previous time window.

[0062] Step 2033: Determine whether the ratio is greater than a predetermined threshold.

[0063] Step 2034: If it is greater than the threshold, obtain the initial time point of the current time window, and use the initial time point of the current time window as the start point of the valid touch electrical signal.

[0064] The predetermined threshold refers to a threshold set in advance. In this embodiment, in order to distinguish the original touch electrical signal in environmental noise from the original touch electrical signal when the touch object contacts the interactive tablet, the predetermined threshold can be set to a value of 4 or more. When the ratio is greater than the predetermined threshold, it indicates that the amplitude of the original touch electrical signal detected in the current time window is larger than that of the original touch electrical signal detected in the previous time window. Since the change in the amplitude is less likely to be caused by the environment, it is considered that the change in the amplitude occurs when the touch object and the operation panel come into contact with each other. Therefore, the initial point of the current window can be used as the start point of the effective touch electrical signal, that is, the point when the touch object first touches the operation panel.

[0065] By combining the predetermined threshold and the signal-to-noise ratio in Scheme 2 for determination, an effective elastic wave signal can be more effectively extracted from the original elastic wave signal, and thereby, by combining a classification model or a neural network identification algorithm, the material information of the touch object can be obtained by identification. Such a scheme can more accurately identify the material information of the touch object as compared with the prior art scheme for determining the start point of the effective elastic wave signal based on the infrared-blocking electrical signal.

[0066] However, in the process of applying Scheme 2, in the case of an interactive tablet, the size of the interactive tablet is generally 65 inches or more, and further 86 inches or more, and the internal operation circuit is complex. There may be environmental noise generated during the operation process. Also, since the usage scenarios of the interactive tablet are relatively diverse, it is often used for playing audio and video information or video conferencing, etc. There may also be environmental noise generated during the process when the speaker sounds. The inventor found that the noise frequency of the above environmental noise of the interactive tablet is not stable and is irregular, so the noise cannot be classified or identified by a classification model or a neural network identification algorithm. In actual applications, since the environmental noise signal may also meet the condition that the signal-to-noise ratio is greater than a predetermined threshold, the extracted effective elastic wave signal may become inaccurate and may include environmental noise. As a result, the accuracy of the identification result of the elastic wave becomes low.

[0067] To solve the problems of Scheme 2, the inventor proposed another solution mechanism (hereinafter referred to as Scheme 3), that is, when there is no touch object, by dynamically adjusting a predetermined threshold according to the noise in the current environment, to avoid misidentifying the noise as a touch object contacting the operation panel due to excessive noise.

[0068] Specifically, it further includes the following steps.

[0069] Step 2035, dynamically update the predetermined threshold based on the first electrical signal generated by the infrared touch sensor and the second electrical signal generated by the elastic wave sensor.

[0070] In this embodiment, a predetermined threshold can be dynamically updated based on a first electrical signal generated by an infrared touch sensor and a second electrical signal generated by an elastic wave sensor. The first electrical signal refers to the electrical signal generated by the infrared touch sensor, and the second electrical signal refers to the electrical signal generated by the elastic wave sensor. The first and second are only used to distinguish the hardware of the electrical signal generation sources. Also, the first electrical signal and the second electrical signal can be respectively generated after the infrared touch sensor is activated and after the elastic wave sensor is activated. When it is determined that a touch object exists based on the first electrical signal, the first electrical signal at this time is the above-mentioned infrared blocking electrical signal, that is, the infrared blocking electrical signal is actually a specific signal confirmed from the first electrical signal. There are only two situations for the touch object, namely shielding and non-shielding. Therefore, when the first electrical signal is not the infrared blocking electrical signal, it is default that there is no current touch object. When it is determined that a touch object is in contact with the interactive tablet based on the second electrical signal, the second electrical signal at this time is the above-mentioned original touch electrical signal. Similarly, the original touch electrical signal is actually a specific signal confirmed from the second electrical signal. Therefore, when the second electrical signal is not the original touch electrical signal, it is default that the current touch object is not touching the interactive tablet. Regarding the original touch electrical signal, the signal state of the second electrical signal when the touch object does not touch should be used as a reference, but the originally set predetermined threshold has a deviation from the signal state of the second electrical signal when no touch occurs in the actual use process. Therefore, when the interactive tablet determines that there is no touch object based on the first electrical signal generated by the infrared touch sensor, it dynamically updates the predetermined threshold based on the amplitude of the second electrical signal generated by the current elastic wave sensor, thereby avoiding misidentifying noise as a touch object contacting the operation panel when the environmental noise is too large.Specifically, the step of dynamically updating a predetermined threshold based on the first electrical signal generated by the infrared touch sensor and the second electrical signal generated by the elastic wave sensor in step 2035 includes the following steps.

[0071] Step 20351: When it is determined based on the first electrical signal generated by the infrared touch sensor that there is no touch object, it is determined in real time whether the instantaneous amplitude of the second electrical signal generated by the elastic wave sensor is greater than a predetermined threshold.

[0072] First, when it is determined based on the first electrical signal generated by the infrared touch sensor that there is currently no touch object, it is determined in real time whether the instantaneous amplitude of the second electrical signal generated by the elastic wave sensor is greater than a predetermined threshold. Here, when it is determined that the first electrical signal is not an infrared blocking electrical signal, it is determined that there is no touch object in the touch detection area, and further, the second electrical signal currently detected by the elastic wave sensor can be obtained. At this time, the second electrical signal reflects the ambient noise of the interactive tablet, that is, environmental noise (mainly including circuit operation noise, etc.). By determining whether the instantaneous amplitude of the second electrical signal is greater than a predetermined threshold, the magnitude of the current environmental noise can be determined, and the predetermined threshold can be dynamically adjusted. Here, the instantaneous amplitude refers to the amplitude of the energy of the current second electrical signal.

[0073] Step 20352: When the instantaneous amplitude of the second electrical signal is greater than the predetermined threshold, set the predetermined threshold to a first predetermined threshold.

[0074] When the instantaneous amplitude of the second electrical signal is greater than the predetermined threshold, it indicates that the environmental noise of the interactive tablet at this time is large. In order to avoid misidentifying the environmental noise as a touch object contacting the operation panel because the noise is too large, it is necessary to increase the value of the predetermined threshold. Therefore, set the predetermined value to a first predetermined threshold with a large value.

[0075] Step 20353: When the instantaneous amplitude of the second electrical signal is smaller than a predetermined threshold value, set the predetermined threshold value to a second predetermined threshold value.

[0076] When the instantaneous amplitude of the second electrical signal is smaller than the predetermined threshold value, it indicates that the environmental noise of the interactive tablet is small. Therefore, the predetermined value can be set to a second predetermined threshold value with a small value. When the instantaneous amplitude of the second electrical signal is equal to the predetermined threshold value, either the first predetermined threshold value or the second predetermined threshold value can be selected as the predetermined threshold value, and the specific process is shown in FIG. 7. In one embodiment, the first predetermined threshold value can be set to 10 and the second predetermined threshold value can be set to 4.

[0077] Exemplarily, when a teacher uses an interactive tablet to play audio and video during a teaching process, the speaker of the interactive tablet sounds, the interactive tablet vibrates, and environmental noise is generated. At this time, when the interactive tablet determines that there is no touch object based on the first electrical signal generated by the infrared touch sensor, it further determines the instantaneous amplitude of the second electrical signal generated by the current elastic wave sensor. When the instantaneous amplitude of the second electrical signal is greater than the predetermined threshold value, it indicates that the environmental noise of the interactive tablet at this time is large. In order to avoid misidentifying the environmental noise generated from the speaker as a touch object contacting the operation panel, it is necessary to increase the value of the predetermined threshold value. The interactive tablet sets the predetermined threshold value to the first predetermined threshold value with a large value. For example, the first predetermined threshold value can be set to 10. When the instantaneous amplitude of the original touch electrical signal is smaller than the predetermined threshold value, it indicates that the environmental noise of the interactive tablet at this time is small (i.e., the speaker does not sound). Therefore, the interactive tablet can set the predetermined value to the second predetermined threshold value with a small value. For example, the second predetermined threshold value can be set to 4.

[0078] The above is a process of dynamically setting a predetermined threshold in Scheme 3. According to Scheme 3, when it is determined that there is no effective touch object and the instantaneous amplitude of the second electrical signal is greater than the predetermined threshold, the interactive tablet is in an operating state, indicating that the environmental noise is large at this time. Therefore, in order to avoid misidentifying the environmental noise as an effective touch electrical signal when a touch object later touches the touch object, the predetermined threshold is adjusted to a large first predetermined value. When the instantaneous amplitude of the second electrical signal is smaller than the predetermined threshold, it indicates that the environmental noise of the interactive tablet is small. Therefore, the predetermined value can be set to a small second predetermined threshold, thereby improving the detection sensitivity of the effective touch electrical signal. According to Scheme 3, when a touch object performs an operation on the interactive tablet, for some click operations, etc., noise can be better removed, and an effective touch electrical signal can be accurately extracted, thereby ensuring high detection accuracy and avoiding inaccurate extraction results due to the influence of noise.

[0079] In one embodiment of the present application, the following steps are further included.

[0080] Step 204, acquire effective touch electrical signals within a predetermined time from the start point as elastic wave data.

[0081] After executing Step 203 to determine the start point of the effective touch electrical signal, further use the effective touch electrical signals generated within a predetermined time from the start point as elastic wave data. Here, the predetermined time may be further set according to actual needs, and in this embodiment, the specific value of the predetermined time is not limited.

[0082] Step 205, determine the first material information of the touch object based on the infrared blocking electrical signal and the elastic wave data.

[0083] Step 206, when it is determined that the first material information is acquired within the first period from the first point in time, use the first material information as the material type of the touch object.

[0084] If it is determined in step 207 that the first material information is not acquired within the first period from the first time point, the second material information is acquired as the material type of the touch object, and the second material information is generated based on the infrared blocking electrical signal.

[0085] As described above, in the embodiment of the present application, when the touch object is not detected, the current environmental noise is estimated based on the instantaneous amplitude of the second electrical signal generated by the elastic wave sensor, and a predetermined threshold is dynamically adjusted according to the environmental noise. When the environmental noise is large, the predetermined threshold is set to a first predetermined threshold with a large value. When the environmental noise is small, the predetermined threshold is set to a second predetermined threshold with a small value. By dynamically adjusting the predetermined threshold, the embodiment of the present application can avoid misidentifying the environmental noise as the touch object contacting the operation panel due to the interference of the environmental noise on the interactive tablet. Thereby, when the touch object performs a touch operation on the interactive tablet, the start time point of the effective touch electrical signal can be determined more accurately to obtain effective elastic wave data, and the identification accuracy of the material using the subsequent effective elastic wave data is improved.

[0086] As shown in FIG. 8, FIG. 8 is a material identification method according to an embodiment of the present application, and the material identification method is a specific implementation of the above material identification method. Referring to FIG. 8, the material identification method includes the following steps.

[0087] In step 301, when the touch object performs a touch operation in the touch detection area, the infrared touch sensor starts to generate an infrared blocking electrical signal from the first time point.

[0088] In step 302, from the first time point, the elastic wave sensor starts to generate an original touch electrical signal.

[0089] In step 303, based on the original touch electrical signal, a predetermined threshold is combined to determine the start time point of the effective touch electrical signal.

[0090] Step 304: Obtain valid touch electrical signals within a predetermined time from the start time as elastic wave data.

[0091] Step 305: Generate probabilities corresponding to different materials based on the elastic wave data.

[0092] In this embodiment, after obtaining valid elastic wave data, probabilities corresponding to different materials are further generated based on the elastic wave data, and the probability represents the magnitude of the possibility of an event occurring. In this embodiment, the probability represents the possibility that the touch object belongs to different materials. For example, the probability that the touch object belongs to a finger is 55%, or the probability that the touch object belongs to a stylus pen is 80%, etc. In one embodiment, the elastic wave data can be input into a trained neural network to obtain probabilities corresponding to different materials.

[0093] Based on the above embodiment, the step of generating probabilities corresponding to different materials based on the elastic wave data in step 305 includes the following steps.

[0094] Step 3051: Preprocess the elastic wave data to obtain target data.

[0095] First, in order to eliminate the influence of different modes of elastic wave data generated by the difference of elastic wave sensors and the touch position on the subsequent calculation results, it is necessary to first preprocess the elastic wave data to obtain target data from the elastic wave data. For example, first remove invalid data in the elastic data, such as elastic wave data with clipping distortion or elastic wave data with too low energy, and then the elastic wave data can be fused to obtain target data.

[0096] In one embodiment, the elastic wave data is generated based on valid touch electrical signals corresponding to a plurality of elastic wave sensors.

[0097] To be described, in this embodiment, one predetermined time is used as one data processing cycle, and the elastic wave data generated within the data processing cycle may be described or recorded in the form of a set. Each element of the set is respectively generated based on an effective touch electrical signal corresponding to one elastic wave sensor. Exemplarily, the elastic wave data set is

[0098] [Number] where T is the predetermined time,

[0099] [Number] is the elastic wave data obtained based on the effective touch electrical signal of the i-th elastic wave sensor, n is the number of elastic wave sensors,

[0100] [Number] and so on.

[0101] Correspondingly, the step of preprocessing the elastic wave data to obtain target data in step 3051 includes the following steps.

[0102] Step 30511, perform time-frequency conversion on the elastic wave data corresponding to a plurality of elastic wave sensors to obtain frequency domain data corresponding to the elastic wave data.

[0103] First, perform time-frequency conversion on the elastic wave data corresponding to a plurality of elastic wave sensors to obtain frequency-domain data corresponding to the elastic wave data. Time-frequency conversion refers to converting time-domain data into frequency-domain data. Exemplarily, Fourier transform can be performed on each elastic wave data in the elastic wave data set D1 to obtain corresponding frequency-domain data D2. The frequency-domain data may similarly be described or recorded in the form of a set, that is, it can be understood that the frequency-domain data set includes the frequency-domain data corresponding to each elastic wave data in the elastic wave data set D1. In one embodiment, before performing time-frequency conversion, the amplitude of each data in the elastic wave data set D1 is compared with a preset effective signal amplitude, data with clipping distortion (i.e., data whose value is greater than the effective signal amplitude) is determined, and the data with clipping distortion can also be removed.

[0104] Step 30512, perform multi-channel signal fusion on the frequency-domain data to obtain fusion data.

[0105] After obtaining the frequency-domain data, multi-channel signal fusion is performed on the frequency-domain data to obtain fusion data. Here, multi-channel signal fusion refers to obtaining fusion data by fusing each data in the frequency-domain data set. In this embodiment, the purpose of multi-channel signal fusion is to eliminate the influence of the differences between the elastic wave sensors and the different modes of elastic wave data generated by the touch position on the subsequent calculation results. Here, the difference between the elastic wave sensors means that due to the influence of the manufacturing process or manufacturing environment in the manufacturing of the elastic wave sensors, there are slight differences in the elastic wave sensors manufactured on the same production line. Even when two elastic wave sensors manufactured on the same production line are provided at the same position, there are slight differences in the elastic waves collected by the two elastic wave sensors. The different modes of elastic waves generated at the touch position refer to the different modes of elastic waves collected by the elastic wave sensors when the elastic waves propagate to different positions. Exemplarily, the area of the operation panel 11 of the interactive tablet is shown in FIG. 9. Under positions 1 to 12 of the operation panel 11, elastic wave sensors 15 are respectively provided. When the touch point between the touch object and the operation panel 11 is at position 1, the elastic wave sensors 15 at different positions will detect different modes of elastic waves. FIG. 10 is a diagram showing the different modes of elastic waves detected by the elastic wave sensors 15 at positions 1 to 12 respectively. Since each elastic wave sensor can only reflect a part of the vibration characteristic information of the touch object, the material identification accuracy of each elastic wave sensor is limited and the material identification accuracy is low. Therefore, in this embodiment, by fusing the frequency-domain data corresponding to the elastic waves, the expression of the material information is strengthened, and the influence of the difference in the signals of each elastic wave sensor due to the difference in the touch position on the subsequent calculation process is avoided.

[0106] In this embodiment, multi-channel signal fusion can be performed by weighted average or neural network, etc. to obtain target data. Specifically, different multi-channel signal fusion methods will be described below.

[0107] In one embodiment, the step of performing multi-channel signal fusion on the frequency-domain data to obtain the fusion data includes: performing weighted averaging on the frequency-domain data to obtain the fusion data.

[0108] In one embodiment, weighted averaging is performed on the frequency-domain data to obtain the fusion data. Exemplarily, if the frequency-domain data is

[0109]

Number

[0110]

Number

[0111]

Number

[0112]

Number

[0113] In one embodiment, the step of performing multi-channel signal fusion on the frequency-domain data to obtain the fusion data includes: inputting the frequency-domain data into a pre-set first neural network to obtain the fusion data.

[0114] In another embodiment, the frequency-domain data can be input into a trained first neural network to obtain the fusion data, specifically as follows.

[0115]

Number

[0116]

Number

[0117]

Number

[0118] Step 30513, Filter the fusion data to obtain target data.

[0119] Finally, the fusion data can be filtered to obtain the required target data.

[0120] In one embodiment, the step of preprocessing the elastic wave data in step 3051 to obtain target data includes the following steps.

[0121] Step 30514, Calculate the system function of the propagation of elastic waves to each elastic wave sensor.

[0122] In this embodiment, first, the system function of the propagation of elastic waves to each elastic wave sensor is calculated. Here, the system function is a rational function of a complex variable S with real coefficients, that is, a real rational function. For the process of calculating the system function of elastic waves, reference can be made to the process of calculating the system function in the prior art, and the repeated description is omitted in this embodiment.

[0123] Step 30515, Divide the elastic wave data corresponding to each elastic wave sensor by the corresponding system function to obtain the first data corresponding to each elastic wave sensor.

[0124] After calculating the system function corresponding to each elastic wave sensor, divide the elastic wave data corresponding to each elastic wave sensor by the corresponding system function to obtain the first data corresponding to each elastic wave sensor. Specifically, in this embodiment, the elastic wave data corresponding to each elastic wave sensor in the elastic wave data set can be divided by the system function corresponding to the elastic wave sensor to obtain the first data corresponding to each elastic wave sensor.

[0125] Step 30516, calculate the average value of all the first data to obtain the target data.

[0126] After the first data corresponding to each elastic wave sensor is obtained, the average value of all the first data can be calculated to obtain the target data. In one embodiment, the calculation formula for the target data D4 is as follows.

[0127]

Equation

[0128]

Equation

[0129] The above is the process of preprocessing the elastic wave data in step 3051 to obtain the target data.

[0130] Step 3052, input the target data into a pre-set neural network to obtain the probabilities corresponding to different materials.

[0131] After the target data set is obtained, the target data set can be input into a trained neural network to obtain the probabilities corresponding to different materials. In one embodiment, the neural network is a multi-layer fully connected network, including a first fully connected layer, a second fully connected layer, a third fully connected layer, a first BN layer, a second BN layer, a Dropout layer, and a Softmax layer. As shown in FIG. 11, the first fully connected layer, the first BN layer, the second fully connected layer, the second BN layer, the Dropout layer, the third fully connected layer, and the Softmax layer are sequentially connected.

[0132] Specifically, for the one-dimensional data input into the multi-layer fully connected network

[0133]

Number

[0134]

Number

[0135]

Number

[0136]

Number

[0137]

Number

[0138] As shown in Formula 5, the feature vector

[0139]

Number

[0140]

Number

[0141]

Number

[0142]

Number

[0143] After stacking the fully connected layer and the BN layer, the one-dimensional data input into the multi-layer fully connected network is linearly mapped to the feature data, and finally the feature data is input into the Dropout layer and the Softmax layer to obtain probabilities. Here, the Dropout layer is used to shield the connection method of the fully connected layer according to the Bernoulli distribution probability in the training process of the multi-layer fully connected network, as shown in Formula 6.

[0144]

Number

[0145]

Number

[0146]

Number

[0147] Finally, as shown in Equation 7, the probability corresponding to each material

[0148]

Number

[0149]

Number

[0150] Furthermore, it should be noted that in this embodiment, the training parameters and weight parameters of the multi-layer fully connected network are obtained by pre-training the multi-layer fully connected network. Exemplarily, in one embodiment, the training parameters and weight parameters of the multi-layer fully connected network are obtained by training the multi-layer fully connected network using the cross-entropy loss function and the Adam optimization algorithm. The specific training process is as follows.

[0151] A sufficient number of historical target data sets are obtained as training data, and a part of the historical target data set is extracted as a validation set. Here, the historical target data set is a historically obtained target data set. Then, the material information corresponding to each historical target data in the training set is annotated to generate a one-hot label.

[0152]

Number

[0153] After that, a multi-layer fully connected network is trained using the cross-entropy loss function and the Adam optimization algorithm, and the cross-entropy loss is shown in Equation 8.

[0154]

Number

[0155] The above training process for the training parameters and weight parameters can be summarized as follows. Collect the first predetermined number of target data as training data, collect the second predetermined number of target data as verification data, receive the material labels corresponding to each training data and verification data, use the cross-entropy loss function and the Adam optimization algorithm in rounds to train a multi-layer fully connected network. Here, in each round, use the training data corresponding to a predetermined data volume for training, and train in each round to obtain the corresponding initial network model. Verify the initial network model with the verification data. When the verification discrimination rate of the initial network model reaches a predetermined index, confirm that the training parameters and weight parameters of the initial network model are the training parameters and weight parameters of the multi-layer fully connected network, and end the training. Based on the training process summarized and described here, the training rounds, the predetermined data volume corresponding to each round, the cross-entropy loss function, etc. in the above specific training process are used as exemplary implementation details and can all be adjusted.

[0156] The above steps are the specific process of obtaining the probabilities corresponding to different materials based on the elastic wave data. Outputting the probabilities corresponding to different materials using a multi-layer fully connected network has higher discrimination accuracy and better robustness compared to the conventional manual characterization method.

[0157] Step 306: Adjust the probability based on the infrared-blocking electrical signal to obtain the target probability.

[0158] After the probability is obtained, in order to avoid the reduction of the material discrimination accuracy due to the elastic wave data being affected by vibration interference, in this embodiment, the probability is further adjusted based on the infrared-blocking electrical signal to obtain the target probability. Specifically, in one embodiment, the step of adjusting the probability based on the infrared-blocking electrical signal to obtain the target probability includes the following steps.

[0159] Step 3061: Determine the area of the touch object based on the infrared blocking electrical signal.

[0160] First, when determining the area of the touch object based on the infrared blocking electrical signal, for the process of determining the area of the touch object based on the infrared blocking electrical signal, reference can be made to conventional literature, but repeated description is omitted in this embodiment.

[0161] Step 3062: Determine the corresponding weight between the area and materials different from the area based on the area of the touch object.

[0162] After the area of the touch object is determined, further determine the corresponding weight between the area and materials different from the area. Here, the weight refers to the relative importance of a certain factor or index for a certain object. In this embodiment, the area has different numerical ranges. For example, the numerical range 8 - 12 mm 2 represents one touch object, and the numerical range 16 - 22 mm 2 represents one touch object, and the numerical range 25 - 35 mm 2 represents one touch object. Therefore, there are corresponding weights between different areas corresponding to different areas and different materials. For example, when the area of the touch object is 10 mm 2 the corresponding weight of the chalk is 0.2, the weight of the stylus pen is 0.6, and the weight of the finger is 0.2. When the area of the touch object is 20 mm 2 the corresponding weight of the chalk is 0.6, the weight of the stylus pen is 0.2, and the weight of the finger is 0.2. When the area of the touch object is 30 mm 2 the corresponding weight of the chalk is 0.2, the weight of the stylus pen is 0.2, and the weight of the finger is 0.6. It can be understood that the weights of different materials corresponding to different areas are set in advance.

[0163] Step 3063: Adjust the probability (the probability corresponding to different materials) based on the weight to obtain a target probability.

[0164] After obtaining the weight between the area of the touch object and different materials, the target probability is obtained by adjusting the probability corresponding to different materials output by the multi-layer fully connected network based on the weight. Specifically, in this embodiment, the probability corresponding to each material is multiplied by the corresponding weight to obtain the target probability of different materials. Exemplarily, the probabilities of different materials output by the multi-layer fully connected network are shown in Table 1.

[0165] [Table 1] The weights of different materials obtained based on the area of the touch object are shown in Table 2.

[0166] [Table 2] At this time, the probability corresponding to each material is multiplied by the corresponding weight. That is, the target probability of chalk is

[0167] [Equation] and the target execution degree of the stylus pen is

[0168] [Equation] and the target probability of the finger is

[0169] [Equation] .

[0170] Step 307: Generate the first material information of the touch object based on the target probability.

[0171] Finally, first material information is generated based on the target probability. Specifically, a material with the largest target probability value is selected from each material, and the first material information is generated based on the selected material. For example, when the target probability value of a finger is the largest, the first material information is generated based on the finger.

[0172] In step 308, if it is determined that the first material information has been obtained within the first period from the first time point, the first material information is used as the material type of the touch object.

[0173] In step 309, if it is determined that the first material information has not been obtained within the first period from the first time point, second material information is obtained as the material type of the touch object, and the second material information is generated based on an infrared-blocking electrical signal.

[0174] As described above, in the embodiment of the present application, when preprocessing elastic wave data, by fusing the elastic wave data according to a multi-channel fusion policy to obtain fused data, when determining the material of a touch object using the elastic wave data detected by a single elastic wave sensor, it is possible to avoid the elastic wave data becoming invalid due to anomalies and resulting in misidentification. In addition, the stability of the target data set can be enhanced, and the influence of the differences in the elastic wave data detected by each elastic wave sensor due to the difference in the touch position on subsequent calculations can be avoided. Further, in the embodiment of the present application, outputting probabilities corresponding to different materials using a multi-layer fully connected network has higher identification accuracy and better robustness compared to the conventional manual characterization method. Next, in the embodiment of the present application, by combining infrared-blocking electrical signals to adjust the probabilities, the identification accuracy of the multi-layer fully connected network can be improved, and the identification accuracy of the material can be improved.

[0175] In one embodiment, the material identification method further includes the following steps.

[0176] Step 401: If the first material information is not obtained within the first period from the first time point, obtain the infrared blocking electrical signal at the immediately previous time point, and based on the current infrared blocking electrical signal, determine whether the current touch operation is associated with the touch operation at the immediately previous time point. If it is associated, obtain the material type corresponding to the immediately previous time point as the material type of the touch event.

[0177] In this embodiment, when it is determined that the first material information is not obtained within the first period from the first time point, obtain the infrared blocking electrical signal at the immediately previous time point, and based on the current infrared blocking electrical signal, determine whether the current touch operation is associated with the touch operation at the immediately previous time point. Here, being associated means that the current touch operation and the touch operation at the immediately previous time point of the touch object on the operation panel are caused by a series of consistent operations such as the user continuously sliding with a finger in the touch detection area of the operation panel. Since the pause time when the touch object performs a series of consistent operations is very short, it is possible to exclude the replacement of the touch object during the touch process, that is, the material of the touch object is maintained without change. When the current touch operation is associated with the touch operation at the immediately previous time point, it is possible to exclude the change of the material of the touch object, and the material type of the previous touch event can be directly used as the material type of the current touch event.

[0178] Exemplarily, when the first touch operation, the second touch operation, and the third touch operation are a plurality of consecutive touch operations caused by a series of consistent touch actions, after confirming the first material type of the touch object in the first touch operation, the first material type of the touch object can be directly obtained as the second material type of the touch object in the second touch operation. Similarly, the second material type can be directly obtained as the third material type of the touch object in the third touch operation. When the fourth touch operation... the Nth touch operation is caused by a series of consistent touch actions, by analogy in this way, repeated description in this embodiment is omitted.

[0179] Specifically, in one embodiment, when determining whether the current touch operation is associated with the touch operation at the immediately previous time, based on the time interval between the current touch operation and the touch operation at the immediately previous time and the distance between the touch points, it is possible to confirm whether the current touch operation is associated with the touch operation at the immediately previous time.

[0180] In one embodiment, in step 401, obtaining the infrared blocking electrical signal at the immediately previous time, and the step of determining whether the current touch operation is associated with the touch operation at the immediately previous time based on the current infrared blocking electrical signal includes the following steps.

[0181] Step 4011: Determine the touch point based on the current infrared blocking electrical signal, and determine the historical touch point based on the infrared blocking electrical signal at the immediately previous time.

[0182] The touch point refers to the point where the touch object contacts the interactive tablet. In this embodiment, first, determine the current touch point of the touch object based on the current infrared blocking electrical signal, and determine the historical touch point at the immediately previous time based on the infrared blocking electrical signal at the immediately previous time.

[0183] Step 4012: Calculate the distance between the touch point and the historical touch point, and determine the time interval between the first time and the immediately previous time.

[0184] After determining the historical touch point at the immediately previous time, further calculate the distance between the touch point and the historical touch point, and determine the time interval between the first time and the immediately previous time.

[0185] Step 4013: If the distance is shorter than a predetermined distance and the time interval is smaller than a predetermined time interval, determine that the current touch operation is associated with the previous touch operation.

[0186] When the distance between the touch point and the historical touch point is shorter than a predetermined distance and the time interval between the first time point and the immediately preceding time point is smaller than a predetermined time interval, it is indicated that the current touch operation is caused by a series of consistent touch operations by the touch object on the touch object, and it is determined that the current touch operation is associated with the touch operation at the immediately preceding time point.

[0187] As described above, the embodiment of the present application determines whether the current touch operation is associated with the touch operation at the immediately preceding time point. If they are associated, the material type corresponding to the immediately preceding time point is obtained as the material type of the touch event. Thereby, the calculation amount in the process of identifying the material of the touch object can be reduced, the delay of the material identification process can be further reduced, the suspension due to calculation delay can be avoided, and the user experience can be improved.

[0188] As shown in FIG. 12, FIG. 12 is a diagram showing the structure of an interactive tablet according to the present application. The interactive tablet includes an operation panel 11, an infrared touch sensor 12, an elastic wave sensor 15, and at least one processing device 16. The infrared touch sensor 12 is provided at at least one edge of the operation panel 11 to form a touch detection area of the interactive tablet. The elastic wave sensor 15 is used to detect the vibration of the operation panel 11 and generate an electrical signal. The at least one processing device 16 When a touch object performs a touch operation in the touch detection area, the infrared touch sensor starts generating an infrared blocking electrical signal from the first time point. From the first time point, the elastic wave sensor starts generating an original touch electrical signal. Calculate the energy average value of the original touch electrical signal in the current time window. Calculate the ratio between the energy average value and the historical energy average value, where the historical energy average value is the energy average value of the original touch electrical signal in the immediately preceding time window. Determine whether the ratio is greater than a predetermined threshold. If it is greater than the threshold value, obtain the initial time point of the current time window, use the initial time point of the current time window as the start time point of the valid touch electrical signal, obtain the valid touch electrical signal within a predetermined time from the start time point as elastic wave data, It is used to determine the first material information of the touch object based on the infrared blocking electrical signal and the elastic wave data.

[0189] In this embodiment, a plurality of elastic wave sensors are attached to the back surface of the operation panel, and the elastic wave sensors may be directly or indirectly attached to the operation panel. For example, in the case of direct attachment, the elastic wave sensor may be attached to the back surface of the operation panel so as to be pasted with glue. In the case of indirect attachment, after attaching the elastic wave sensor to the FPC circuit board, the FPC circuit board is attached to the back surface of the operation panel, and the elastic wave sensor can be attached to the back surface of the operation panel with an attachment member such as transmitting an electrical signal through the FPC circuit board. Here, the FPC (Flexible Printed Circuit board) circuit board, that is, the flexible circuit board, is generally made of polyimide or polyester film as a base material, and has the advantages of being lightweight, thin, excellent in bendability, and having a high wiring density.

[0190] In an embodiment of the present application, when a touch object performs a touch operation within a touch detection area, an infrared interruption electrical signal and an original touch electrical signal transmitted from an infrared touch sensor and an elastic wave sensor are respectively received, and if it is possible to generate first material information based on the infrared interruption electrical signal and the original touch electrical signal within a first period from the first time point when starting to generate the infrared interruption electrical signal, the material type of the touch object is determined based on the first material information. Conversely, the material type of the touch object is determined based on second material information generated by the infrared interruption electrical signal. The embodiment of the present application can improve the identification accuracy of the material of the touch object by combining the infrared interruption electrical signal and the original touch electrical signal, thereby overcoming the problem that the infrared interruption electrical signal has no material specificity and the original touch electrical signal is easily affected by vibration interference, resulting in a low material identification rate, and solving the technical problem of the prior art that an interactive device cannot accurately identify the material of a touch object. Next, in the embodiment of the present application, by setting the first period, when it is not possible to generate the first material information in a timely manner based on the infrared interruption electrical signal and the original touch electrical signal, the material information of the touch object can be confirmed directly using the second material information generated based on the infrared interruption electrical signal, thereby avoiding the technical problem that the calculation delay is too long and the user experience deteriorates.

[0191] Based on the above embodiment, the material identification method further includes the following steps.

[0192] Dynamically update a predetermined threshold based on a first electrical signal generated by an infrared touch sensor and a second electrical signal generated by an elastic wave sensor.

[0193] Based on the above embodiment, the step of dynamically updating a predetermined threshold based on a first electrical signal generated by an infrared touch sensor and a second electrical signal generated by an elastic wave sensor is When it is determined that there is no touch object based on the first electrical signal generated by the infrared touch sensor, a step of determining in real time whether the instantaneous amplitude of the second electrical signal generated by the elastic wave sensor is greater than a predetermined threshold value; When the instantaneous amplitude of the second electrical signal is greater than the predetermined threshold value, a step of setting the predetermined threshold value to a first predetermined threshold value; When the instantaneous amplitude of the second electrical signal is smaller than the predetermined threshold value, a step of setting the predetermined threshold value to a second predetermined threshold value.

[0194] Based on the above embodiment, the step of determining the first material information of the touch object based on the infrared interruption electrical signal and the elastic wave data includes: A step of generating probabilities corresponding to different materials based on the elastic wave data; A step of adjusting the probabilities based on the infrared interruption electrical signal to obtain target probabilities; A step of generating the first material information of the touch object based on the target probabilities.

[0195] Based on the above embodiment, the step of adjusting the probabilities based on the infrared interruption electrical signal to obtain target probabilities includes: A step of calculating the area of the touch object based on the infrared interruption electrical signal; A step of determining the corresponding weights between the area and different materials based on the area of the touch object; A step of adjusting the probabilities based on the weights to obtain target probabilities.

[0196] Based on the above embodiment, the step of generating probabilities corresponding to different materials based on the elastic wave data includes: A step of preprocessing the elastic wave data to obtain target data; A step of inputting the target data into a pre-set neural network to obtain probabilities corresponding to different materials.

[0197] Based on the above embodiments, the step of generating elastic wave data based on effective touch electrical signals corresponding to a plurality of elastic wave sensors and preprocessing the elastic wave data to obtain target data includes: Performing time-frequency conversion on the elastic wave data corresponding to a plurality of elastic wave sensors to obtain frequency domain data corresponding to the elastic wave data; Performing multi-channel signal fusion on the frequency domain data to obtain fusion data; And filtering the fusion data to obtain target data.

[0198] Based on the above embodiments, the step of performing multi-channel signal fusion on the frequency domain data to obtain fusion data includes: Performing weighted average on the frequency domain data to obtain fusion data.

[0199] Based on the above embodiments, the step of performing multi-channel signal fusion on the frequency domain data to obtain fusion data includes: Inputting the frequency domain data into a pre-set first neural network to obtain fusion data.

[0200] Based on the above embodiments, the step of preprocessing the elastic wave data to obtain target data includes: Calculating the system function of the propagation of elastic waves to each elastic wave sensor; Dividing the elastic wave data corresponding to each elastic wave sensor by the corresponding system function to obtain first data corresponding to each elastic wave sensor; And calculating the average value of all the first data to obtain target data.

[0201] Based on the above embodiments, the neural network is a multi-layer fully connected network, including a first fully connected layer, a second fully connected layer, a third fully connected layer, a first BN layer, a second BN layer, a Dropout layer, and a Softmax layer. The first fully connected layer, the first BN layer, the second fully connected layer, the second BN layer, the Dropout layer, the third fully connected layer, and the Softmax layer are sequentially connected.

[0202] Based on the above embodiments, the training parameters and weight parameters of the multi-layer fully connected network are obtained by training the multi-layer fully connected network using the cross-entropy loss function and the Adam optimization algorithm.

[0203] Based on the above embodiments, the training parameters and weight parameters are obtained by training as follows.

[0204] Collect a first predetermined number of target data as training data and a second predetermined number of target data as verification data. Receive the material labels corresponding to each training data and verification data. Use the cross-entropy loss function and the Adam optimization algorithm in rounds to train the multi-layer fully connected network. Here, in each round, train using the training data corresponding to a predetermined data volume, and obtain the corresponding initial network model by training in each round. Verify the initial network model with the verification data. When the verification discrimination rate of the initial network model reaches a predetermined index, confirm that the training parameters and weight parameters of the initial network model are the training parameters and weight parameters of the multi-layer fully connected network, and end the training.

[0205] Based on the above embodiments, the material identification method further includes the following steps.

[0206] If the first material information is not obtained within the first period from the first time point, obtain the infrared blocking electrical signal at the immediately preceding time point, and based on the current infrared blocking electrical signal, determine whether the current touch operation is associated with the touch operation at the immediately preceding time point. If it is associated, obtain the material type corresponding to the immediately preceding time point as the material type of the touch event.

[0207] Based on the above embodiment, the step of obtaining the infrared blocking electrical signal at the immediately preceding time point and determining whether the current touch operation is associated with the touch operation at the immediately preceding time point based on the current infrared blocking electrical signal is as follows: Determine a touch point based on the current infrared blocking electrical signal, and determine a historical touch point based on the infrared blocking electrical signal at the immediately preceding time point; Calculate the distance between the touch point and the historical touch point, and determine the time interval between the first time point and the immediately preceding time point; If the distance is shorter than a predetermined distance and the time interval is smaller than a predetermined time interval, determine that the current touch operation is associated with the touch operation at the immediately preceding time point.

[0208] The embodiment of the interactive tablet of the present application is used to execute related operations in the material identification method provided by any embodiment of the present application, and has corresponding functions and beneficial effects.

[0209] In addition, the embodiment of the present application provides a storage medium including computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are used to execute related operations in the material identification method according to any embodiment of the present application, and have corresponding functions and beneficial effects.

[0210] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program.

[0211] Accordingly, this application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining hardware and software. Moreover, this application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code. This application is described with reference to the flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of this application. It should be understood that computer program instructions can implement each flow and / or block in the flowchart and / or block diagram, as well as combinations of flows and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, thereby generating an apparatus for realizing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram by instructions executed by the processor of the computer or other programmable data processing devices. These computer program instructions may be stored in a computer-readable memory that can guide the computer or other programmable data processing devices to operate in a specific manner, thereby generating a manufactured product including an instruction device by the instructions stored in the computer-readable memory, and the instruction device realizes the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.These computer program instructions may be loaded onto a computer or other programmable data processing apparatus, and cause the computer or other programmable device to execute a series of operational steps to generate a process implemented on the computer, whereby the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one flow or a plurality of flows in the flowchart and / or one block or a plurality of blocks in the block diagram.

[0212] In one typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory. The memory can include forms such as non-persistent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0213] A computer-readable medium includes any permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies that can be used to store information accessible by a computing device, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical memory, magnetic cassette tapes, magnetic tapes, magnetic disk memories, or other magnetic memories, or any other non-transmission medium, but are not limited thereto. As defined herein, a computer-readable medium does not include transitory media such as modulated data signals and carrier waves.

[0214] Also, it should be noted that the term "comprising," "including," or any variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or device. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of another same element in the process, method, article, or device that includes the element.

[0215] In the above specific embodiments, the object, technical solution, and beneficial effects of the present application are further described in detail. It should be understood that the above are only specific embodiments of the present application and are not intended to limit the protection scope of the embodiments of the present application. It is particularly pointed out that any modifications, equivalent replacements, improvements, etc. made by those skilled in the art within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying a material used in an interactive tablet, wherein the interactive tablet comprises an operation panel, an infrared touch sensor, and an elastic wave sensor, the infrared touch sensor is provided at at least one edge of the operation panel to form a touch detection area of the interactive tablet, the elastic wave sensor is used to detect vibrations of the operation panel and generate an electrical signal, and the material identification method comprises: When a touch object performs a touch operation in the touch detection area, the infrared touch sensor starts generating an infrared blocking electrical signal from a first time point; From the first time point, the elastic wave sensor starts generating an original touch electrical signal; Calculating an energy average value of the original touch electrical signal in a current time window; Calculating a ratio between the energy average value and a historical energy average value, where the historical energy average value is the energy average value of the original touch electrical signal in a previous time window; Determining whether the ratio is greater than a predetermined threshold; When it is greater than the threshold, obtaining an initial time point of the current time window and using the initial time point of the current time window as a start time point of a valid touch electrical signal; Obtaining valid touch electrical signals within a predetermined time from the start time point as elastic wave data; And determining first material information of the touch object based on the infrared blocking electrical signal and the elastic wave data. A material identification method characterized by comprising the above steps.

2. The method further comprises: Dynamically updating the predetermined threshold based on a first electrical signal generated by the infrared touch sensor and a second electrical signal generated by the elastic wave sensor. The material identification method according to claim 1.

3. The step of dynamically updating the predetermined threshold based on a first electrical signal generated by the infrared touch sensor and a second electrical signal generated by the elastic wave sensor is: When it is determined based on the first electrical signal generated by the infrared touch sensor that the touch object does not exist, a step of determining in real time whether the instantaneous amplitude of the second electrical signal generated by the elastic wave sensor is greater than a predetermined threshold value; When the instantaneous amplitude of the second electrical signal is greater than the predetermined threshold value, a step of setting the predetermined threshold value to a first predetermined threshold value; The material identification method according to claim 2, further comprising: when the instantaneous amplitude of the second electrical signal is smaller than the predetermined threshold value, a step of setting the predetermined threshold value to a second predetermined threshold value.

4. The step of determining the first material information of the touch object based on the infrared blocking electrical signal and the elastic wave data includes: Generating probabilities corresponding to different materials based on the elastic wave data; Adjusting the probabilities based on the infrared blocking electrical signal to obtain target probabilities; The material identification method according to claim 1, further comprising generating the first material information of the touch object based on the target probabilities.

5. The step of adjusting the probabilities based on the infrared blocking electrical signal to obtain target probabilities includes: Calculating the area of the touch object based on the infrared blocking electrical signal; Determining corresponding weights between the area and the different materials based on the area of the touch object; The material identification method according to claim 4, further comprising adjusting the probabilities based on the weights to obtain target probabilities.

6. The step of generating probabilities corresponding to different materials based on the elastic wave data includes: Preprocessing the elastic wave data to obtain target data; The material identification method according to claim 4, further comprising inputting the target data into a pre-set neural network to obtain probabilities corresponding to different materials.

7. The elastic wave data is generated based on valid touch electrical signals corresponding to the plurality of elastic wave sensors, and the step of preprocessing the elastic wave data to obtain target data includes: Performing time-frequency conversion on the elastic wave data corresponding to the plurality of elastic wave sensors to obtain frequency domain data corresponding to the elastic wave data; Performing multi-channel signal fusion on the frequency-domain data to obtain fused data; Filtering the fused data to obtain target data, characterized in that the method for identifying a material according to claim 6 includes these steps.

8. The step of performing multi-channel signal fusion on the frequency-domain data to obtain fused data: The method for identifying a material according to claim 7 is characterized in that the step includes performing weighted averaging on the frequency-domain data to obtain fused data.

9. The step of performing multi-channel signal fusion on the frequency-domain data to obtain fused data: The method for identifying a material according to claim 7 is characterized in that the step includes inputting the frequency-domain data into a pre-set first neural network to obtain fused data.

10. The step of preprocessing the elastic wave data to obtain target data: Calculating the system function of the propagation of the elastic wave to each elastic wave sensor; Dividing the elastic wave data corresponding to each elastic wave sensor by the corresponding system function to obtain first data corresponding to each elastic wave sensor; The method for identifying a material according to claim 6 is characterized in that the step includes calculating the average value of all the first data to obtain target data.

11. The neural network is a multi-layer fully connected network, including a first fully connected layer, a second fully connected layer, a third fully connected layer, a first BN layer, a second BN layer, a dropout layer and a softmax layer, and the first fully connected layer, the first BN layer, the second fully connected layer, the second BN layer, the dropout layer, the third fully connected layer and the softmax layer are sequentially connected, characterized in that the method for identifying a material according to claim 6 includes these features.

12. The training parameters and weight parameters of the multi-layer fully connected network are obtained by training the multi-layer fully connected network using a cross-entropy loss function and an Adam optimization algorithm, characterized in that the method for identifying a material according to claim 11 includes these features.

13. The training parameters and weight parameters are obtained by training as follows: Collect the target data of the first predetermined number as training data, and collect the target data of the second predetermined number as verification data. Receive the material labels corresponding to each of the training data and the verification data. Train the multi-layer fully connected network using the cross-entropy loss function and the Adam optimization algorithm in rounds. Here, train using the training data corresponding to a predetermined data volume for each round, and obtain the corresponding initial network model by training for each round. Verify the initial network model with the verification data. When the verification identification rate of the initial network model reaches a predetermined index, confirm that the training parameters and weight parameters of the initial network model are the training parameters and weight parameters of the multi-layer fully connected network, and end the training. The material identification method according to claim 12, characterized in that.

14. The method is If the first material information is not obtained within the first period from the first time point, obtain the infrared blocking electrical signal at the immediately previous time point, and based on the current infrared blocking electrical signal, determine whether the current touch operation is associated with the touch operation at the immediately previous time point. If it is associated, the method further includes the step of obtaining the material type corresponding to the immediately previous time point as the material type of the touch event. The material identification method according to claim 1.

15. The step of obtaining the infrared blocking electrical signal at the immediately previous time point and determining whether the current touch operation is associated with the touch operation at the immediately previous time point based on the current infrared blocking electrical signal is Determining a touch point based on the current infrared blocking electrical signal and determining a historical touch point based on the infrared blocking electrical signal at the immediately previous time point; Calculating the distance between the touch point and the historical touch point, and determining the time interval between the first time point and the immediately previous time point; If the distance is shorter than a predetermined distance and the time interval is smaller than a predetermined time interval, determining that the current touch operation is associated with the previous touch operation. The material identification method according to claim 14, characterized in that it includes.

16. An interactive tablet, comprising an operation panel, an infrared touch sensor, an elastic wave sensor, and at least one processing device, wherein the infrared touch sensor is provided at at least one edge of the operation panel to form a touch detection area of the interactive tablet, the elastic wave sensor is used to detect vibrations of the operation panel and generate an electrical signal, and the at least one processing device When a touch object performs a touch operation in the touch detection area, the infrared touch sensor starts generating an infrared interruption electrical signal from a first time point. From the first time point, the elastic wave sensor starts generating an original touch electrical signal. Calculate an energy average value of the original touch electrical signal in the current time window. Calculate a ratio between the energy average value and a historical energy average value, where the historical energy average value is the energy average value of the original touch electrical signal in the immediately preceding time window. Determine whether the ratio is greater than a predetermined threshold. If it is greater than the threshold, obtain an initial time point of the current time window and use the initial time point of the current time window as a start time point of a valid touch electrical signal. Obtain valid touch electrical signals within a predetermined time from the start time point as elastic wave data. An interactive tablet, characterized in that it is used to determine first material information of the touch object based on the infrared interruption electrical signal and the elastic wave data.

17. The interactive tablet according to claim 16, characterized in that the number of the elastic wave sensors is plural, and the elastic wave sensors are directly or indirectly attached to the back surface of the operation panel.

18. A storage medium for storing computer-executable instructions, wherein the computer-executable instructions are used to execute the material identification method according to any one of claims 1 to 15 when executed by a computer processor.

Images