Material identification method, interactive panel and storage medium

Abstract

A material identification method, an interactive panel and a storage medium, the material identification method is used for the interactive panel, the interactive panel includes an operation panel (11), an infrared touch sensor (12) and an elastic wave sensor (15), the infrared touch sensor (12) is arranged at least one edge of the operation panel (11) to form the touch detection area of the interactive panel, the elastic wave sensor (15) is used to detect the vibration of the operation panel (11) to generate an electric signal, and the material of the touch object is identified by combining the infrared shielding electric signal and the original touch electric signal.Secondly, by setting the first time length, in the case that the first material information cannot be generated in time according to the infrared shielding electric signal and the original touch electric signal, the second material information generated according to the infrared shielding electric signal is directly used to confirm the material type of the touch object, the calculation delay can be avoided for too long, and the use experience of the user is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of material identification, and in particular to a material identification method, an interactive panel and a storage medium. BACKGROUND

[0002] Currently, for interactive devices such as interactive panels or display panels, the interaction between the interactive devices and the user is mainly through the infrared touch-sensitive display screen or the capacitive touch screen set by the interactive devices. In the interaction process, the perception of the interactive device to the material type of the touch object can only be determined according to the touch area of the touch object. However, when the touch area measurement is inaccurate or cannot be measured, the interactive device cannot accurately determine the material type of the touch object.

[0003] To sum up, how to accurately determine the material of the touch object has become a technical problem to be solved at present.

[0004] SUMMARY

[0005] Embodiments of the present application provide a material identification method, an interactive panel and a storage medium, which can accurately identify the material of the touch object and solve the technical problem that the interactive device cannot accurately identify the material of the touch object in the prior art.

[0006] In a first aspect, the embodiments of the present application provide a material identification method for an interactive panel, the interactive panel comprising an operation panel, an infrared touch sensor and an elastic wave sensor, the infrared touch sensor being arranged at at least one edge of the operation panel to form a touch detection area of the interactive panel, and the elastic wave sensor being used to detect the vibration of the operation panel and generate an electrical signal. The material identification method comprises:

[0007] When the touch object performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared blocking electrical signal from a first time;

[0008] calculating an energy average value of the original touch electrical signal in a current time window;

[0009] calculating a ratio of the energy average value and a historical energy average value, the historical energy average value being an energy average value of the original touch electrical signal in a previous time window;

[0010] determining whether the ratio is greater than a preset threshold value;

[0011] If yes, an initial time of the current time window is obtained, and the initial time of the current time window is taken as a starting time of the valid touch electrical signal;

[0012] obtaining the valid touch electrical signal in a preset time length after the starting time as elastic wave data;

[0013] The first material information of the object being touched is determined based on infrared blocking electrical signals and elastic wave data.

[0014] Secondly, embodiments of this application provide an interactive flat panel, which includes an operation panel, an infrared touch sensor, an elastic wave sensor, and at least one processing device. The infrared touch sensor is disposed at at least one edge of the operation panel to form a touch detection area of ​​the interactive flat panel. The elastic wave sensor is used to detect vibrations of the operation panel and generate electrical signals. The at least one processing device is used for:

[0015] When the object being touched performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared blocking electrical signal from the very first moment.

[0016] From the very first moment, the elastic wave sensor begins to generate the original contact electrical signal;

[0017] Calculate the average energy of the original touch electrical signal within the current time window;

[0018] Calculate the ratio of the average energy value to the historical average energy value, where the historical average energy value is the average energy value of the original touch electrical signal within the previous time window;

[0019] Determine if the ratio is greater than a preset threshold;

[0020] If it is greater than, then the initial time of the current time window is obtained, and the initial time of the current time window is taken as the start time of the valid touch signal;

[0021] The effective touch electrical signals within a preset time period after the start time are acquired as elastic wave data.

[0022] The first material information of the object being touched is determined based on infrared blocking electrical signals and elastic wave data.

[0023] Thirdly, embodiments of this application provide a storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform a material identification method as described in the first aspect.

[0024] The above, the embodiment of the application provides a material identification method, an interactive panel and a storage medium, the method is used for the interactive panel, the interactive panel includes an operation panel, an infrared touch sensor and an elastic wave sensor, the infrared touch sensor is arranged at at least one edge of the operation panel to form a touch detection area of the interactive panel, the elastic wave sensor is used for detecting vibration of the operation panel and generating an electrical signal, the material identification method comprises the following steps: when a touch object performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared shielding electrical signal from a first time; the energy average value of the original touch electrical signal in a current time window is calculated; the ratio of the energy average value and a historical energy average value is calculated, the historical energy average value is the energy average value of the original touch electrical signal in a previous time window; it is judged whether the ratio is greater than a preset threshold value; if greater, the initial time of the current time window is obtained, and the initial time of the current time window is taken as the starting time of the effective touch electrical signal; the effective touch electrical signal in a preset time length after the starting time is obtained as elastic wave data; the first material information of the touch object is determined according to the infrared shielding electrical signal and the elastic wave data.

[0025] The embodiment of the application can improve the accuracy of material identification by combining the infrared shielding electrical signal and the original touch electrical signal for material identification of the touch object, overcome the low material identification rate caused by the fact that the infrared shielding electrical signal does not have material specificity and the original touch electrical signal is easily disturbed by vibration, and solve the technical problem that the interactive device in the prior art cannot accurately identify the material of the touch object. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1A A structural schematic diagram of an infrared touch-sensitive display screen is provided for the embodiment of the application.

[0027] Figure 1B A partial structural schematic diagram of an infrared touch-sensitive display screen is provided for the embodiment of the application.

[0028] Figure 1C A schematic diagram of a touch object in different touch states is provided for the embodiment of the application.

[0029] Figure 2 A schematic diagram of a touch object tilting the operation panel is provided for the embodiment of the application.

[0030] Figure 3 A principle schematic diagram of a capacitive touch screen for checking a touch object is provided for the embodiment of the application.

[0031] Figure 4 A method flowchart of a material identification method is provided for the embodiment of the application.

[0032] Figure 5 A structural schematic diagram of an interactive panel is provided for the embodiment of the application.

[0033] Figure 6 A method flowchart of another material identification method provided for the embodiments of the present application.

[0034] Figure 7 A flowchart of dynamically updating a preset threshold provided for the embodiments of the present application.

[0035] Figure 8 A method flowchart of another material identification method provided for the embodiments of the present application.

[0036] Figure 9 A schematic diagram of an elastic wave sensor provided on an operation panel for the embodiments of the present application.

[0037] Figure 10 A schematic diagram of different modal elastic wave data detected by elastic wave sensors at different positions for the embodiments of the present application.

[0038] Figure 11 A structural schematic diagram of a multi-layer full connection network provided for the embodiments of the present application.

[0039] Figure 12 A structural schematic diagram of another interactive tablet provided for the embodiments of the present application.

[0040] Reference signs:

[0041] Infrared touch-sensitive display screen 10, operation panel 11, infrared touch sensor 12, first frame 111, second frame 112, third frame 113, fourth frame 114, infrared emitter 121, infrared receiver 122, first touch state 131, second touch state 132, third touch state 133, fourth touch state 134, fifth touch state 135, capacitive touch screen 14, elastic wave sensor 15. DETAILED DESCRIPTION

[0042] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of embodiments of this application includes the entire scope of the claims and all available equivalents of the claims. In this document, each embodiment may be referred to individually or collectively by the term "invention," which is merely for convenience and is not intended to automatically limit the scope of the application to any single invention or inventive concept if more than one invention is disclosed. Relational terms such as "first" and "second" are used herein only to distinguish one entity or operation from another, without requiring or implying any actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed. The various embodiments in this document are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the structures, products, etc., disclosed in the embodiments, since they correspond to the disclosed parts, the descriptions are relatively simple; relevant details can be found in the method section.

[0043] Interactive devices are devices capable of exchanging information with users through specific interactive methods. For example, mobile phones or tablets exchange information with users via gesture recognition. Currently, interactive devices typically identify the type of touch object using an infrared touch-sensitive display or a capacitive touchscreen. A touch object is an object that comes into contact with the interactive device; such objects include chalk, erasers, markers, styluses, or fingers. An infrared touch-sensitive display is a touchscreen equipped with an infrared touch sensor. For example,... Figure 1AAs shown, the infrared touch-sensitive display 10 includes an operation panel 11 and infrared touch sensors, which are arranged at at least one edge of the operation panel 11 to form a touch detection area. The operation panel 11 includes a first side frame 111, a second side frame 112, a third side frame 113 and a fourth side frame 114. The first side frame 111 is located at the top and is also called the top side, the third side frame 113 is located at the bottom and is also called the bottom side, and the second side frame 112 and the fourth side frame 114 are located at opposite sides, so the second side frame 112 is also called the left side and the fourth side frame 114 is also called the right side. The infrared touch sensors include infrared emitters 121 and infrared receivers 122. In an embodiment, the infrared emitters 121 of one group of infrared touch sensors are arranged at the first side frame 111, and the infrared receivers 122 are arranged at the third side frame 113. The infrared emitters 121 of another group of infrared touch sensors are arranged at the second side frame 112, and the infrared receivers 122 are arranged at the fourth side frame 114. The infrared emitters 121 are used to emit infrared rays, and the infrared receivers 122 are used to receive infrared rays. When the infrared rays emitted by the infrared emitters 121 are blocked and the infrared receivers 122 cannot receive the infrared rays, it can be determined that there is an obstruction between the infrared emitters 121 and the infrared receivers 122. Therefore, according to the range covered by the infrared rays emitted by all the infrared touch sensors (as shown by the straight line of Figure 1A , the detection range of the infrared touch sensors can be determined, and the corresponding touch detection area in the infrared touch-sensitive display can be determined according to the detection range. The touch detection range refers to the range that can be detected by the touch object when the touch object performs a touch operation on the operation panel. It can be understood that the size and position of the touch detection range are determined by the number and position of the infrared touch sensors.

[0044] When the infrared touch-sensitive display is in operation, the infrared emitter 121 of the infrared touch sensor emits infrared rays at a set frequency, and the infrared receiver 122 receives the infrared rays. This process is referred to as the scanning process of the infrared touch sensor. In this embodiment, when the touch object enters the touch detection area, the infrared rays emitted by the infrared touch sensor will be blocked by the touch object, and the electrical signal received by the infrared receiver 122 in the infrared touch sensor will change, thereby generating an infrared blocking electrical signal. By analyzing the infrared blocking electrical signal, the position of the touch object and the area of the touch object can be further obtained. In an embodiment, an infrared processor is arranged in the interior of the interactive panel, which is referred to as an infrared touch frame together with the infrared touch sensor. The infrared processor is an MCU (Microcontroller Unit) such as an 811 SOC (System on Chip) which is matched with the infrared touch sensor. When the infrared processor has the logical processing capability of the infrared blocking electrical signal, the infrared processor can provide touch processing services for the upper-layer application according to the infrared blocking electrical signal, that is, the infrared processor can process the infrared blocking electrical signal scanned by the infrared touch sensor on the touch object, for example, calculate at least one of the following data: the coordinates (X coordinate, Y coordinate) of the touch object, calculate the width and height, mark the time when the touch object appears, calculate the cross-sectional area of the touch object, and the like, to form a touch data packet (i.e., touch point data).

[0045] In an embodiment, the interior of the interactive panel is further provided with a central processor, and a USB component such as a USB HUB (hub), a USB switch, and a USB Redriver (signal repeater) is arranged between the central processor and the infrared processor. The central processor can serve as a HID (Human Interface Device), and the infrared processor and the central processor communicate through the USB component. When the infrared processor generates the touch data packet, the touch data packet is transmitted to the central processor, and the central processor reports the touch data packet to the upper-layer application of the operating system.

[0046] It should be noted that in some models of interactive panels, the central processor has the logical processing capability of the infrared blocking electrical signal. At this time, the central processor can provide touch processing services for the upper-layer application according to the infrared blocking electrical signal, that is, the infrared processor uploads the infrared blocking electrical signal to the central processor, and the central processor can process the infrared blocking electrical signal scanned by the infrared touch sensor on the touch object, for example, calculate the coordinates (X coordinate, Y coordinate) of the touch object, calculate the width and height, mark the time when the touch object appears, and calculate the cross-sectional area of the touch object, and the like, to form a touch data packet (i.e., touch point data).

[0047] It should be further noted that, as Figure 1BAs shown, generally, because the infrared touch sensor 12 is higher than the surface of the operation panel 11 (also referred to as a cover plate), the infrared touch sensor 12 is transmitted by filtering through a light filtering strip (also referred to as a light filter), which is usually made by adding a dye to the raw material and then using an injection molding or casting process. The light filtering strip can transmit infrared light while filtering out other ambient light, thereby improving the signal-to-noise ratio of the infrared blocking electrical signal. Therefore, the infrared light scanned by the infrared touch sensor 12 on the surface of the operation panel 11 exists within a certain height range. Thus, the infrared touch sensor 12 forms a touch detection area in the vertical direction of the operation panel 11, and the height H of the touch detection area is relatively large, generally greater than 2 mm (millimeters) and cannot be ignored.

[0048] The process of the infrared touch-sensitive display screen implementing the touch function of the interactive panel: when a touch object (such as a hand or a pen) writes on the operation panel, the general process is to first press down the touch object, move the touch object when it touches the surface of the operation panel, and finally lift up the touch object. As shown in Figure 1C As shown, from the first touch state 131, the second touch state 132, the third touch state 133, the fourth touch state 134 to the fifth touch state 135, the process of the infrared touch-sensitive display screen when the touch object is pressed down (Down), moved (Move) to lifted up (Up) in a touch operation is described.

[0049] In the first touch state 131, the touch object starts to press down and is above the touch detection area, that is, the distance between the touch object and the plane in which the surface of the operation panel 11 is located is greater than H. The infrared light in the scanning process of the infrared touch sensor 12 is not blocked by the touch object, and no infrared blocking electrical signal is generated. At this time, the infrared processor or the central processor does not report touch point data.

[0050] In the second touch state 132, the touch object continues to press down and is in the touch detection area without touching the surface of the operation panel 11, that is, the distance between the touch object and the plane in which the surface of the operation panel 12 is located is greater than 0 and less than H. The infrared light in the scanning process of the infrared touch sensor 12 is blocked by the touch object, generating an infrared blocking electrical signal. At this time, the infrared processor or the central processor reports touch point data according to the infrared blocking electrical signal.

[0051] In the third touch state 133, the touch object is in the touch detection area and has touched the surface of the operation panel 11, and the touch object can move on the surface of the operation panel 11, that is, the distance between the touch object and the plane in which the surface of the operation panel 11 is located is less than or equal to 0 (less than 0 means that the surface of the operation panel 11 is concave under the action of the touch object). Obviously, the distance between the touch object and the plane in which the surface of the operation panel 11 is located is less than H, at this time, the infrared rays in the scanning process of the infrared touch sensor 12 are blocked by the touch object, generating an infrared blocking electrical signal, and the infrared processor or the central processor reports touch point data according to the infrared blocking electrical signal.

[0052] In the fourth touch state 134, the touch object starts to lift up, is in the touch detection area, and does not touch the surface of the operation panel 11, that is, the distance between the touch object and the plane in which the surface of the operation panel 11 is located is greater than 0 and less than H, the infrared rays in the scanning process of the infrared touch sensor 12 are blocked by the touch object, generating an infrared blocking electrical signal, at this time, the infrared processor or the central processor reports touch point data according to the infrared blocking electrical signal.

[0053] In the fifth touch state 135, the touch object continues to lift up and is above the touch detection area, that is, the distance between the touch object and the plane in which the surface of the operation panel 11 is located is greater than H, the infrared rays in the scanning process of the infrared touch sensor 12 are not blocked by the touch object, and no infrared blocking electrical signal is generated, at this time, the infrared processor or the central processor does not report touch point data.

[0054] Therefore, when the infrared touch sensor 12 first scans the touch object, the touch object does not contact the operation panel 11, and the touch object needs to continue to move to the operation panel in the direction perpendicular to the operation panel 11 by a height H before contacting the operation panel 11. That is, when the infrared touch sensor 12 first scans the touch object, a period of time needs to elapse before the touch object touches the operation panel 11. If the touch object has not touched the operation panel 11 within a period of time after the infrared touch sensor 12 first scans the touch object, it means that the touch object may have accidentally entered the touch detection area of the infrared touch sensor 12, and not a touch operation on the operation panel 11. In this case, that is, in the second touch state 132 described above, the infrared rays scanned by the infrared touch sensor 12 are blocked by the touch object, the infrared touch sensor 12 generates an infrared blocking electrical signal, at this time, the infrared processor or the central processor still reports touch point data. That is, when the touch object writes on the operation panel of the infrared touch-sensitive display screen, touch point data is generated when the touch object enters the touch detection area of the infrared touch sensor 12 but does not contact the operation panel.

[0055] In addition, the current method of determining the area of the touch object based on the touch point data is only applicable to the case where the touch object is perpendicular to the operation panel 11. When the touch object is inclined to touch the operation panel, the range of the infrared rays blocked by the touch object will be larger, as shown in Figure 2 , which results in a larger area of the touch object determined based on the touch point data, and thus the type of the touch object is incorrectly identified.

[0056] In another case, when a capacitive touch screen is provided on the interactive device, the user currently uniformly uses capacitive touch operation on the interactive device, i.e., pressing the touch object until the touch object touches the surface of the screen, so as to realize the touch operation on the interactive device. As shown in Figure 3 Figure 3 is a structural diagram of the capacitive touch screen 14. When the touch object contacts the metal layer of the capacitive touch screen 14, the capacitance at the touch point on the metal layer changes, as shown by the darker gray squares in Figure 3 , which causes the frequency of the oscillator connected to the metal layer to change. The position of the touch point and the area of the touch object can be determined by measuring the frequency change of the oscillator, so that the interactive device can determine the type of the touch object based on the area. However, when the touch object is an insulating touch object, the capacitive touch screen 14 cannot effectively identify the touch object, and cannot identify the area of the touch object, which easily results in incorrect identification of the type of the touch object. In another embodiment, when the touch object contacting the capacitive touch screen is an active capacitive pen, the position detection circuit inside the active capacitive pen collects the electrical signals on the capacitive touch screen to determine the position information of the active capacitive pen on the capacitive touch screen, and sends the position information to the interactive device. At the same time, the interactive device can also determine the type of the active capacitive pen based on the signal sent by the active capacitive pen. However, this method requires the touch object to be an active device, and the touch object also needs to have the function of sending signals to the interactive device, which is a harsh implementation condition and cannot be applied to most use scenarios.

[0057] Therefore, embodiments of the present application provide a material identification method, as shown in Figure 4 Figure 4 ​​This is a flowchart illustrating a material identification method provided in an embodiment of this application. The material identification method provided in this application can be executed by a material identification device, which can be implemented through software and / or hardware. The material identification device can consist of two or more physical entities, or it can consist of a single physical entity. For example, the material identification device can be an interactive device such as a mobile phone, an interactive tablet, or a large display panel. The material identification method provided in this application is used in an interactive tablet, which includes an operation panel, an infrared touch sensor, and an elastic wave sensor. The infrared touch sensor is disposed at at least one edge of the operation panel to form a touch detection area of ​​the interactive tablet, and the elastic wave sensor is used to detect vibrations of the operation panel and generate electrical signals.

[0058] An interactive flat panel refers to a tablet computer capable of interacting with a user. For example, it may be a tablet product that interacts with a user through multi-point infrared or optical interactive touch technologies. It should be further noted that the structure of the interactive flat panel provided in this embodiment is as follows: Figure 5 As shown, the interactive flat panel includes an operation panel 11, an infrared touch sensor 12, and an elastic wave sensor 15. The infrared touch sensor 12 is disposed on at least one edge of the operation panel 11. The operation panel 11 serves as the link between the user and the interactive flat panel. The interactive flat panel can display different interfaces or content through the operation panel 11, and the user can send different commands to the interactive device through gestures or touch operations on the operation panel 11. For example, the operation panel 11 can be a touch screen. In this embodiment, the detection range of the infrared touch sensor can be determined based on the range covered by the infrared rays emitted by the infrared touch sensor on different edges, thereby determining the touch detection area of ​​the infrared touch sensor on the operation panel 11 of the interactive flat panel. Additionally, the elastic wave sensor 15 disposed on the interactive flat panel is used to detect the elastic waves generated on the operation panel 11 due to the contact of a touch object when the operation panel 11 vibrates, and generates an electrical signal based on the elastic waves, the electrical signal including the waveform information of the elastic waves.

[0059] It should be further explained that when an object touches the surface of the control panel, the surface deforms at the point of contact, generating elastic waves that propagate within the control panel. Elastic waves are a type of stress wave, and stress waves are the propagation form of stress and strain disturbances; that is, elastic waves are the form in which stress and strain caused by disturbances or external forces are transmitted in an elastic medium. In an elastic medium, there are elastic forces interacting between particles. When a particle moves from its equilibrium position due to disturbance or external force, the elastic restoring force causes that particle to vibrate, thereby causing displacement and vibration of surrounding particles. This vibration then propagates within the elastic medium, accompanied by energy transfer. Stress and strain change at the point of vibration.

[0060] The frequency of the elastic wave generated by the touch object touching the operation panel surface of the interactive panel is determined by the two contact media, i.e., the touch object and the operation panel. During the touching process, a low-frequency fundamental wave and high-order harmonics are generated. The energy of the low-frequency fundamental wave is generally much higher than that of the high-order harmonics. Therefore, in the embodiment, the resonance frequency of the elastic wave sensor is designed to be consistent with the frequency of the fundamental wave generated by some commonly used touch objects touching the operation panel surface of the interactive panel, and the frequency of the fundamental wave is positioned as the working frequency for material identification, so as to improve the signal-to-noise ratio of the sensor.

[0061] In one embodiment, at least one elastic wave sensor is installed on the operation panel of the interactive panel. The working frequency of the elastic wave sensor is substantially consistent with the frequency of the elastic wave generated by the touch object touching the surface of the operation panel for distinguishing different materials. In this case, the sensitivity of detecting the elastic wave generated by the touch object writing on the operation panel can be enhanced, so that the elastic wave sensor detects the elastic wave generated by the touch object touching the surface of the operation panel, thereby generating a high-quality electrical signal and improving the accuracy of detecting the material of the touch object.

[0062] Since the elastic wave can be transmitted in the operation panel, in theory, the elastic wave sensor can be installed at any position of the elastic wave generated in the operation panel, which can be any position on the operation panel or any position in direct or indirect contact with the operation panel, and all of them can achieve the detection of the elastic wave generated by the touch object touching the surface of the operation panel.

[0063] In one embodiment, the elastic wave sensor can be a piezoelectric sensor. The piezoelectric sensor is a sensing component made by using the piezoelectric effect of some dielectric under stress. The piezoelectric effect refers to the phenomenon that some dielectric will generate electric charge on its surface when it is deformed (including bending and stretching deformation) under the action of external force in a certain direction, due to the change of the internal electric charge distribution of the material.

[0064] Further, the piezoelectric sensor is affected by the elastic wave and generates an electrical signal by applying the piezoelectric effect. The piezoelectric effect refers to the phenomenon that some dielectric in the piezoelectric sensor will generate a certain electric charge on the electrode surface when it is deformed by force in a certain direction (such as the deformation generated by the touch object touching the surface of the operation panel), and is in a charged state. When the external force is removed, it returns to the normal non-charged state.

[0065] Generally, the operation panel of the interactive panel is large in size, and the interactive panel provides high freedom of services such as electronic whiteboard and document annotation. A user can touch the operation panel of the interactive panel at any coordinate on the surface of the operation panel by using a touch object, so as to trigger a touch operation (such as clicking an icon, writing a trace, and the like). The position of the elastic wave sensor installed on the operation panel is fixed, and the posture of the elastic wave sensor is fixed. The sensitivity and accuracy of the elastic wave sensor for detecting an electric signal generated at different coordinates on the operation panel are different. Generally, the closer the coordinate triggering the touch operation to the elastic wave sensor, the higher the accuracy of the elastic wave sensor in detecting the elastic wave. Conversely, the farther the coordinate triggering the touch operation to the elastic wave sensor, the lower the accuracy of the elastic wave sensor in detecting the elastic wave.

[0066] It should be noted that the coordinate triggering the touch operation is too close to the elastic wave sensor, which can cause the electric signal generated by the elastic wave sensor to exceed the maximum voltage of the operational amplifier, resulting in distortion. In the case of maintaining a certain sensitivity and accuracy, it can be considered that there is a detection range for the elastic wave sensor in detecting the elastic wave. Therefore, the number and position of the elastic wave sensors installed on the operation panel of the same interactive panel can be determined according to the relationship between the detection range and the operation panel of the interactive panel.

[0067] It should be noted that the coordinate triggering the touch operation is too close to the elastic wave sensor, which can cause the electric signal generated by the elastic wave sensor to exceed the maximum voltage of the operational amplifier, resulting in distortion. In the case of maintaining a certain sensitivity and accuracy, it can be considered that there is a detection range for the elastic wave sensor in detecting the elastic wave. Therefore, the number and position of the elastic wave sensors installed on the operation panel of the same interactive panel can be determined according to the relationship between the detection range and the operation panel of the interactive panel.

[0068] The material identification method provided by the embodiment of the present application comprises:

[0069] In step 101, when the touch object performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared shielding electric signal from the first time.

[0070] In the embodiment, when the touch object performs a touch operation in the touch detection area of the infrared touch sensor, the infrared rays emitted by the infrared touch sensor scan the touch object, and the infrared touch sensor starts to generate an infrared blocking electrical signal. The first time point refers to the time when the infrared rays of the infrared touch sensor first scan the touch object.

[0071] Step 102, starting from the first time point, the elastic wave sensor starts to generate an original touch electrical signal.

[0072] At the same time, starting from the first time point, the elastic wave sensor generates an original touch electrical signal according to the vibration of the operation panel. The original touch electrical signal is an electrical signal generated by the elastic wave sensor after the first time point according to the vibration of the operation panel.

[0073] It should be noted that the elastic wave sensor starts to generate the original touch electrical signal from the first time point, which does not mean that the original touch electrical signal is generated at the same time as the first time point, but that the elastic wave sensor will generate the original touch electrical signal after the first time point. Based on the height H touch detection area formed by the infrared touch sensor described in detail above, it can be understood that for a complete touch operation, the touch object first enters the touch detection area with a height of H, and then contacts the operation panel with a height of 0. Corresponding to the movement process of the touch object, the generation of the infrared blocking signal is earlier than that of the touch electrical signal, that is, the touch electrical signal is not generated at the same time as the first time point, but lags behind the first time point. The absolute time length of this lag may be very small, but the relative sequence cannot be eliminated. In the present scheme, the first time point is defined, and the generation of the original touch electrical signal with the first time point as a reference, so as to confirm the real starting time of the touch electrical signal, that is, to confirm the accurate time when the touch object contacts the operation panel, so as to confirm the infrared blocking signal generated when the touch object contacts the operation panel, and then to realize the touch response corresponding to the actual contact position and time accurately according to the confirmed infrared blocking signal.

[0074] Step 103, determining the first material information of the touch object according to the infrared blocking electrical signal and the original touch electrical signal.

[0075] After the infrared blocking electrical signal and the original touch electrical signal are generated, the first material information of the touch object is determined according to the infrared blocking electrical signal and the original touch electrical signal. The first material information includes information of the material of the touch object, for example, the first material information is a stylus or a chalk, etc. In an embodiment, in the process of determining the first material information of the touch object, first, the probabilities corresponding to different materials can be generated according to the original touch electrical signal, for example, the probability of the chalk is 65%, the probability of the stylus is 35%, etc. Then, the probabilities are further calibrated using the infrared blocking electrical signal to obtain target probabilities. For example, according to the infrared blocking electrical signal, the area of the touch object can be determined, for example, the area of the touch object is 5mm 2 or 10mm 2 , etc. Then, the probabilities corresponding to different materials are further adjusted based on the area to obtain the target probabilities. Finally, the first material information of the touch object is generated according to the target probabilities.

[0076] Step 104, determining whether the first material information is obtained within a first time length after the first time, and taking the first material information as the material type of the touch object.

[0077] In the embodiment, if the first material information is obtained within the first time length after the first time, the first material information is taken as the material type of the touch object. It needs to be further explained that the reason for setting the first time length in the embodiment is that, in the process of using the interactive panel, the situation that the touch object blocks the infrared touch sensor but does not contact the operation panel or the situation that the elastic wave data is missed may occur. Therefore, by setting the first time length, whether the touch object blocks the infrared touch sensor and the touch object contacts the interactive panel is a coherent action. If the first material information is obtained within the first time length after the first time, it indicates that the touch object blocks the infrared touch sensor and then touches the operation panel in a very short time, that is, the touch action is a valid touch action. At this time, the first material information can be taken as the material type of the touch object. Otherwise, the situation that the original touch electrical signal is missed or the touch object accidentally blocks the infrared sensor may occur. At this time, since the first material information cannot be obtained from the original touch electrical signal, the material information is not taken as the material type of the touch object. It can be understood that the first time length can be set according to actual needs in the embodiment. For example, considering the delay that can be received by the user in the process and the time required to confirm the material type of the touch object, the first time length can be set to 20-50 milliseconds. In an embodiment, the first time length can be set to 32 milliseconds.

[0078] Step 105, determining whether the first material information is obtained within the first time length after the first time, and obtaining the second material information as the material type of the touch object, wherein the second material information is generated according to the infrared blocking electrical signal.

[0079] If the first material information cannot be obtained within the first time duration after the first time point, in order to avoid too large calculation delay or too long waiting time for the original touch signal due to the elastic wave missing, the second material information is generated according to the infrared blocking electric signal, and the second material information is used as the material type of the touch object, wherein the second material information also includes information of the material of the touch object. In an embodiment, in the process of generating the second material information according to the infrared blocking electric signal, the area of the touch object can be determined according to the infrared blocking electric signal, and then the second material information of the touch object is further generated according to the area of the touch object. For example, the touch area range corresponding to different touch objects is stored in advance, so that the second material information of the touch object can be generated according to the area of the touch object. For example, the touch area range of the stylus is 3-6mm 2 , the touch area range of the chalk is 8-12mm 2 , when the area of the touch object is 5mm 2 , the second material information generated at this time is the stylus, when the area of the touch object is 10mm 2 , the second material information generated at this time is the chalk.

[0080] In the above, when the touch object performs a touch operation in the touch detection area, the infrared touch sensor and the elastic wave sensor respectively send the infrared blocking electric signal and the original touch electric signal. If the first material information can be generated according to the infrared blocking electric signal and the original touch electric signal within the first time duration after the first time point when the infrared blocking electric signal is generated, the material type of the touch object is determined according to the first material information; otherwise, the material type of the touch object is determined according to the second material information generated according to the infrared blocking electric signal. The material identification of the touch object is combined with the infrared blocking electric signal and the original touch electric signal in the embodiment of the application, which can improve the accuracy of material identification, overcome the low material identification rate caused by the fact that the infrared blocking electric signal does not have material specificity and the original touch electric signal is easily disturbed by vibration, and solve the technical problem that the interactive device in the prior art cannot accurately identify the material of the touch object.

[0081] In the embodiments of the present application, by setting the elastic wave sensor, the original touch electric signal generated by the elastic wave sensor when the touch object operates on the operation panel can be collected, so that the material information of the touch object can be obtained in combination with the original touch electric signal. In the material identification process, how to locate the starting time of the effective touch electric signal in the original touch electric signal is related to the problems of identification efficiency and identification accuracy. One solution (hereinafter referred to as solution one) in the prior art is to take the time when the infrared touch sensor starts to generate the infrared shielding electric signal as the starting time of the effective touch electric signal, and take the original touch electric signal after the starting time as the effective touch electric signal, and calculate the material information of the touch object according to the effective touch electric signal. However, as described above, when the touch object writes on the operation panel, the infrared touch sensor will generate the infrared shielding electric signal when entering the touch detection area of the infrared touch sensor but not contacting the operation panel, and at this time the touch object has not actually contacted the operation panel. Therefore, according to this way of determining the effective touch electric signal, there is a time difference between the starting time of the effective touch electric signal and the real starting time of the effective touch electric signal, and the signal in this time difference is actually a noise signal, which will adversely affect the identification accuracy and speed of the material of the touch object.

[0082] To solve the above problems, the inventors creatively propose a solution mechanism (hereinafter referred to as solution two) to determine the starting time according to the signal-to-noise ratio change of the original touch electric signal, and extract the effective touch electric signal in the original touch electric signal by setting a preset threshold and combining whether the signal-to-noise ratio change of the original touch electric signal is greater than the preset threshold. Specifically, the signal-to-noise ratio refers to the ratio of signal to noise in an electronic device or electronic system. In one embodiment, the signal-to-noise ratio refers to the ratio between the original touch electric signal and the noise. When the signal-to-noise ratio is greater than a certain value (preset threshold), it means that the value of the original touch electric signal at this time is relatively large, and the original touch electric signal is not generated by noise, but by the mutual touch between the touch object and the operation panel, so the starting time of the effective touch electric signal can be determined. Specifically, the signal-to-noise ratio can be obtained by calculating the ratio of the energy average value of the original touch electric signal in the current time window and the historical energy average value. The historical energy average value is the energy average value of the original touch electric signal in the previous time window. If the ratio is greater than the preset threshold, it means that the original touch electric signal detected in the current window has a larger amplitude relative to the original touch electric signal detected in the previous time window, and the change of this amplitude is difficult to be generated by the environment, so the change of this amplitude can be considered to be generated by the mutual touch between the touch object and the operation panel. Therefore, the initial time of the current window can be taken as the starting time of the original touch electric signal, that is, the time when the touch object and the operation panel first touch.

[0083] Specifically, as shown in Figure 6 , the signal-to-noise ratio of the original touch electric signal is calculated, and the starting time of the effective touch electric signal is determined according to the signal-to-noise ratio of the original touch electric signal. Figure 6The material identification method provided in the second scheme of the embodiments of the present application is a specific embodiment of the material identification method. Referring to Figure 6 The material identification method comprises the following steps.

[0084] In step 201, when the touch object performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared shielding electric signal from a first time point.

[0085] In step 202, the elastic wave sensor generates an original touch electric signal from the first time point.

[0086] In step 203, the original touch electric signal is used to determine a starting time point of an effective touch electric signal in combination with a preset threshold value.

[0087] In this embodiment, after the original touch electric signal is obtained, the starting time point of the effective touch electric signal is further determined based on the original touch electric signal in combination with the preset threshold value. The original touch electric signal generated by the elastic wave sensor can include noise signals and interference signals, and the effective touch electric signal contained therein needs to be further confirmed. In this embodiment, the original touch electric signal collected by the elastic wave sensor is calculated to obtain the starting time point of the effective touch electric signal contained therein, so that the effective touch electric signal can be correctly positioned. The starting time point is the time point at which the touch object first contacts the operation panel.

[0088] In the above embodiment, the starting time point of the effective touch electric signal is determined based on the original touch electric signal in combination with the preset threshold value in step 203, which comprises the following steps.

[0089] In step 2031, an energy average value of the original touch electric signal in a current time window is calculated.

[0090] The time window refers to a fixed time length. The time point at which the previous time window ends is the time point at which a new time window starts. The time length of the time window can be set according to actual needs, for example, the time length of the time window can be set to 5 milliseconds or 10 milliseconds, etc. In this embodiment, the specific time length of the time window is not limited. In this embodiment, the energy of the original touch electric signal detected by the elastic wave sensor is determined in real time in the current time window, and the average value of the energy is calculated. The energy of the original touch electric signal refers to the sum of the kinetic energy and potential energy of the elastic medium. Since the elastic wave propagates in the elastic medium, each particle vibrates near its equilibrium position, so that the elastic medium has vibration kinetic energy. Because the vibration causes the distance between particles to change, the medium deforms, so that it has deformation potential energy. The sum of the vibration kinetic energy and the deformation potential energy is called the energy of the elastic wave in the medium.

[0091] Step 2032, calculate the ratio of the energy average value in the current time window and the history energy average value which is the energy average value of the original touch electric signal in the last time window.

[0092] After the energy average value in the current time window is calculated, the ratio of the corresponding energy average value in the current window and the history energy average value in the last time window is further calculated. The history energy average value is the energy average value of the original touch electric signal detected in the last time window.

[0093] Step 2033, judge whether the ratio is greater than a preset threshold value.

[0094] Step 2034, if yes, obtain the initial time of the current time window and take the initial time of the current time window as the starting time of the effective touch electric signal.

[0095] The preset threshold value refers to a threshold value set in advance. In the embodiment, in order to distinguish the original touch electric signal under the environmental noise and the original touch electric signal when the touch object contacts the interactive panel, the preset threshold value can be set as a value of 4 or more. If the ratio is greater than the preset threshold value, it indicates that the original touch electric signal detected in the current window has a larger amplitude compared with the original touch electric signal detected in the last time window, and the change of the amplitude is difficult to be caused by the environment, so the change of the amplitude can be considered as being caused when the touch object contacts the interactive panel. Therefore, the initial time of the current window can be taken as the starting time of the effective touch electric signal, that is, the time when the touch object first contacts the interactive panel.

[0096] By the method of combining the preset threshold value with the signal-to-noise ratio judgment in the second scheme, the effective elastic wave signal can be effectively extracted from the original elastic wave signal, so as to identify the material information of the touch object by combining the classification model or the neural network recognition algorithm. Compared with the scheme in the prior art for determining the starting time of the effective elastic wave signal based on the infrared shielding electric signal, the material information of the touch object can be more accurately identified by the scheme.

[0097] However, the inventors found in the application of the second solution that, for the interactive panel, due to the large size of the interactive panel, generally more than 65 inches, even 86 inches or more, the internal working circuit is relatively complex, and environmental noise may be generated during the working process. Moreover, due to the diversified use scenarios of the interactive panel, it is often used to play audio and video information or conduct video conference, etc., and in the process of the loudspeaker sounding, environmental noise is also brought. The noise frequency of the above environmental noise of the interactive panel is not stable and has no regularity, and it is impossible to classify or identify the noise through a classification model or a neural network recognition algorithm. In actual application, the environmental noise signal may also satisfy the condition that the signal-to-noise ratio is greater than the preset threshold, so that the effective elastic wave signal extracted is not accurate and may contain environmental noise, resulting in a decrease in the accuracy of the identification result of the material information of the touch object.

[0098] In order to solve the problems of the second solution, the inventors provide another solution mechanism (hereinafter referred to as the third solution): in the absence of a touch object, the preset threshold is dynamically adjusted according to the noise in the current environment, so as to avoid the case that the noise is incorrectly identified as the touch object and the operation panel being in contact due to the too large noise.

[0099] Specifically, it further includes:

[0100] Step 2035, dynamically updating the preset threshold according to the first electric signal generated by the infrared touch sensor and the second electric signal generated by the elastic wave sensor.

[0101] In the embodiment, the preset threshold value can be dynamically updated according to the first electric signal generated by the infrared touch sensor and the second electric signal generated by the elastic wave sensor. The first electric signal refers to the electric signal generated by the infrared touch sensor, and the second electric signal refers to the electric signal generated by the elastic wave sensor. The first and second are only used to distinguish the different hardware from which the electric signals are derived. In addition, the first electric signal and the second electric signal can be generated after the infrared touch sensor is started and the elastic wave sensor is started, respectively. When it is determined that there is a touch object according to the first electric signal, the first electric signal at this time is the infrared shielding electric signal, that is, the infrared shielding electric signal is actually a specific signal identified from the first electric signal. There are only two kinds of shielding and non-shielding for the touch object, so if the first electric signal is not the infrared shielding electric signal, it is defaulted that there is no touch object at present. When it is determined that the touch object contacts the interactive panel according to the second electric signal, the second electric signal at this time is the original touch electric signal, and for the same reason, the original touch electric signal is actually a specific signal identified from the second electric signal, so if the second electric signal is not the original touch electric signal, it is defaulted that the touch object does not touch the interactive panel. For the original touch electric signal, the signal state of the second electric signal when the touch object does not touch should be taken as a reference, and the originally set preset threshold value deviates from the signal state of the second electric signal when there is no touch in the actual use process. Therefore, when it is determined that there is no touch object according to the first electric signal generated by the infrared touch sensor, the preset threshold value is dynamically updated according to the amplitude of the second electric signal generated by the current elastic wave sensor, so that the case that noise is incorrectly identified as the touch object contacting the operation panel when the environment noise is too large can be avoided. Specifically, the preset threshold value is dynamically updated according to the first electric signal generated by the infrared touch sensor and the second electric signal generated by the elastic wave sensor in step 2035, including:

[0102] In step 20351, when it is determined that there is no touch object according to the first electric signal generated by the infrared touch sensor, it is judged whether the instantaneous amplitude of the second electric signal generated by the elastic wave sensor is greater than the preset threshold value.

[0103] First, when it is determined that there is no touch object according to the first electric signal generated by the infrared touch sensor, it is judged whether the instantaneous amplitude of the second electric signal generated by the elastic wave sensor is greater than the preset threshold value. When it is determined that the first electric signal is not the infrared shielding electric signal, it is determined that there is no touch object in the touch detection area, and the second electric signal detected by the current elastic wave sensor is further acquired. At this time, the second electric signal reflects the bottom noise of the interactive panel, that is, the environmental noise (mainly including circuit working noise, etc.). By judging whether the instantaneous amplitude of the second electric signal is greater than the preset threshold value, the size of the current environmental noise can be judged, so as to dynamically adjust the preset threshold value. The instantaneous amplitude refers to the amplitude of the energy of the current second electric signal.

[0104] Step 20352, when the instantaneous amplitude of the second electric signal is greater than the preset threshold value, setting the preset threshold value as the first preset threshold value.

[0105] When the instantaneous amplitude of the second electric signal is greater than the preset threshold value, it indicates that the environmental noise of the interactive panel is large at this time, and the value of the preset threshold value needs to be increased to avoid the situation that the environmental noise is too large to be incorrectly identified as the touch object and the operation panel being in contact. Therefore, the preset threshold value is set as the first preset threshold value with a larger value.

[0106] Step 20353, when the instantaneous amplitude of the second electric signal is less than the preset threshold value, setting the preset threshold value as the second preset threshold value.

[0107] If the instantaneous amplitude of the second electric signal is less than the preset threshold value, it indicates that the environmental noise of the interactive panel is small at this time, and therefore the preset threshold value can be set as the second preset threshold value with a smaller value. When the instantaneous amplitude of the second electric signal is equal to the preset threshold value, either the first preset threshold value or the second preset threshold value can be selected as the preset threshold value at this time, and the specific process is shown in Figure 7 In an embodiment, the first preset threshold value can be set as 10, and the second preset threshold value can be set as 4.

[0108] For example, when a teacher uses the interactive panel to play audio and video in a teaching process, the speaker in the interactive panel will vibrate and produce environmental noise at the same time. At this time, when the interactive panel determines that there is no touch object according to the first electric signal generated by the infrared touch sensor, it further determines the instantaneous amplitude of the second electric signal generated by the elastic wave sensor. When the instantaneous amplitude of the second electric signal is greater than the preset threshold value, it indicates that the environmental noise of the interactive panel is large at this time, and the value of the preset threshold value needs to be increased to avoid the situation that the environmental noise produced by the speaker is incorrectly identified as the touch object and the operation panel being in contact. The interactive panel can set the preset threshold value as the first preset threshold value with a larger value, for example, set the first preset threshold value as 10. If the instantaneous amplitude of the original touch electric signal is less than the preset threshold value, it indicates that the environmental noise of the interactive panel is small at this time (i.e., the speaker is not sounding), and the interactive panel can set the preset threshold value as the second preset threshold value with a smaller value, for example, set the second preset threshold value as 4.

[0109] The above is the process of dynamically setting the preset threshold according to scheme three. According to scheme three, when it is determined that there is no effective touch object and the instantaneous amplitude of the second electric signal is greater than the preset threshold, it indicates that the interactive panel is in a working state, at this time, the environmental noise is large, therefore, the preset threshold is adjusted to the first preset threshold which is larger, to avoid the case that the environmental noise is misrecognized as an effective touch electric signal when the touch object contacts the interactive panel subsequently. If the instantaneous amplitude of the second electric signal is less than the preset threshold, it indicates that the environmental noise of the interactive panel at this time is small, therefore, the preset threshold can be set to the second preset threshold which is smaller, to improve the detection sensitivity of the effective touch electric signal. According to scheme three, when the touch object operates on the interactive panel, for some click operations and the like, the noise can be filtered out well, and the effective touch electric signal can be extracted accurately, to ensure high detection accuracy and avoid inaccurate extraction results due to noise.

[0110] In an embodiment of the present application, the following steps are further included:

[0111] Step 204: obtaining the effective touch electric signal in a preset time length after the starting time as the elastic wave data.

[0112] After the starting time of the effective touch electric signal is determined in step 203, the effective touch electric signal generated in a preset time length after the starting time can be further obtained as the elastic wave data. The preset time length can be set according to actual needs, and the specific value of the preset time length is not limited in the embodiment.

[0113] Step 205: determining the first material information of the touch object according to the infrared shielding electric signal and the elastic wave data.

[0114] Step 206: determining that the first material information is obtained in a first time length after the first time, and taking the first material information as the material type of the touch object.

[0115] Step 207: determining that the first material information is not obtained in the first time length after the first time, and obtaining the second material information as the material type of the touch object, wherein the second material information is generated according to the infrared shielding electric signal.

[0116] According to the embodiment of the present application, when no touch object is detected, the noise of the current environment is estimated according to the instantaneous amplitude of the second electric signal generated by the elastic wave sensor, and the preset threshold is dynamically adjusted according to the noise of the environment. When the noise of the environment is large, the preset threshold is set to the first preset threshold with a larger value. When the noise of the environment is small, the preset threshold is set to the second preset threshold with a smaller value. By dynamically adjusting the preset threshold, the embodiment of the present application can avoid the interference of the environmental noise on the interactive panel, so as to avoid the case that the environmental noise is incorrectly recognized as the touch object contacting the operation panel. Therefore, when the touch object performs a touch operation on the interactive panel, the starting moment of the effective touch electric signal can be more accurately determined to obtain effective elastic wave data, and the accuracy of subsequent material identification using the effective elastic wave data is improved.

[0117] As shown in Figure 8 , Figure 8 A material identification method is provided in the embodiment of the present application, which is a specific embodiment of the above-mentioned material identification method. Referring to Figure 8 , the material identification method comprises the following steps.

[0118] Step 301: When a touch object performs a touch operation on the touch detection area, the infrared touch sensor generates an infrared shielding electric signal from a first moment.

[0119] Step 302: From the first moment, the elastic wave sensor starts to generate an original touch electric signal.

[0120] Step 303: Based on the original touch electric signal, the starting moment of the effective touch electric signal is determined in combination with a preset threshold.

[0121] Step 304: The effective touch electric signal within a preset time length after the starting moment is obtained as elastic wave data.

[0122] Step 305: Probabilities corresponding to different materials are generated according to the elastic wave data.

[0123] In the embodiment, after obtaining the effective elastic wave data, probabilities corresponding to different materials are further generated according to the elastic wave data, where the probability represents the size of the possibility of an event. In the embodiment, the probability represents the possibility of the touch object belonging to different materials. For example, if the probability of the touch object belonging to a finger is 55%, or the probability of the touch object belonging to a touch pen is 80%, etc. In an embodiment, the elastic wave data can be input into a trained neural network to obtain probabilities corresponding to different materials.

[0124] On the basis of the above-mentioned embodiment, in step 305, the probabilities corresponding to different materials are generated according to the elastic wave data, which comprises the following steps.

[0125] Step 3051, pre-processing the elastic wave data to obtain target data.

[0126] Firstly, in order to eliminate the influence of the difference of the elastic wave sensors and the different modal elastic wave data caused by the touch position on the subsequent calculation results, it is necessary to pre-process the elastic wave data first to obtain target data from the elastic wave data. For example, invalid data in the elastic wave data can be first removed, such as amplitude clipping distorted elastic wave data or elastic wave data with too low energy, and then the elastic wave data is fused to obtain the target data.

[0127] In one embodiment, the elastic wave data is generated according to the effective touch electric signals corresponding to the plurality of elastic wave sensors.

[0128] It should be noted that in the present embodiment, a preset time length is taken as a data processing period, and the elastic wave data generated in the data processing period can be described or recorded in a set manner, and each element in the set is generated according to the effective touch electric signal corresponding to an elastic wave sensor. For example, the elastic wave data set D1=(d1, d2, ···dn) is obtained, where d n T , T is a preset time length, di is the elastic wave data obtained according to the effective touch electric signal of the i-th elastic wave sensor, n is the number of elastic wave sensors, i

[0129] Correspondingly, the pre-processing of the elastic wave data in step 3051 to obtain the target data includes:

[0130] Step 30511, time-frequency transform is performed on the elastic wave data corresponding to the plurality of elastic wave sensors to obtain frequency domain data corresponding to the elastic wave data.

[0131] Firstly, time-frequency transform is performed on the elastic wave data corresponding to the plurality of elastic wave sensors to obtain frequency domain data corresponding to the elastic wave data, wherein the time-frequency transform refers to converting time domain data into frequency domain data. For example, Fourier transform can be performed on each elastic wave data in the elastic wave data set D1 to obtain the corresponding frequency domain data D2. It can be understood that the frequency domain data can also be described or recorded in a set manner, that is, the frequency domain data set includes the frequency domain data corresponding to each elastic wave data in the elastic wave data set D1. In one embodiment, before time-frequency transform, the amplitude of each data in the elastic wave data set D1 can be compared with the preset effective signal amplitude to determine the amplitude clipping distorted data (i.e. the data whose value is greater than the effective signal amplitude), and the amplitude clipping distorted data is removed.

[0132] ​​Step 30512, multi-channel signal fusion is performed on the frequency domain data to obtain fused data.

[0133] After obtaining the frequency domain data, multi-channel fusion is performed on the frequency domain data to obtain fused data. The multi-channel fusion refers to fusing each data in the frequency domain data set to obtain the fused data. In this embodiment, the purpose of performing multi-channel signal fusion is to eliminate the differences between the elastic wave sensors and the influence of different modal elastic wave data generated by the touch position on the subsequent calculation results. The difference between the elastic wave sensors refers to the fact that, due to the influence of production process or production environment during the production process of the elastic wave sensors, there are slight differences between the elastic wave sensors produced on the same production line. Even if two elastic wave sensors produced on the same production line are placed at the same position, the elastic waves collected by the two elastic wave sensors will still have slight differences. The different modal elastic waves generated by the touch position refer to the fact that, when the elastic wave propagates to different positions, the elastic wave collected by the elastic wave sensor has different modalities. For example, as shown in FIG. 11, the area of the operation panel 11 is as shown in FIG. 11, and the elastic wave sensors 15 are arranged below the positions 1-12 of the operation panel 11. When the touch point of the touch object and the operation panel 11 is position 1, the elastic wave sensors 15 at different positions will detect different modal elastic waves, as shown in FIG. 12. Figure 9 Figure 10 Figure 10 FIGS. 12A-12L are schematic diagrams of different modal elastic waves detected by the elastic wave sensors 15 at positions 1-12, respectively. Since each elastic wave sensor can only reflect part of the information of the vibration characteristics of the touch object, the material identification accuracy of each elastic wave sensor is limited, resulting in poor material identification accuracy. Therefore, in this embodiment, the frequency domain data corresponding to the elastic wave is fused, so that the expression of the material information can be enhanced, and the influence of the difference between the signals of each elastic wave sensor on the subsequent calculation process due to the different touch positions can be avoided.

[0134] In this embodiment, the multi-channel signal fusion can be performed by weighted average or neural network, so as to obtain the target data. Specifically, different multi-channel signal fusion methods are described below.

[0135] In one embodiment, the multi-channel signal fusion is performed on the frequency domain data to obtain fused data, including:

[0136] The frequency domain data is weighted averaged to obtain the fused data.

[0137] In one embodiment, the frequency domain data can be weighted averaged to obtain the fused data. For example, it is assumed that the frequency domain data is D2=(d 21 ,d 22 ···d​​2n T The formula for weighted average of the frequency domain data is as follows:

[0138]

[0139] wherein D3 is the fusion data, w i (i=1, 2...n) is an empirical weight parameter, d 2i is the i-th data in the frequency domain data D2.

[0140] In an embodiment, the frequency domain data is subjected to multi-channel signal fusion to obtain the fusion data, comprising:

[0141] The frequency domain data is input into a preset first neural network to obtain the fusion data.

[0142] In another embodiment, the frequency domain data can be input into a trained first neural network to obtain the fusion data, specifically as follows:

[0143]

[0144] wherein c i , h i (i=1, 2...n) is a training weight parameter of the first neural network, and f is an activation function of the first neural network. The frequency domain data D2 is input into the trained first neural network for calculation, and the fusion signal can be obtained.

[0145] Step 30513, filtering the fusion data to obtain target data.

[0146] Finally, filtering the fusion data can obtain the required target data.

[0147] In an embodiment, the elastic wave data is preprocessed in step 3051 to obtain the target data, comprising:

[0148] Step 30514, calculating the system function of the elastic wave propagating to each elastic wave sensor.

[0149] In the embodiment, the system function of the elastic wave propagating to each elastic wave sensor is first calculated. The system function is a rational function of complex variable S with real coefficients, i.e., a real rational function. The process of calculating the system function of the elastic wave can refer to the process of calculating the system function in the prior art, which will not be described in detail in the embodiment.

[0150] Step 30515, dividing the elastic wave data corresponding to each elastic wave sensor by the corresponding system function to obtain the first data corresponding to each elastic wave sensor.

[0151] ​After calculating the system function corresponding to each elastic wave sensor, the elastic wave data corresponding to each elastic wave sensor is divided by the corresponding system function to obtain the first data corresponding to each elastic wave sensor. Specifically, in this embodiment, the elastic wave data corresponding to each elastic wave sensor in the elastic wave data set is divided by the system function corresponding to the elastic wave sensor to obtain the first data corresponding to each elastic wave sensor.

[0152] Step 30516: Calculate the average of all the first data to obtain the target data.

[0153] After obtaining the first data corresponding to each elastic wave sensor, the average of all the first data is calculated to obtain the target data. In one embodiment, the formula for calculating the target data D4 is as follows:

[0154]

[0155] Among them, H i This represents the system function corresponding to the i-th elastic wave sensor.

[0156] The above describes the process of preprocessing the elastic wave data in step 3051 to obtain the target data.

[0157] Step 3052: Input the target data into the preset neural network to obtain the probability corresponding to different materials.

[0158] Once the target data set is obtained, it is input into a trained neural network to obtain the probabilities corresponding to different materials. In one embodiment, the neural network is a multi-layer fully connected network, including a first fully connected layer, a second fully connected layer, a third fully connected layer, a first batch normalization (BN) layer, a second batch normalization (BN) layer, a dropout layer, and a softmax layer; the first fully connected layer, the first batch normalization (BN) layer, the second fully connected layer, the second batch normalization (BN) layer, the dropout layer, the third fully connected layer, and the softmax layer are connected sequentially, as follows: Figure 11 As shown.

[0159] Specifically, for one-dimensional data X = (x1, x2, ... xn) input to a multi-layer fully connected network... n ) T The feature vector S = (s1, s2, ..., s) is obtained by feature mapping through a fully connected layer. m ) T Where n is the number of one-dimensional data points, and m is the number of nodes (feature vector dimension) output by the fully connected layer, as shown in Formula 4:

[0160]

[0161] where g is an activation function, w ij and b ij are weight parameters of the full connection layer corresponding to nodes respectively, and k is the number of input nodes.

[0162] The feature vector S = (s1, s2...s m ) T After normalization by the BN layer, the normalized features V = (v1, v2...v m ) T are obtained, as shown in equation 5.

[0163]

[0164] where β b , γ b , m b , and δ b are training parameters.

[0165] After the full connection layer and the BN layer are stacked, the one-dimensional data input into the multi-layer full connection network is linearly mapped into feature data, and finally the feature data is input into the Dropout layer and the Softmax layer to obtain the probability. The Dropout layer is used to shield the connection mode of the full connection layer according to the Bernoulli distribution probability in the training process of the multi-layer full connection network, as shown in equation 6.

[0166]

[0167] where r ij is a random 0 and 1 generated according to the Bernoulli distribution with a probability p. In actual use, the Dropout probability value p is 0, that is, r ij is always equal to 1.

[0168] Finally, the softmax layer obtains the probability Y = (y1, y2...y t ) T corresponding to each material, as shown in equation 7.

[0169]

[0170] where t is the number of materials.

[0171] It should be further explained that in the present embodiment, the training parameters and the weight parameters of the multi-layer full connection network are obtained by pre-training the multi-layer full connection network. For example, in one embodiment,

[0172] The training parameters and the weight parameters of the multi-layer full connection network are obtained by training the multi-layer full connection network using the cross-entropy loss function and the Adam optimization algorithm. The specific training process is as follows:

[0173] A sufficient number of historical target data sets are obtained as training data, and part of the historical target data sets are extracted as a validation set. The historical target data sets are target data sets obtained in the past. Then, the corresponding material information is labeled for each historical target data in the training set, and a one-hot label is generated:

[0174]

[0175] Where L is the label of the qth material.

[0176] Then, the multi-layer fully connected network is trained using a cross-entropy loss function and an Adam optimization algorithm,

[0177] The cross-entropy loss is shown in equation 8:

[0178]

[0179] The process of training the multi-layer fully connected network using the cross-entropy loss function and the Adam optimization algorithm can refer to the relevant neural network training process. It should be noted that in this embodiment, the training rounds of the multi-layer fully connected network are 100, and the data amount in the training data is 512 each time. After training is completed, the validation set is input into the trained multi-layer fully connected network, and according to the probability output by the multi-layer fully connected network, it is determined whether the trained multi-layer fully connected network meets the requirements. If the recognition rate of the multi-layer fully connected network on the validation set reaches 0.9975 or more, it is determined that the multi-layer fully connected network is a trained multi-layer fully connected network, and the training parameters and weight parameters can be obtained from the multi-layer fully connected network.

[0180] The above training process of the training parameters and the weight parameters can be summarized as follows: collecting a first preset number of target data as training data, and collecting a second preset number of target data as validation data; receiving material labels corresponding to each training data and validation data; training the multi-layer fully connected network using a cross-entropy loss function and an Adam optimization algorithm by rounds, wherein the training data corresponding to a preset data amount is used for training in each round, and an initial network model is obtained in each round; the initial network model is verified by the validation data, and when the verification recognition rate of the initial network model reaches a preset index, the training parameters and weight parameters of the initial network model are confirmed as the training parameters and weight parameters of the multi-layer fully connected network and the training is ended. Based on the training process described in the summary, the training rounds, the preset data amount corresponding to each round, the cross-entropy loss function, and other implementation details described in the foregoing specific training process are exemplary and can be adjusted.

[0181] The above steps are the specific process of obtaining the probabilities corresponding to different materials according to the elastic wave data. The multi-layer fully connected network is used to output the probabilities corresponding to different materials, and compared with the traditional artificial feature method, the recognition accuracy is high and the robustness is good.

[0182] Step 306, adjusting the probability according to the infrared shielding electric signal to obtain the target probability.

[0183] After obtaining the probability, in order to avoid the elastic wave data being disturbed by the vibration to reduce the accuracy of material recognition, in the embodiment, the probability is further adjusted according to the infrared shielding electric signal, so as to obtain the target probability. Specifically, in one embodiment, adjusting the probability according to the infrared shielding electric signal to obtain the target probability, comprising:

[0184] Step 3061, determining the area of the touch object according to the infrared shielding electric signal.

[0185] Firstly, the area of the touch object is determined according to the infrared shielding electric signal. The process of determining the area of the touch object according to the infrared shielding electric signal can refer to the existing literature, which is not described in detail in the embodiment.

[0186] Step 3062, determining the weight corresponding to the area and different materials according to the area of the touch object.

[0187] After determining the area of the touch object, the weight corresponding to the area and different materials is further determined. The weight refers to the importance of a certain factor or index relative to a certain thing. In the embodiment, the area has different numerical ranges, for example, the numerical range 8-12mm 2 represents a touch object, the numerical range 16-22mm 2 represents a touch object, the numerical range 25-35mm 2 represents a touch object, so different areas falling into different area ranges have corresponding weights with different materials. For example, when the area of the touch object is 10mm 2 , the weight corresponding to the chalk is 0.2, the weight of the stylus is 0.6, and the weight of the finger is 0.2. When the area of the touch object is 20mm 2 , the weight corresponding to the chalk is 0.6, the weight of the stylus is 0.2, and the weight of the finger is 0.2. When the area of the touch object is 30mm 2 , the weight corresponding to the chalk is 0.2, the weight of the stylus is 0.2, and the weight of the finger is 0.6. It can be understood that the weights of different materials corresponding to different areas are set in advance.

[0188] Step 3063, adjusting the probability (the aforementioned probability corresponding to different materials) according to the weight to obtain the target probability.

[0189] After the weight between the area of the touch object and different materials is obtained, the probability corresponding to different materials output by the multi-layer full connection network is adjusted according to the weight, so as to obtain the target probability. Specifically, in the embodiment, the probability corresponding to each material is multiplied by the corresponding weight, so as to obtain the target probability of different materials. For example, the probability of different materials output by the multi-layer full connection network is shown in Table 1:

[0190] Table 1

[0191]

[0192] The weight of different materials obtained according to the area of the touch object is shown in Table 2:

[0193] Table 2

[0194]

[0195] At this time, the probability corresponding to each material is multiplied by the corresponding weight. That is, the target probability of chalk is 0.3x0.2=0.06, the target probability of the stylus is 0.2x0.2=0.04, and the target probability of the finger is 0.5x0.6=0.3.

[0196] Step 307, generating the first material information of the touch object according to the target probability.

[0197] Finally, the first material information is generated according to the target probability. Specifically, in each material, the material with the maximum value of the target probability is selected, and the first material information is generated according to the material. For example, when the target probability of the finger is the largest, the first material information is generated according to the finger.

[0198] Step 308, determining that the first material information is obtained within the first time length after the first time, and taking the first material information as the material type of the touch object.

[0199] Step 309, determining that the first material information is not obtained within the first time length after the first time, and obtaining the second material information as the material type of the touch object, wherein the second material information is generated according to the infrared blocking electrical signal.

[0200] In the foregoing, in the preprocessing of the elastic wave data, the elastic wave data is fused by the multi-channel fusion strategy to obtain fused data, so that the case of misrecognition due to invalid elastic wave data caused by abnormalities when determining the material of the touch object using the elastic wave data detected by a single elastic wave sensor can be avoided, and the stability of the target data set can be enhanced to avoid the influence of the difference in the elastic wave data detected by each elastic wave sensor due to different touch positions on subsequent calculation. In addition, the multi-layer fully connected network is used to output the probability corresponding to different materials, and compared with the traditional artificial feature method, the recognition accuracy is high and the robustness is good. Secondly, the probability is adjusted in combination with the infrared blocking electrical signal, which can improve the recognition accuracy of the multi-layer fully connected network and improve the accuracy of identifying the material.

[0201] In one embodiment, further comprising:

[0202] In step 401, if the first material information is not obtained within the first time length after the first time, the infrared blocking electrical signal of the previous time is obtained, and whether the current touch operation is associated with the touch operation at the previous time is determined according to the current infrared blocking electrical signal. If associated, the material type corresponding to the previous time is obtained as the material type of the touch event.

[0203] In the embodiment, when it is determined that the first material information is not obtained within the first time length after the first time, the infrared blocking electrical signal of the previous time is also obtained, and whether the current touch operation is associated with the touch operation at the previous time is determined according to the current infrared blocking electrical signal. Associated means that the touch operation of the touch object on the operation panel at the current time and the touch operation at the previous time are generated by a series of consecutive actions, such as continuous sliding of the user's finger on the touch detection area of the operation panel. Since the touch object pauses for a very short time when performing a series of consecutive actions, the case of replacing the touch object during the touch process can be excluded, that is, the material of the touch object remains unchanged. If the current touch operation is associated with the touch operation at the previous time, the case of changing the material of the touch object can be excluded, and the material type of the previous touch event can be directly used as the material type of the current touch event.

[0204] For example, if the first touch operation, the second touch operation and the third touch operation are a plurality of continuous touch operations generated by a series of consecutive touch actions, after confirming the first material type of the touch object in the first touch operation, the first material type of the touch object can be directly obtained as the second material type of the touch object in the second touch operation, and the second material type can be directly obtained as the third material type of the touch object in the third touch operation. It can be understood that when a series of consecutive touch actions also generate a fourth touch operation, …, an Nth touch operation, the same applies, which is not described in detail in the embodiment.

[0205] Specifically, in one embodiment, when determining whether the current touch operation is associated with the touch operation at the last time, the time interval between the current touch operation and the touch operation at the last time and the distance of the touch point can be used to determine whether the current touch operation is associated with the touch operation at the last time.

[0206] In one embodiment, the infrared blocking electrical signal at the last time is obtained in step 401, and whether the current touch operation is associated with the touch operation at the last time is determined according to the current infrared blocking electrical signal, including:

[0207] In step 4011, the touch point is determined according to the current infrared blocking electrical signal, and the historical touch point is determined according to the infrared blocking electrical signal at the last time.

[0208] The touch point refers to the point at which the touch object contacts the interactive panel. In the embodiment, the current touch point of the touch object is first determined according to the current infrared blocking electrical signal, and the historical touch point of the touch object at the last time is determined according to the infrared blocking electrical signal at the last time.

[0209] In step 4012, the distance between the touch point and the historical touch point is calculated, and the time interval between the first time and the last time is determined.

[0210] After the historical touch point at the last time is determined, the distance between the touch point and the historical touch point is further calculated, and the time interval between the first time and the last time is determined.

[0211] In step 4013, when the distance is less than a preset distance and the time interval is less than a preset time interval, it is determined that the current touch operation is associated with the touch operation at the last time.

[0212] When the distance between the touch point and the historical touch point is less than a preset distance, and the time interval between the first time and the last time is less than a preset time interval, it is determined that the current touch operation and the last touch operation are generated by a series of consecutive touch actions of the touch object, and it is determined that the current touch operation is associated with the touch operation at the last time.

[0213] The above, the embodiment of the application judges whether the current touch operation is associated with the touch operation at the last time, if associated, the material type corresponding to the last time is obtained as the material type of the touch event. Thus, the calculation amount in the process of identifying the material of the touch object can be reduced, the delay of the material identification process is further reduced, the situation of lag caused by calculation delay is avoided, and the user's use experience is improved.

[0214] As Figure 12 shown, Figure 12 a structural schematic diagram of an interactive panel provided by the application, the interactive panel includes an operation panel 11, an infrared touch sensor 12, an elastic wave sensor 15, and at least one processing device 16, the infrared touch sensor 12 is arranged at at least one edge of the operation panel 11 to form a touch detection area of the interactive panel, the elastic wave sensor 15 is used for detecting the vibration of the operation panel 11 and generating an electrical signal, and the at least one processing device 16 is used for:

[0215] When the touch object performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared shielding electrical signal from the first time;

[0216] From the first time, the elastic wave sensor starts to generate an original touch electrical signal;

[0217] The energy average value of the original touch electrical signal in the current time window is calculated;

[0218] The ratio of the energy average value and the historical energy average value is calculated, the historical energy average value is the energy average value of the original touch electrical signal in the last time window;

[0219] It is judged whether the ratio is greater than a preset threshold value;

[0220] If greater, the initial time of the current time window is obtained, and the initial time of the current time window is taken as the starting time of the effective touch electrical signal;

[0221] The effective touch electrical signal in a preset time length after the starting time is obtained as the elastic wave data;

[0222] The first material information of the touch object is determined according to the infrared shielding electrical signal and the elastic wave data.

[0223] In the embodiment, the back of the operation panel is provided with a plurality of elastic wave sensors which can be directly or indirectly mounted on the operation panel. For example, when directly mounted, the elastic wave sensors can be mounted on the back of the operation panel by means of adhesive bonding; when indirectly mounted, the elastic wave sensors can be indirectly mounted on the back of the operation panel by means of mounting components, for example, the elastic wave sensors are mounted on an FPC circuit board, and then the FPC circuit board is mounted on the back of the operation panel to transmit electrical signals through the FPC circuit board. The FPC (Flexible Printed Circuit board) circuit board is generally made of polyimide or polyester film as a base material, and has the advantages of light weight, thin thickness, strong folding resistance, and high wiring density.

[0224] When the touch object performs a touch operation in the touch detection area, the infrared blocking electrical signal and the original touch electrical signal sent by the infrared touch sensor and the elastic wave sensor are respectively received, if the first material information can be generated according to the infrared blocking electrical signal and the original touch electrical signal within the first time length after the first time when the infrared blocking electrical signal is generated, the material type of the touch object is determined according to the first material information, otherwise, the material type of the touch object is determined according to the second material information generated by the infrared blocking electrical signal. The material identification of the touch object is performed by combining the infrared blocking electrical signal and the original touch electrical signal, which can improve the accuracy of material identification, overcome the low material identification rate caused by the lack of material specificity of the infrared blocking electrical signal and the vibration interference of the original touch electrical signal, and solve the technical problem that the interactive device cannot accurately identify the material of the touch object in the prior art. Secondly, in the embodiment, by setting the first time length, the second material information generated according to the infrared blocking electrical signal can be directly used to confirm the material type of the touch object when the first material information cannot be generated in time according to the infrared blocking electrical signal and the original touch electrical signal, thereby avoiding the technical problem that the user's experience is poor due to too long calculation delay.

[0225] On the basis of the above-mentioned embodiment, the method further comprises:

[0226] The preset threshold value is dynamically updated according to the first electrical signal generated by the infrared touch sensor and the second electrical signal generated by the elastic wave sensor.

[0227] On the basis of the above-mentioned embodiment, the preset threshold value is dynamically updated according to the first electrical signal generated by the infrared touch sensor and the second electrical signal generated by the elastic wave sensor, comprising:

[0228] When it is determined that there is no touch object according to the first electrical signal generated by the infrared touch sensor, it is judged in real time whether the instantaneous amplitude of the second electrical signal generated by the elastic wave sensor is greater than the preset threshold value;

[0229] when the instantaneous amplitude of the second electrical signal is greater than the preset threshold, setting the preset threshold as the first preset threshold;

[0230] when the instantaneous amplitude of the second electrical signal is less than the preset threshold, setting the preset threshold as the second preset threshold.

[0231] On the basis of the above-mentioned embodiments, the first material information of the touch object is determined according to the infrared shielding electrical signal and the elastic wave data, comprising:

[0232] generating a probability corresponding to different materials according to the elastic wave data;

[0233] adjusting the probability according to the infrared shielding electrical signal to obtain a target probability;

[0234] generating the first material information of the touch object according to the target probability.

[0235] On the basis of the above-mentioned embodiments, the target probability is obtained by adjusting the probability according to the infrared shielding electrical signal, comprising:

[0236] calculating the area of the touch object according to the infrared shielding electrical signal;

[0237] determining a weight corresponding to the area and different materials according to the area of the touch object;

[0238] adjusting the probability according to the weight to obtain the target probability.

[0239] On the basis of the above-mentioned embodiments, the probability corresponding to different materials is generated according to the elastic wave data, comprising:

[0240] preprocessing the elastic wave data to obtain target data;

[0241] inputting the target data into a preset neural network to obtain the probability corresponding to different materials.

[0242] On the basis of the above-mentioned embodiments, the elastic wave data is obtained according to the effective touch electrical signal corresponding to a plurality of elastic wave sensors, the target data is obtained by preprocessing the elastic wave data, comprising:

[0243] performing time-frequency transformation on the elastic wave data corresponding to a plurality of elastic wave sensors to obtain frequency domain data corresponding to the elastic wave data;

[0244] performing multi-channel signal fusion on the frequency domain data to obtain fusion data;

[0245] filtering the fusion data to obtain the target data.

[0246] On the basis of the above-mentioned embodiments, the fusion data is obtained by performing multi-channel signal fusion on the frequency domain data, comprising:

[0247] The frequency domain data is weighted and averaged to obtain fused data.

[0248] On the basis of the above embodiment, the frequency domain data is subjected to multi-channel signal fusion to obtain fused data, comprising:

[0249] The frequency domain data is input into a preset first neural network to obtain fused data.

[0250] On the basis of the above embodiment, the elastic wave data is preprocessed to obtain target data, comprising:

[0251] The system function of the elastic wave propagating to each elastic wave sensor is calculated;

[0252] The elastic wave data corresponding to each elastic wave sensor is divided by the corresponding system function to obtain first data corresponding to each elastic wave sensor;

[0253] The average of all first data is calculated to obtain target data.

[0254] On the basis of the above embodiment, the neural network is a multi-layer fully connected network, comprising a first fully connected layer, a second fully connected layer, a third fully connected layer, a first BN layer, a second BN layer, a Dropout layer and a Softmax layer; the first fully connected layer, the first BN layer, the second fully connected layer, the second BN layer, the Dropout layer, the third fully connected layer and the Softmax layer are connected in sequence.

[0255] On the basis of the above embodiment, the training parameters and weight parameters of the multi-layer fully connected network are obtained by training the multi-layer fully connected network using a cross-entropy loss function and an Adam optimization algorithm.

[0256] On the basis of the above embodiment, the training parameters and weight parameters are obtained by training in the following manner:

[0257] A first preset number of target data is collected as training data, and a second preset number of target data is collected as validation data;

[0258] The material label corresponding to each training data and validation data is received;

[0259] The multi-layer fully connected network is trained using a cross-entropy loss function and an Adam optimization algorithm by rounds, wherein each round uses training data corresponding to a preset data amount for training, and each round of training obtains a corresponding initial network model;

[0260] The initial network model is verified by verification data, and when the verification identification rate of the initial network model reaches a preset index, the training parameters and weight parameters of the initial network model are confirmed as the training parameters and weight parameters of the multi-layer full connection network, and the training is ended.

[0261] On the basis of the above-mentioned embodiments, further comprising:

[0262] If the first material information is not obtained within the first time duration after the first time, the infrared shielding electrical signal of the previous time is acquired, and whether the current touch operation is associated with the touch operation of the previous time is determined according to the current infrared shielding electrical signal, if associated, the material type corresponding to the previous time is acquired as the material type of the touch event.

[0263] On the basis of the above-mentioned embodiments, the infrared shielding electrical signal of the previous time is acquired, and whether the current touch operation is associated with the touch operation of the previous time is determined according to the current infrared shielding electrical signal, comprising:

[0264] The touch point is determined according to the current infrared shielding electrical signal, and the historical touch point is determined according to the infrared shielding electrical signal of the previous time;

[0265] The distance between the touch point and the historical touch point is calculated, and the time interval between the first time and the previous time is determined;

[0266] When the distance is less than a preset distance and the time interval is less than a preset time interval, it is determined that the current touch operation is associated with the touch operation of the previous time.

[0267] The interactive tablet embodiment of the present application is used to perform the related operations in the material identification method provided in any embodiment of the present application, and has the corresponding functions and beneficial effects.

[0268] In addition, the embodiment of the present application further provides a storage medium containing computer executable instructions, which are used to perform the related operations in the material identification method provided in any embodiment of the present application when executed by a computer processor, and have the corresponding functions and beneficial effects.

[0269] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product.

[0270] Accordingly, embodiments of the present application can be embodied in the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product on one or more computer-readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code. Embodiments of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0271] In one typical arrangement, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. The memory can include non-persistent memory in the form of random access memory (RAM), and / or non-volatile memory such as read only memory (ROM) or flash memory, among others, in a computer readable medium. The memory is an example of a computer readable medium.

[0272] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0273] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0274] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only for specific embodiments of the present application and is not intended to limit the protection scope of the present application. It is particularly pointed out that any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A material identification method for an interactive whiteboard, comprising: The interactive panel comprises an operation panel, an infrared touch sensor and an elastic wave sensor, the infrared touch sensor is arranged at at least one edge of the operation panel to form a touch detection area of the interactive panel, and the elastic wave sensor is used to detect vibration of the operation panel and generate an electric signal, and the material identification method comprises: When a touch object performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared shielding electric signal from a first time point; The elastic wave sensor starts to generate an original touch electric signal from the first time point; An energy average value of the original touch electric signal in a current time window is calculated; A ratio of the energy average value and a historical energy average value is calculated, the historical energy average value being an energy average value of the original touch electric signal in a previous time window; It is judged whether the ratio is greater than a preset threshold value; If yes, an initial time point of the current time window is obtained, and the initial time point of the current time window is taken as a starting time point of a valid touch electric signal; A valid touch electric signal in a preset time length after the starting time point is obtained as elastic wave data; First material information of the touch object is determined according to the infrared shielding electric signal and the elastic wave data; The preset threshold value is dynamically updated according to a first electric signal generated by the infrared touch sensor and a second electric signal generated by the elastic wave sensor.

2. The material identification method of claim 1, wherein The preset threshold value is dynamically updated according to a first electric signal generated by the infrared touch sensor and a second electric signal generated by the elastic wave sensor, comprising: When it is determined according to the first electric signal generated by the infrared touch sensor that the touch object does not exist, it is judged in real time whether an instantaneous amplitude of the second electric signal generated by the elastic wave sensor is greater than a preset threshold value; When the instantaneous amplitude of the second electric signal is greater than the preset threshold value, the preset threshold value is set as a first preset threshold value; When the instantaneous amplitude of the second electric signal is less than the preset threshold value, the preset threshold value is set as a second preset threshold value.

3. The material identification method of claim 1, wherein The first material information of the touch object is determined according to the infrared shielding electric signal and the elastic wave data, comprising: Probabilities corresponding to different materials are generated according to the elastic wave data; A target probability is obtained by adjusting the probabilities according to the infrared shielding electric signal; The first material information of the touch object is generated according to the target probability.

4. The material identification method of claim 3, wherein The target probability is obtained by adjusting the probabilities according to the infrared shielding electric signal, comprising: The area of the touch object is calculated according to the infrared shielding electric signal; Weights corresponding to the area and the different materials are determined according to the area of the touch object; The target probability is obtained by adjusting the probabilities according to the weights.

5. The material identification method of claim 3, wherein The probabilities corresponding to the different materials are generated according to the elastic wave data, comprising: Target data is obtained by preprocessing the elastic wave data; The target data is input into a preset neural network to obtain the probabilities corresponding to the different materials.

6. The material identification method of claim 5, wherein The elastic wave data is generated according to effective touch electric signals corresponding to the plurality of elastic wave sensors, the elastic wave data is preprocessed to obtain target data, and the preprocessing includes: Time-frequency transformation is performed on the elastic wave data corresponding to the plurality of elastic wave sensors to obtain frequency domain data corresponding to the elastic wave data; Multi-channel signal fusion is performed on the frequency domain data to obtain fused data; Filtering is performed on the fused data to obtain target data.

7. The material identification method of claim 6, wherein The multi-channel signal fusion on the frequency domain data to obtain fused data includes: Weighted average is performed on the frequency domain data to obtain fused data.

8. The material identification method of claim 6, wherein, The multi-channel signal fusion on the frequency domain data to obtain fused data includes: The frequency domain data is input into a preset first neural network to obtain fused data.

9. The material identification method of claim 5, wherein, The preprocessing of the elastic wave data to obtain target data includes: System functions of the elastic wave propagating to each of the plurality of elastic wave sensors are calculated; The elastic wave data corresponding to each of the plurality of elastic wave sensors is divided by the corresponding system function to obtain first data corresponding to each of the plurality of elastic wave sensors; An average value of all the first data is calculated to obtain target data.

10. The material identification method of claim 5, wherein The neural network is a multi-layer fully connected network, including a first fully connected layer, a second fully connected layer, a third fully connected layer, a first BN layer, a second BN layer, a Dropout layer, and a Softmax layer; the first fully connected layer, the first BN layer, the second fully connected layer, the second BN layer, the Dropout layer, the third fully connected layer, and the Softmax layer are connected in sequence.

11. The material identification method of claim 10, wherein The training parameters and weight parameters of the multi-layer fully connected network are obtained by training the multi-layer fully connected network using a cross-entropy loss function and an Adam optimization algorithm.

12. The material identification method of claim 11, wherein The training parameters and weight parameters are obtained by: Collecting a first preset number of target data as training data and a second preset number of target data as validation data; Receiving material labels corresponding to each of the training data and the validation data; Training the multi-layer fully connected network using a cross-entropy loss function and an Adam optimization algorithm by rounds, wherein the training data corresponding to a preset data amount is used in each round, and an initial network model is obtained in each round of training; The initial network model is verified by the validation data, and when the verification recognition rate of the initial network model reaches a preset index, the training parameters and weight parameters of the initial network model are confirmed as the training parameters and weight parameters of the multi-layer fully connected network and the training is ended.

13. The material identification method of claim 1, wherein Further comprising: If the first material information is not obtained within a first time period after the first time, an infrared shielding electric signal at the previous time is obtained, and it is determined whether the current touch operation is associated with the touch operation at the previous time according to the current infrared shielding electric signal, if associated, the material type corresponding to the previous time is obtained as the material type of the touch event.

14. The material identification method of claim 13, wherein, The infrared blocking electrical signal of the previous time is acquired, and whether the current touch operation is associated with the touch operation of the previous time is determined according to the current infrared blocking electrical signal, including: A touch point is determined according to the current infrared blocking electrical signal, and a historical touch point is determined according to the infrared blocking electrical signal of the previous time; A distance between the touch point and the historical touch point is calculated, and a time interval between the first time and the previous time is determined; When the distance is less than a preset distance and the time interval is less than a preset time interval, it is determined that the current touch operation is associated with the touch operation of the previous time.

15. An interactive whiteboard, comprising: The interactive panel includes an operation panel, an infrared touch sensor, an elastic wave sensor, and at least one processing device, the infrared touch sensor is arranged at at least one edge of the operation panel to form a touch detection area of the interactive panel, the elastic wave sensor is used to detect the vibration of the operation panel and generate an electrical signal, and the at least one processing device is used to: When a touch object performs a touch operation in the touch detection area, the infrared touch sensor generates an infrared blocking electrical signal from a first time; From the first time, the elastic wave sensor starts to generate an original touch electrical signal; An energy average value of the original touch electrical signal in a current time window is calculated; A ratio of the energy average value and a historical energy average value is calculated, the historical energy average value being an energy average value of the original touch electrical signal in a previous time window; It is determined whether the ratio is greater than a preset threshold value; If so, an initial time of the current time window is obtained, and the initial time of the current time window is taken as a starting time of valid touch electrical signal; After the starting time, valid touch electrical signal in a preset time length is obtained as elastic wave data; According to the infrared blocking electrical signal and the elastic wave data, first material information of the touch object is determined; According to a first electrical signal generated by the infrared touch sensor and a second electrical signal generated by the elastic wave sensor, the preset threshold value is dynamically updated.

16. An interactive whiteboard according to claim 15, wherein, The number of the elastic wave sensors is multiple, and the elastic wave sensors are directly or indirectly installed on the back of the operation panel.

17. A storage medium storing computer-executable instructions, wherein: The computer executable instructions, when executed by a computer processor, are used to perform a material identification method as claimed in any one of claims 1-14.

Images