Touch recognition method and device, intelligent glasses, medium and program product

Abstract

The invention discloses a touch recognition method and device, intelligent glasses, a medium and a program product. Comprising the following steps: acquiring a touch vibration signal acquired by a vibration sensor in a time window of a touch event, and extracting a vibration feature vector from the touch vibration signal; inputting the vibration feature vector into a pre-trained touch event recognition model, and determining an effective touch confidence coefficient according to a model output result of the touch event recognition model; and determining a touch identification result of the touch event according to the effective touch confidence coefficient. According to the intelligent glasses, the touch event is subjected to supplementary detection through other sensors except the touch sensor, so that misjudgment caused when a single sensor is used for touch detection is avoided, and on the basis of not increasing the construction complexity of the intelligent glasses equipment, on the basis of function reuse of the existing sensors in the intelligent glasses, the detection accuracy is improved. The recognition accuracy of the intelligent glasses for the touch intention is improved, and the user touch experience is better improved under the condition that the light weight level of the intelligent glasses is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of smart wearable device technology, and in particular to a touch recognition method, device, smart glasses, medium, and program product. Background Technology

[0002] With the continuous development of smart wearable technology, products such as smart glasses are increasingly being given diverse functions and are widely used in various scenarios. Currently, smart glasses often interact with users through capacitive touch sensors, with interaction methods including touch, swipe, tap, and wear detection. However, based on the detection principle of capacitive touch sensors, accidental touches or insensitive recognition often occur during user adjustments and use of smart glasses. For example, unintentional touches may occur when the user adjusts their glasses or moves their head, and touches made with the tip of a fingernail instead of the fingertip may result in the touch intent not being recognized. In other words, current touch detection using only a single type of sensor is prone to misjudgment, while adding more sensors would make the device structure more complex, causing considerable inconvenience for users. Summary of the Invention

[0003] This invention provides a touch recognition method, device, smart glasses, medium, and program product. Based on existing sensors on the smart glasses, it assists in verifying touch intentions during the use of the smart glasses. Without increasing the structural complexity of the smart glasses, it improves the accuracy of touch intention recognition and enhances the practicality of the smart glasses.

[0004] In a first aspect, embodiments of the present invention provide a touch recognition method applied to smart glasses, wherein the smart glasses include at least a vibration sensor for picking up voice information, and the method includes: The system acquires touch vibration signals collected by the vibration sensor within the time window of the touch event, and extracts vibration feature vectors from the touch vibration signals. The vibration feature vector is input into the pre-trained touch event recognition model, and the effective touch confidence is determined based on the model output of the touch event recognition model. The touch recognition result of the touch event is determined based on the effective touch confidence level.

[0005] Secondly, embodiments of the present invention also provide a touch recognition device for use in smart glasses, wherein the smart glasses include at least a vibration sensor for picking up voice information, and the touch recognition device includes: The vibration vector extraction module is used to acquire the touch vibration signal collected by the vibration sensor within the time window of the touch event, and extract the vibration feature vector from the touch vibration signal; The effective confidence determination module is used to input the vibration feature vector into the pre-trained touch event recognition model and determine the effective touch confidence based on the model output of the touch event recognition model. The recognition result determination module is used to determine the touch recognition result of the touch event based on the effective touch confidence level.

[0006] Thirdly, embodiments of the present invention also provide smart glasses, comprising: Vibration sensors used to pick up voice information; At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor to enable at least one processor to implement the touch recognition method of any embodiment of the present invention.

[0007] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the touch recognition method of any embodiment of the present invention.

[0008] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program, which, when executed by a processor, is used to perform the touch recognition method of any embodiment of the present invention.

[0009] This invention provides a touch recognition method, device, smart glasses, medium, and program product, based on smart glasses including at least a vibration sensor for picking up voice information. The method involves acquiring touch vibration signals collected by the vibration sensor within a time window of a touch event, extracting vibration feature vectors from these signals, inputting the vibration feature vectors into a pre-trained touch event recognition model, determining the effective touch confidence level based on the model's output, and determining the touch recognition result based on the effective touch confidence level. By employing this technical solution, the vibration sensor, already installed on the smart glasses for picking up voice information, collects touch vibration signals within a time window of the touch event detected during smart glasses use. Analyzing the collected touch vibration signals helps determine, from a vibration audio perspective, whether audio information generated by the touch action actually exists within the time window of the touch event, thus enabling auxiliary verification of touch intent during smart glasses use based on existing sensors on the smart glasses. By using additional sensors besides the touch sensor to supplement touch event detection, misjudgments that occur when a single sensor is used for touch detection are avoided. Without increasing the structural complexity of the smart glasses device, the accuracy of the smart glasses in recognizing touch intentions is improved by reusing the functions of existing sensors in the smart glasses. This enhances the user's touch experience while maintaining the lightweight design of the smart glasses.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a touch recognition method provided in Embodiment 1 of the present invention; Figure 2 This is an example diagram showing the placement of a vibration sensor on smart glasses according to Embodiment 1 of the present invention; Figure 3 The following is an example waveform diagram of the touch vibration signal corresponding to each touch event type provided in Embodiment 1 of the present invention; Figure 4 This is a flowchart of a touch recognition method provided in Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of the structure of a touch recognition device provided in Embodiment 3 of the present invention; Figure 6 This is a schematic diagram of the structure of a smart glasses provided in Embodiment 4 of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] Example 1 Figure 1 This is a flowchart illustrating a touch recognition method according to Embodiment 1 of the present invention. This embodiment is applicable to identifying whether touch events generated during the use of smart glasses are accidental touches. The method can be executed by a touch recognition device, which can be implemented by software and / or hardware, and can be configured within the smart glasses. It should be noted that the smart glasses include at least a vibration sensor for picking up voice information.

[0016] In this embodiment, smart glasses can be specifically understood as smart wearable devices that take the form of glasses and integrate microprocessors, sensors, communication modules, display and audio components, etc., and have independent or collaborative computing capabilities. For example, smart glasses may include artificial intelligence (AI) glasses and augmented reality (AR) glasses, etc., and this embodiment of the invention does not impose any limitations on them.

[0017] In this embodiment, the vibration sensor on the smart glasses can be specifically understood as a sensor disposed on the smart glasses body, used to pick up audio information or user-related voice information in the environment in which the smart glasses are located through vibration. For example, the vibration sensor on the smart glasses can be a bone conduction sensor, or a bone conduction voice pick-up sensor (VPU sensor), or other vibration sensors capable of picking up voice information; this embodiment of the invention does not limit this.

[0018] In some examples, vibration sensors may be mounted on the nose pads or frame of the smart glasses. Figure 2 This is an example diagram illustrating the placement of a vibration sensor on smart glasses according to Embodiment 1 of the present invention. Figure 2 As indicated by the middle arrow, the vibration sensor can be placed on the nose pads or frame of the glasses, or at other locations on the glasses that have direct or indirect contact with the wearer. This embodiment of the invention does not impose any limitations on this. It is understood that in smart glasses, the proximity sensor or capacitive touch sensor used for touch detection is often located on the outside of the temples. To avoid accidental touches or conflicting settings, the vibration sensor and capacitive touch sensor are often placed in different locations on the smart glasses.

[0019] like Figure 1 As shown in the figure, a touch recognition method provided by an embodiment of the present invention specifically includes the following steps: S101. Obtain the touch vibration signal collected by the vibration sensor within the time window of the touch event, and extract the vibration feature vector from the touch vibration signal.

[0020] In this embodiment, a touch event can be specifically understood as an event in which, during the use of smart glasses, a capacitive touch sensor located on the temple or other position of the smart glasses determines, based on changes in capacitance, that the user's finger touches or approaches the sensor and thus the user has a need to control the smart glasses.

[0021] In this embodiment, the time window of a touch event can be specifically understood as a continuous period of time before and after the moment when the smart glasses detect the touch event through the capacitive touch sensor.

[0022] In this embodiment, the touch vibration signal can be specifically understood as a vibration signal generated by a touch event, separated from the vibration signal collected by the vibration sensor. It is understood that when the vibration sensor picks up vibration signals, the vibration signal received by the smart glasses from the touch operation differs fundamentally in physical nature and mathematical characteristics from the vibration signal generated when the user provides voice information. For example, the vibration signal corresponding to the touch operation is a broadband, non-harmonic, and transient mechanical impact signal; while the vibration signal corresponding to the voice information is a narrowband, harmonic, and quasi-periodic biological vibration signal. Therefore, the vibration signals collected by the vibration sensor can be distinguished based on signal characteristics to obtain a touch vibration signal that indicates information related to the touch event.

[0023] In this embodiment, the vibration feature vector can be specifically understood as a feature vector used to indicate the waveform characteristics of the touch vibration signal in the time domain and frequency domain.

[0024] Specifically, based on the touch functionality of smart glasses, the touch detection module will function normally during normal operation. When this module detects a user's touch operation, a touch event is considered to have occurred. For example, for capacitive touch sensors, a touch event is considered to have occurred when the capacitance change exceeds a preset threshold. The vibration sensor in the smart glasses, used for voice information pickup, can pick up voice information in real time during use, meaning it is always capable of picking up vibration signals. Therefore, when a touch event occurs, the vibration signal collected by the vibration sensor within the time window of the touch event can be acquired. The touch vibration signal can then be extracted from the acquired vibration signal based on the relevant characteristics of the vibration generated by the touch operation. Furthermore, the obtained touch vibration signal is analyzed from both the time and frequency domains to extract features related to the vibration characteristics. These extracted features are then combined to determine the vibration feature vector.

[0025] S102. Input the vibration feature vector into the pre-trained touch event recognition model, and determine the effective touch confidence based on the model output of the touch event recognition model.

[0026] In this embodiment, the touch event recognition model can be specifically understood as a classification neural network model that determines the touch event type to which the feature vector belongs, and the confidence level of belonging to that touch event type, based on the vibration-related feature vector input therein. Optionally, the touch event types that can be recognized in the touch event recognition model can be determined during training according to the actual situation, such as including single-click, double-click, long-press, swipe, and non-touch types, etc., and this embodiment of the invention does not impose any limitations on this.

[0027] In this embodiment, the effective touch confidence can be specifically understood as the confidence of the touch event type to which the vibration feature vector input therein most likely belongs, as determined by the touch event recognition model.

[0028] Specifically, the vibration feature vector is input into a pre-trained touch event recognition model. Based on the touch event recognition model, the vibration feature vector is extracted and segmented to obtain an output result containing the confidence level of the vibration feature vector belonging to each touch event type. Then, based on the obtained output result, it can be determined whether the vibration signal corresponding to the vibration feature vector is the vibration generated by the touch event, what type of touch event it belongs to, and the confidence level of the touch event type. Based on the above information, the confidence level of the touch event type to which the vibration feature vector most likely belongs is determined as the effective touch confidence level of the vibration feature vector.

[0029] Understandably, the touch event recognition model needs to be trained before smart glasses are put into use. The training method for this touch event recognition model can be as follows: 1) For each touch event type, obtain multiple touch vibration signals corresponding to that touch event type, and use the touch time type as the label of the corresponding touch vibration signal. Determine each set of touch vibration signals and the corresponding touch event type as a training sample to construct a training sample set.

[0030] 2) Input the training sample set into the pre-built initial touch event recognition model, compare the intermediate results output by the initial touch event recognition model with the labels in each training sample, and construct a loss function based on the comparison results.

[0031] 3) Train the initial touch event recognition model based on the determined loss function until the preset convergence condition is met to obtain the trained touch event recognition model.

[0032] It is understandable that the touch vibration signals corresponding to different touch event types have their own parameter characteristics. The following table shows the parameter characteristics corresponding to different touch event types. When constructing the training sample set, the selection and labeling of touch vibration signals are completed according to the following table. For example, to more clearly define the waveform characteristics of touch vibration signals corresponding to different touch event types, Figure 3 Here are example waveform diagrams of the touch vibration signals corresponding to each touch event type provided in Embodiment 1 of the present invention, such as... Figure 3 As shown, the waveforms corresponding to the touch vibration signals for single click, double click, long press, and swipe, respectively, are shown from top to bottom.

[0033] S103. Determine the touch recognition result of the touch event based on the effective touch confidence level.

[0034] In this embodiment, the touch recognition result can be specifically understood as the recognition result used to indicate whether a touch event is an event generated by a valid touch operation by the user.

[0035] Specifically, based on the effective touch confidence level, the reliability of the touch vibration signal collected within the time window can be determined as to whether it is indeed a vibration generated by the touch action given by the user's touch adjustment needs. When it exceeds a certain threshold, the touch event that generates the touch vibration signal is more likely to be an intentional operation by the user. When it does not exceed a certain threshold, the touch event that generates the touch vibration signal is more likely to be a mis-touch by the user. Based on this, it can be determined whether the touch event is a touch event caused by a mis-touch by the user, and the corresponding touch recognition result can be obtained.

[0036] The technical solution of this embodiment is based on smart glasses that include at least a vibration sensor for picking up voice information. It acquires touch vibration signals collected by the vibration sensor within a time window of a touch event, and extracts vibration feature vectors from these signals. The vibration feature vectors are then input into a pre-trained touch event recognition model. The effective touch confidence level is determined based on the model's output. Finally, the touch event recognition result is determined based on the effective touch confidence level. By employing this technical solution, the vibration sensor, already installed on the smart glasses for picking up voice information, collects touch vibration signals within a time window of the touch event detected during smart glasses use. Analyzing the collected touch vibration signals helps determine, from a vibration audio perspective, whether audio information generated by the touch action actually exists within the time window of the touch event. This achieves auxiliary verification of touch intent during smart glasses use based on existing sensors on the smart glasses. By using additional sensors besides the touch sensor to supplement touch event detection, misjudgments that occur when a single sensor is used for touch detection are avoided. Without increasing the structural complexity of the smart glasses device, the accuracy of the smart glasses in recognizing touch intentions is improved by reusing the functions of existing sensors in the smart glasses. This enhances the user's touch experience while maintaining the lightweight design of the smart glasses.

[0037] Example 2 Figure 4This is a flowchart of a touch recognition method provided in Embodiment 2 of the present invention. The technical solution of this embodiment further optimizes the above-mentioned optional technical solutions. First, a vibration sensor collects the complete audio signal within the time window of a touch event. Then, based on the touch vibration characteristics of touch events generated during smart glasses operation, the collected audio signal is separated to obtain a touch vibration signal that can be used to assist in touch event judgment. Next, waveform analysis is performed on the separated touch vibration signal. Feature extraction of the touch vibration signal is completed from aspects such as the duration of single peaks, peak intervals, and duration in the time domain, and frequency amplitude, frequency range, amplitude fluctuation, dominant frequency, high-frequency component proportion, and wideband distribution information in the frequency domain. The extracted features are combined to obtain the vibration feature vector of the touch vibration signal. After obtaining the model recognition results of the touch event recognition model, the confidence level of the touch vibration signal for different touch event types can be determined based on the obtained model recognition results. Since the touch event type with the highest confidence level can be considered as the touch event type to which the touch vibration signal most likely belongs, the maximum value among each confidence level can be selected as the effective touch confidence level for determining the touch event recognition result. By comparing the effective touch confidence level with the preset confidence threshold, it can be determined whether the touch event is a false touch based on the comparison result. Without increasing the structural complexity of the smart glasses device, the accuracy of the smart glasses in recognizing touch intentions is improved by reusing the functions of existing sensors in the smart glasses. This can then be used as a basis to determine whether the smart glasses need to respond to touch, thus better improving the user touch experience while ensuring the lightweight level of the smart glasses.

[0038] like Figure 4 As shown in the figure, a touch recognition method provided by an embodiment of the present invention specifically includes the following steps: S201. Obtain the audio signal collected by the vibration sensor within the time window of the touch event.

[0039] Specifically, the vibration signal within the time window of the touch event, which is directly obtained by the vibration sensor, is identified as the audio signal collected by the vibration sensor.

[0040] S202. Separate the audio signal based on the touch vibration characteristics to determine the touch vibration signal.

[0041] In some examples, the extracted audio signal can be separated to obtain the touch vibration signal based on the touch vibration characteristics corresponding to various touch events proposed in the core time-domain features, frequency-domain features, and parameter thresholds of various features given in the table above.

[0042] Optionally, the touch vibration characteristics that the touch vibration signal can have can be determined in advance based on theory. For example, the broadband, non-harmonic and transient characteristics mentioned above can be used as touch vibration characteristics. Based on this, the audio signal collected by the vibration sensor can be separated to obtain the touch vibration signal.

[0043] It is understandable that audio signals may also include environmental noise and vibrations caused by interference from other operations. When performing audio signal separation, the audio signal can be separated and extracted by using signal separation algorithms such as filtering and time-frequency domain segmentation to achieve the separation of touch vibration signals with specific frequency peaks, time-domain waveforms and pulse characteristics.

[0044] S203. Perform waveform analysis on the touch vibration signal to determine the time-domain and frequency-domain characteristic information in the touch vibration signal.

[0045] The time-domain feature information includes at least one of the following: single peak duration; peak interval; duration; The frequency domain characteristic information includes at least one of the following: frequency amplitude; frequency range; amplitude fluctuation; dominant frequency; proportion of high-frequency components; and broadband distribution information.

[0046] S204. Combine the time-domain feature information with the frequency-domain feature information to determine the vibration feature vector of the touch vibration signal.

[0047] Specifically, the time-domain feature information and the frequency-domain feature information are spliced ​​and fused to make their form meet the input requirements of the pre-trained touch event recognition model, and determined as the vibration feature vector of the touch vibration signal.

[0048] S205. Input the vibration feature vector into the pre-trained touch event recognition model, and determine the confidence level of the touch event belonging to each touch event type based on the model output of the touch event recognition model.

[0049] Specifically, the vibration feature vector is input into a pre-trained touch event recognition model, the vibration feature vector is classified based on the touch event recognition model, and the confidence level of the vibration feature vector belonging to different touch event types is output.

[0050] S206. The maximum value among the confidence levels is determined as the effective touch confidence level.

[0051] S207. Determine whether the confidence level of the effective touch is greater than the preset confidence threshold. If yes, proceed to S208; otherwise, proceed to S209.

[0052] In this embodiment, the preset confidence threshold can be understood as a threshold set in advance based on actual conditions, used to determine whether the confidence level meets the effective touch judgment threshold. For example, the preset confidence threshold can be 0.8, and this embodiment of the invention does not limit this.

[0053] Specifically, by comparing the effective touch confidence with a preset confidence threshold, if the effective touch confidence is greater than the preset confidence threshold, the touch event corresponding to the vibration feature vector can be considered to be a touch event of the type of touch event corresponding to the effective touch confidence, and S208 is executed; otherwise, the touch event corresponding to the vibration feature vector can be considered not to be a touch intentionally performed by the user, and S209 is executed.

[0054] S208. Determine the touch recognition result of the touch event as a valid touch.

[0055] S209. Determine the touch recognition result of the touch event as a false touch.

[0056] The technical solution of this embodiment first acquires the complete audio signal within the time window of a touch event using a vibration sensor. Then, based on the touch vibration characteristics of touch events generated during smart glasses operation, it performs separation processing on the acquired audio signal to obtain a touch vibration signal that can be used to assist in touch event judgment. Furthermore, it performs waveform analysis on the separated touch vibration signal, extracting features from aspects such as single peak duration, peak interval, and duration in the time domain, and frequency amplitude, frequency range, amplitude fluctuation, dominant frequency, high-frequency component proportion, and broadband distribution information in the frequency domain. The extracted features are then combined to obtain the vibration feature vector of the touch vibration signal. After obtaining the model recognition results of the touch event recognition model, the confidence level of the touch vibration signal for different touch event types can be determined based on the obtained model recognition results. Since the touch event type with the highest confidence level can be considered as the touch event type to which the touch vibration signal most likely belongs, the maximum value among each confidence level can be selected as the effective touch confidence level for determining the touch event recognition result. By comparing the effective touch confidence level with the preset confidence threshold, it can be determined whether the touch event is a false touch based on the comparison result. Without increasing the structural complexity of the smart glasses device, the accuracy of the smart glasses in recognizing touch intentions is improved by reusing the functions of existing sensors in the smart glasses. This can then be used as a basis to determine whether the smart glasses need to respond to touch, thus better improving the user touch experience while ensuring the lightweight level of the smart glasses.

[0057] Example 3 Figure 5This is a schematic diagram of a touch recognition device according to Embodiment 3 of the present invention. This touch recognition device is applied to smart glasses, which at least include a vibration sensor for picking up voice information. Figure 5 As shown, the touch recognition device includes a vibration vector extraction module 31, an effective confidence level determination module 32, and a recognition result determination module 33.

[0058] The vibration vector extraction module 31 is used to acquire the touch vibration signal collected by the vibration sensor within the time window of the touch event, and extract the vibration feature vector from the touch vibration signal; the effective confidence determination module 32 is used to input the vibration feature vector into the pre-trained touch event recognition model, and determine the effective touch confidence based on the model output of the touch event recognition model; the recognition result determination module 33 is used to determine the touch recognition result of the touch event based on the effective touch confidence.

[0059] The technical solution of this invention, based on the vibration sensor already installed on smart glasses for picking up voice information, collects touch vibration signals within the time window of a touch event detected during smart glasses use. By analyzing the collected touch vibration signals, it helps determine from a vibration audio perspective whether audio information generated by the touch action actually exists within the time window of the touch event. This achieves auxiliary verification of touch intentions during smart glasses use based on existing sensors on the smart glasses. By supplementing touch event detection with other sensors besides the touch sensor, it avoids misjudgments that occur when a single sensor is used for touch detection. Without increasing the structural complexity of the smart glasses device, it improves the accuracy of touch intention recognition by reusing the functions of existing sensors in the smart glasses, and better enhances the user touch experience while maintaining the lightweight design of the smart glasses.

[0060] Optionally, the vibration vector extraction module 31 is specifically used for: Acquire the audio signal collected by the vibration sensor within the time window of the touch event; The audio signal is separated based on the characteristics of touch vibration to determine the touch vibration signal; Waveform analysis is performed on the touch vibration signal to determine the time-domain and frequency-domain features of the touch vibration signal; By combining time-domain and frequency-domain feature information, the vibration feature vector of the touch vibration signal is determined. The time-domain feature information includes at least one of the following: single peak duration; peak interval; duration; The frequency domain characteristic information includes at least one of the following: frequency amplitude; frequency range; amplitude fluctuation; dominant frequency; proportion of high-frequency components; and broadband distribution information.

[0061] Optional, the effective confidence level determination module 32 is specifically used for: The vibration feature vector is input into the pre-trained touch event recognition model. Based on the model output, the confidence level of the touch event belongs to each touch event type is determined. The maximum value among the confidence levels is determined as the effective touch confidence level.

[0062] Optionally, the recognition result determination module 33 is specifically used for: If the confidence level of a valid touch is greater than a preset confidence threshold, the touch recognition result of the touch event is determined as a valid touch; otherwise, the touch recognition result of the touch event is determined as a false touch.

[0063] Optionally, the vibration sensor can be placed on the nose pads or frame of the smart glasses.

[0064] The touch recognition device provided in the embodiments of the present invention can execute the touch recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0065] Example 4 Figure 6 This is a schematic diagram of the structure of smart glasses according to Embodiment 4 of the present invention. Smart glasses can also refer to various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0066] like Figure 6 As shown, the smart glasses include a vibration sensor 40 for picking up voice information, at least one processor 41, and a memory, such as a read-only memory (ROM) 42 or a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 can also store various programs and data required for the operation of the smart glasses. The vibration sensor 40, processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0067] Multiple components in the smart glasses are connected to I / O interface 45, including: input unit 46, such as a keyboard, mouse, etc.; output unit 47, such as various types of displays, speakers, etc.; storage unit 48, such as a disk, optical disk, etc.; and communication unit 49, such as a network card, modem, wireless transceiver, etc. The communication unit 49 allows the smart glasses to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0068] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as touch recognition methods.

[0069] In some embodiments, the touch recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded into and / or installed on smart glasses via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the touch recognition method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the touch recognition method by any other suitable means (e.g., by means of firmware).

[0070] Optionally, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the touch recognition method as provided in any embodiment of the present invention.

[0071] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0072] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0073] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0074] To provide interaction with the user, the systems and techniques described herein can be implemented on smart glasses that include: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the smart glasses. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0075] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0076] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0077] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0078] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0079] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A touch recognition method, characterized in that, Applied to smart glasses, the smart glasses including at least a vibration sensor for picking up voice information, the method includes: The touch vibration signal collected by the vibration sensor within the time window of the touch event is acquired, and the vibration feature vector is extracted from the touch vibration signal; The vibration feature vector is input into a pre-trained touch event recognition model, and the effective touch confidence is determined based on the model output of the touch event recognition model. The touch recognition result of the touch event is determined based on the effective touch confidence level.

2. The touch recognition method according to claim 1, characterized in that, The touch vibration signal collected by the vibration sensor within the time window for acquiring the touch event includes: Acquire the audio signal collected by the vibration sensor within the time window of the touch event; The audio signal is separated based on the touch vibration characteristics to determine the touch vibration signal.

3. The touch recognition method according to claim 1, characterized in that, The extraction of vibration feature vectors from the touch vibration signal includes: Waveform analysis is performed on the touch vibration signal to determine the time-domain and frequency-domain feature information in the touch vibration signal; The vibration feature vector of the touch vibration signal is determined by combining the time-domain feature information with the frequency-domain feature information. The time-domain feature information includes at least one of the following: single peak duration; peak interval; duration; The frequency domain feature information includes at least one of the following: frequency amplitude; frequency range; amplitude fluctuation; dominant frequency; proportion of high-frequency components; and broadband distribution information.

4. The touch recognition method according to claim 1, characterized in that, The step of determining the effective touch confidence level based on the model output of the touch event recognition model includes: Based on the model output of the touch event recognition model, determine the confidence level of the touch event belonging to each touch event type; The maximum value among the aforementioned confidence levels is determined as the effective touch confidence level.

5. The touch recognition method according to claim 1, characterized in that, The step of determining the touch recognition result of the touch event based on the effective touch confidence score includes: When the confidence level of the effective touch is greater than a preset confidence threshold, the touch recognition result of the touch event is determined as a valid touch; otherwise, the touch recognition result of the touch event is determined as a false touch.

6. The touch recognition method according to any one of claims 1-5, characterized in that, The vibration sensor is mounted on the nose pad or frame of the smart glasses.

7. A touch recognition device, characterized in that, Applied to smart glasses, the smart glasses at least include a vibration sensor for picking up voice information, and the touch recognition device includes: The vibration vector extraction module is used to acquire the touch vibration signal collected by the vibration sensor within the time window of the touch event, and extract the vibration feature vector from the touch vibration signal; The effective confidence determination module is used to input the vibration feature vector into a pre-trained touch event recognition model and determine the effective touch confidence based on the model output of the touch event recognition model. The recognition result determination module is used to determine the touch recognition result of the touch event based on the effective touch confidence level.

8. A type of smart glasses, characterized in that, include: Vibration sensors used to pick up voice information; At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the touch recognition method according to any one of claims 1-6.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the touch recognition method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the touch recognition method as described in any one of claims 1-6.

Images