A non-contact smart earphone or glasses interaction method

Abstract

The application discloses a non-contact intelligent earphone or glasses interaction method, and a specific process is as follows: a sensor receives a vibration signal transmitted through human tissue, a calculation unit analyzes and judges whether the vibration signal is an operation instruction according to an intelligent analysis algorithm to execute further operation or continue to keep an existing state; the sensor is connected with the calculation unit. The sensor judges whether the vibration signal is from a head instruction area according to the intensity and vibration characteristic information of the vibration signal; the head instruction area comprises an ear behind area and an ear front area; the intelligent analysis algorithm sequentially comprises a data preprocessing module, a frequency spectrum processing module and an identification module, and further comprises a training module for optimizing the identification module. The application has the beneficial technical effects of good privacy, a large operation area and comfort and randomness, and is a new intelligent earphone or glasses interaction method which is interesting and has a high-tech feeling.

Description

Technical Field

[0001] This invention relates to smart wearable devices, and more particularly to a non-contact smart earphone or glasses interaction method. Background Technology

[0002] Currently, human-computer interaction in smart wearable devices such as true wireless earbuds (TWS) or smart glasses can be achieved through voice, touch, and tapping. Voice interaction lacks privacy because it requires speaking, and it is not suitable for use in places such as classrooms, libraries, offices, cafes, or restaurants. The sensing area on smart wearable devices is very limited, so touch and tapping are prone to misoperation or insensitivity. In addition, the vibration transmission of tapping can cause ear discomfort or pain.

[0003] In summary, the existing technology has the following problems:

[0004] 1. Voice interaction lacks privacy;

[0005] 2. The sensing area for touch and tap methods is too small, resulting in unsatisfactory operation;

[0006] 3. The vibration transmitted through tapping can irritate the eardrum and cause discomfort. Summary of the Invention

[0007] The purpose of this invention is to provide a human-computer interaction method for wireless smart headphones or smart glasses that offers good privacy, a large operating area, and comfort and ease of use.

[0008] To address the aforementioned problems, this invention proposes a non-contact smart earphone or glasses interaction method. The specific process is as follows: a sensor receives vibration signals transmitted through human tissue; a computing unit analyzes the vibration signal using an intelligent analysis algorithm to determine whether it is an operation command and whether to execute further operations or maintain the current state. After determining that the vibration signal is an operation command, the computing unit further determines the content of the command to execute further operations. The sensor is connected to the computing unit. The intelligent analysis algorithm sequentially includes a data preprocessing module, a spectrogram processing module, and a recognition module, and also includes a training module for optimizing the recognition module.

[0009] The interaction method also includes a head command area, and the sensor determines whether the vibration signal originates from the head command area by measuring the intensity and vibration characteristic information of the vibration signal.

[0010] The vibration signal is generated by tapping or accidental touching within the head command area.

[0011] The head command area includes the area behind the ear and the area in front of the ear.

[0012] The operation actions include single taps, double taps, triple taps, and multiple taps of three or more times. Furthermore, it includes combinations of single taps, double taps, triple taps, and multiple taps of three or more times, for example: a single tap followed by a 1-3 second pause, then a double tap. Different operation actions or combinations of operation actions are used to achieve human-computer interaction with different commands.

[0013] The human tissues include bones and soft tissues, but the transmission of vibrations mainly relies on bone conduction. Sensors in the smart headphones or smart glasses worn by the user collect vibration signals transmitted from the human tissues. The computing unit connected to the sensors judges and identifies the vibration signals. If it is not a command signal, it maintains the current state; if it is a command signal, it executes the content of the command signal.

[0014] The sensor includes a vibration sensor, an acceleration sensor, or a six-axis sensor. The vibration signal is converted into an electrical signal that the computing unit can recognize for further identification.

[0015] The computing unit is located inside or outside the smart earphones or smart glasses; for example, the computing unit is located inside the charging case of the smart earphones. Since the charging case has more space and power, it can have more powerful computing capabilities to provide better recognition accuracy and faster response.

[0016] The data preprocessing module passes the data from the six-axis sensor through a low-pass filter to remove high-frequency noise, leaving only short-duration pulses caused by human movement. Then, the amplitude (absolute value) of the purified signal is calculated, and the obtained amplitude sequence is smoothed using a two-round moving average. Next, the maximum value Mmax and minimum value Mmin of the smoothed amplitude sequence are found.

[0017] The spectrogram processing module first divides the signal into multiple short segments with fixed overlap. The segment length and overlap length are both set to 128. Then, each segment is windowed using a Hamming window, and its spectrum is calculated using an STFT, which generates a series of complex coefficients for each segment. Now, the signal along each axis is converted into an STFT matrix, which records the amplitude and phase at each time and frequency.

[0018] The recognition module, through the two processing steps described above, can input the adjusted acceleration spectrum image into various standard neural networks, such as VGG, ResNet, Wide-ResNet, and DenseNet. DenseNet is typically chosen as the base network for all recognition tasks.

[0019] During the training phase, the training module uses cross-entropy as the training loss and optimizes the model weights through a piecewise momentum optimizer to learn more generalizable features and promote convergence, thereby enhancing the accuracy of vibration signal judgment and recognition and its universality for different users.

[0020] In summary, this invention offers the advantages of good privacy, a large operating area, and comfortable and convenient operation. It represents a very interesting and high-tech new way of interacting with smart headphones or glasses. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the operation of Embodiment 1 of the present invention;

[0022] Figure 2 This is a schematic diagram of the dense block network structure of Embodiment 1 of the present invention;

[0023] Figure 3 This is a schematic diagram of the overall network structure of Embodiment 1 of the present invention;

[0024] Figure 4 This is a schematic diagram of Embodiment 3 of the present invention;

[0025] In the diagram: 1. Behind-the-ear area; 2. In front-the-ear area; 3. Smart earphone; 4. Smart glasses; 5. Six-axis sensor; 6. Computing unit. Detailed Implementation

[0026] The following is in conjunction with the appendix Figure 1-4 The technical solutions will be further explained with reference to specific embodiments to help understand the content of the present invention.

[0027] Example 1:

[0028] like Figure 1 As shown, a non-contact smart earphone interaction method involves tapping the area behind the ear (1) or in front of the ear (2) on the head. The vibration signal generated by the tap is transmitted through the body tissue to a six-axis sensor (5) inside the smart earphone (3). The built-in intelligent control software's computing unit (6) analyzes the vibration signal using an intelligent analysis algorithm to determine whether it is an operation command and whether to execute further operations or maintain the current state. The six-axis sensor (5) is connected to the computing unit (6). Other related components in the earphone, such as the battery, are not shown in the figure.

[0029] The intelligent analysis algorithm sequentially includes a data preprocessing module, a spectrogram processing module, and a recognition module, and also includes a training module for optimizing the recognition module.

[0030] The data preprocessing module passes the data from the six-axis sensor through a low-pass filter to remove high-frequency noise, leaving only short-duration pulses caused by human movement. The amplitude (absolute value) of the purified signal is then calculated, and the resulting amplitude sequence is smoothed using a two-round moving average. Next, the maximum value Mmax and minimum value Mmin of the smoothed amplitude sequence are found. During this process, the first and last 100 amplitude values ​​are discarded because they do not have enough neighboring samples for averaging. The obtained minimum value is approximately the magnitude of the noise signal. The smoothed amplitude sequence is then iterated through, identifying all regions with amplitudes greater than the threshold of 0.8Mmin + 0.2Mmax. The presence of a knock signal is detected in each located region. The start and end points of each located region are moved forward and backward by 100 and 200 samples, respectively. Finally, the filtered acceleration signal is segmented into multiple short signals, each corresponding to a specific test state.

[0031] The spectrogram processing module first divides the signal into multiple short segments with fixed overlap. The segment length and overlap length are both set to 128. Then, each segment is windowed using a Hamming window, and its spectrum is calculated using an STFT, which generates a series of complex coefficients for each segment. Now, the signal along each axis is converted into an STFT matrix, which records the amplitude and phase at each time and frequency, as shown below:

[0032]

[0033] Where x(n) and |STFT{x(n)}(m,w)| represent the single-axis acceleration signal and the size of its corresponding STFT matrix, respectively. Since there are acceleration signals along all three axes, each single-word signal can generate three spectrograms. Further, the three two-dimensional spectrograms of the signal are converted into a PNG format RGB image. For this, the three m×n spectrograms are first fitted into an m×n×3 tensor. Then, the square root of all elements in the tensor is taken, and the resulting values ​​are mapped to integers between 0 and 255. The reason for taking the square root is that most elements in the original two-dimensional spectrogram are very close to zero. Directly mapping these elements to integers between 0 and 255 would result in a significant loss of information. Finally, the m×n×3 tensor is exported as a PNG format image.

[0034] The recognition module, through the two processing steps described above, can input the adjusted acceleration spectrum image into various standard neural networks, such as VGG, ResNet, Wide-ResNet, and DenseNet. DenseNet is typically chosen as the base network for all recognition tasks. Compared to traditional deep networks like VGG and ResNet, DenseNet introduces connections between each layer and all layers preceding it, resulting in a total of (L + 1)L / 2 connections in an L-layer network. For example, as shown in the common block diagram of DenseNet (e.g.) Figure 2 As shown), layers one through four are directly linked to layer five. In other words, the l-th element takes the concatenation of feature maps from layer 0 (input image) to layer (l-1) as input, which can be expressed mathematically as:

[0035]

[0036] H1 and x1 represent the function and feature map of layer l, respectively. [x0, x1, ..., xl-1] represents the juxtaposition of feature maps from layer 0 to layer 1-1. These direct connections allow all layers to receive and reuse functions from their previous layers; therefore, DenseNet does not need to use redundant parameters or nodes to maintain information from previous layers. Thus, DenseNet can achieve performance comparable to VGG and ResNet with fewer nodes (parameters). Furthermore, the improved information flow and gradient flow throughout the network mitigate the vanishing gradient trend and make DenseNet easier to train. Empirically, DenseNet has indeed achieved the best accuracy in recognition tasks with fewer parameters and lower computational cost (compared to VGG and ResNet). Figure 3 The overall network structure utilized is shown, which consists of Figure 2 It consists of multiple dense blocks as shown.

[0037] During the training phase, the training module uses cross-entropy as the training loss and optimizes the model weights through a piecewise momentum optimizer to learn more generalizable features and promote convergence. Specifically, the adaptive momentum optimization process is first executed with a large step size (e.g., 0.1) to learn generalizable features, and then fine-tuned with smaller step sizes to promote convergence. Weight reduction is also added to the training loss, and the dropout rate is set to 0.3 to enhance generality.

[0038] Example 2:

[0039] Still Figure 1As shown, a non-contact smart earphone interaction method involves tapping the area behind the ear (1) or in front of the ear (2) on the head. The vibration signal generated by the tap is transmitted through the body tissue to a six-axis sensor (5) inside the smart earphone (3). An external intelligent control software computing unit (6) analyzes the vibration signal using an intelligent analysis algorithm to determine whether it is an operation command and whether to execute further operations or maintain the current state. The six-axis sensor (5) is connected to the computing unit (6). Other related components, such as the battery, are not shown in the figure.

[0040] Its specific working method and algorithm are the same as those in Example 1, so they will not be described again.

[0041] Example 3:

[0042] like Figure 4 As shown, a non-contact smart glasses interaction method involves tapping the area behind the ear (1) or in front of the ear (2) on the head. The vibration signal generated by the tap is transmitted through the body tissue to a six-axis sensor (5) inside the smart glasses (4). The built-in intelligent control software's computing unit (6) analyzes the vibration signal using an intelligent analysis algorithm to determine whether it is an operation command and whether to execute further operations or maintain the current state. The six-axis sensor (5) is connected to the computing unit (6). Other related components in the smart glasses, such as the battery, are not shown in the figure.

[0043] Its specific working method and algorithm are the same as those in Example 1, so they will not be described again.

[0044] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any simple equivalent changes and modifications made in accordance with the scope and description of this patent application shall fall within the scope of this patent.

Claims

1. A non-contact smart earphone or glasses interaction method, characterized in that: The sensor receives vibration signals transmitted through human tissue. The computing unit analyzes the vibration signals using an intelligent analysis algorithm to determine whether the vibration signals represent an operation command and whether to execute further operations or maintain the current state. The sensor is connected to the computing unit. The intelligent analysis algorithm sequentially includes a data preprocessing module, a spectrogram processing module, and a recognition module, and also includes a training module for optimizing the recognition module. The data preprocessing module includes the following processing steps for the vibration signal: A low-pass filter is used to retain short-duration pulses caused by human movement; Then calculate the absolute value of the pulse signal amplitude; The obtained amplitude sequence was smoothed using two rounds of moving averages; Find the maximum value Mmax and minimum value Mmin of the smooth amplitude sequence; the first 100 and last 100 amplitude values ​​are discarded; the obtained minimum value is the magnitude of the noise signal; Traverse the smoothed amplitude sequence to find all regions where the amplitude is greater than the threshold of 0.8Mmin + 0.2Mmax; Detect the presence of a tapping signal in each located area; The starting point of each positioning area is moved forward by 100 samples, and the ending point is moved backward by 200 samples; The filtered acceleration signal is segmented into multiple short signals, each corresponding to a specific test state.

2. The non-contact smart earphone or glasses interaction method as described in claim 1, characterized in that: It also includes a head command area, and the sensor determines whether the vibration signal comes from the head command area by the intensity and vibration characteristic information of the vibration signal.

3. The non-contact smart earphone or glasses interaction method as described in claim 1, characterized in that: The vibration signal is generated when the user taps the head command area or when there is an accidental contact.

4. The non-contact smart earphone or glasses interaction method as described in claim 2, characterized in that: The head command area includes the area behind the ear and the area in front of the ear.

5. The non-contact smart earphone or glasses interaction method as described in claim 3, characterized in that: The operation includes a single tap, a double tap, a triple tap, or multiple taps of three or more.

6. The non-contact smart earphone or glasses interaction method as described in claim 3, characterized in that: The operation includes a single tap, a double tap, a triple tap, or a combination of multiple taps of three or more.

7. The non-contact smart earphone or glasses interaction method as described in claim 1, characterized in that: The sensor is a vibration sensor, an acceleration sensor, or a six-axis sensor.

8. The non-contact smart earphone or glasses interaction method as described in claim 1, characterized in that: The computing unit is located inside or outside the smart earphone or smart glasses.

9. The non-contact smart earphone or glasses interaction method as described in claim 1, characterized in that: The computing unit has built-in intelligent control software with the intelligent analysis algorithm.

Images