Earphone control method and system based on AR glasses

Abstract

The invention provides an earphone control method and system based on AR glasses, and the method comprises the following steps: obtaining an earphone wearing state signal, and obtaining an environment audio signal collected by a microphone; preprocessing the environment audio signal, extracting a characteristic parameter sequence, and calculating a matching degree value with a reference scene sequence; when the wearing state signal indicates that the earphone is worn and the matching degree value is lower than a preset threshold value, controlling the earphone to be switched to a noise reduction output mode; and when the wearing state signal indicates that the earphone is worn and the matching degree is higher than a preset threshold value, controlling the earphone to be switched to the transparent output module. By calculating the matching degree, the output mode of the earphone is automatically switched, the user experience is improved, on the basis of existing capacitive in-ear detection, secondary judgment is carried out through the image sensor on the AR glasses to achieve in-ear detection, high temperature or sweat is prevented from affecting capacitance judgment, and the user experience is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AR glasses, in particular to a headset control method and system based on AR glasses. BACKGROUND

[0002] The current AR glasses audio output system adopts a scheme of setting a loudspeaker at the glasses leg, which faces the pain point of insufficient voice clarity in a noisy environment, and although switching to headset output can improve the signal-to-noise ratio, it needs to be manually operated by the user, affecting the immersive experience. SUMMARY

[0003] In view of the deficiencies of the prior art, the purpose of the present application is to provide a headset control method and system based on AR glasses, which aims to solve the technical problem that the headset output needs to be manually operated by the user in the prior art, affecting the immersive experience.

[0004] In order to achieve the above-mentioned purpose, in a first aspect, the present application provides a headset control method based on AR glasses, comprising the following steps: obtaining a headset wearing state signal and obtaining an environmental audio signal collected by a microphone; preprocessing the environmental audio signal and extracting a feature parameter sequence, while calculating the matching degree value of the feature parameter sequence and a pre-stored reference scene sequence; when the wearing state signal is worn and the matching degree value is lower than a preset threshold, controlling the headset to switch to a noise reduction output mode; when the wearing state signal is worn and the matching degree is higher than the preset threshold, controlling the headset to switch to a transparent output mode.

[0005] According to one aspect of the above technical solution, the capacitance change rate is obtained based on the capacitance sensor of the headset; when the capacitance change rate is greater than a preset change value, an ear image is obtained based on the image sensor on the AR glasses to calculate the displacement value of the headset relative to the auricle according to the ear image; if the displacement value is less than a preset displacement range, it is determined that the wearing signal state is worn; if the displacement value is greater than the preset displacement range, it is determined that the wearing signal state is not worn.

[0006] According to one aspect of the above technical solution, the step of preprocessing the environmental audio signal specifically comprises: obtaining the current output audio data of the AR glasses to filter the environmental audio signal according to the output audio data to obtain environmental audio data.

[0007] According to one aspect of the above technical solution, the characteristic parameter sequence includes audio frequency distribution parameters and energy parameters, and the calculation expression of the audio frequency distribution parameters is: ; Where C is the spectrum centroid value corresponding to the audio frequency distribution parameter, is the corresponding frequency value of frequency point k, is the power spectrum value of frequency point k, and m is the total number of spectrum analysis points; The calculation expression of the energy parameter is: ; Where E is the energy parameter, n is the total number of frames, is the amplitude value of the audio signal in the i-th frame.

[0008] According to one aspect of the above technical solution, the step of calculating the matching value between the feature parameter sequence and the pre-stored reference scene sequence specifically includes: Receive the feature parameter sequence corresponding to the current environment, and calculate the dynamic weight value of each feature point based on the short-term energy parameter in the feature parameter sequence; Based on the dynamic weight value of each feature point and the feature value of the reference scene sequence, the weighted matching value between the feature parameter sequence and the reference scene sequence is output.

[0009] According to one aspect of the above technical solution, the calculation expression of the weighted matching value is: ; Where, is the weighted matching value, i is the frame index, is the dynamic weight of the i-th frame, , are the energy value and spectrum centroid value of the current feature sequence respectively, , are the energy value and spectrum centroid value of the reference scene sequence respectively, is the energy mean of the current feature sequence, is the mean of the spectrum centroid of the current feature sequence, is the energy mean of the reference scene sequence, is the mean of the spectral centroid of the reference scene sequence, , are the weight coefficients of energy value and spectrum centroid value respectively.

[0010] According to one aspect of the above technical solution, the calculation expression of the dynamic weight value is: ; Where, a maximum value of short-term energies of all frames of the current sequence.

[0011] In another aspect, the application provides an AR glasses-based earphone control system, comprising: a collection module configured to acquire an earphone wearing state signal and an ambient audio signal collected by a microphone; a matching module configured to pre-process the ambient audio signal and extract a feature parameter sequence, and calculate a matching degree value of the feature parameter sequence and a pre-stored reference scene sequence; a first switching module configured to control the earphone to switch to a noise reduction output mode when the wearing state signal is worn and the matching degree value is lower than a preset threshold; a second switching module configured to control the earphone to switch to a transparent output mode when the wearing state signal is worn and the matching degree value is higher than the preset threshold.

[0012] According to an aspect of the above technical solution, the system further comprises: a wearing module configured to acquire a capacitance change rate based on a capacitance sensor of the earphone; when the capacitance change rate is greater than a preset change value, acquire an ear image based on an image sensor on the AR glasses to calculate a displacement value of the earphone relative to an auricle according to the ear image; if the displacement value is less than a preset displacement range, determine that the wearing signal state is worn; if the displacement value is greater than the preset displacement range, determine that the wearing signal state is not worn.

[0013] According to an aspect of the above technical solution, the matching module is specifically configured to: acquire current output audio data of the AR glasses to filter and process the ambient audio signal based on the output audio data to obtain ambient audio data.

[0014] According to an aspect of the above technical solution, the matching module is specifically further configured to: receive a feature parameter sequence corresponding to a current environment, and calculate a dynamic weight value of each feature point based on a short-term energy parameter in the feature parameter sequence; output a weighted matching degree value of the feature parameter sequence and the reference scene sequence based on the dynamic weight value of each feature point and a feature value of the reference scene sequence.

[0015] Compared with the prior art, the beneficial effects of the present application are that by processing the environmental audio signal, the matching degree of the feature parameter sequence and the pre-stored reference scene sequence is calculated, thereby automatically switching the output mode of the earphone, improving the user experience, and on the basis of the existing capacitive in-ear detection, the in-ear detection is realized through the image sensor on the AR glasses, the temperature is high or the sweat affects the capacitive value judgment, so as to further improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 a flowchart of the earphone control method based on AR glasses in the first embodiment of the present application is shown in the figure, as shown in the figure, the method comprises the following steps: Figure 2 a structure block diagram of the earphone control system based on AR glasses in the second embodiment of the present application is shown in the figure, as shown in the figure, the system comprises the following steps: The following specific embodiments will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0017] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. The drawings show several embodiments of the present application. However, the present application can be realized in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0018] It should be noted that when an element is referred to as being "fixedly attached" to another element, it can be directly on the other element or there can be an intervening element. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements can be present. The terms "vertical", "horizontal", "left", "right", and the like as used herein are for purposes of illustration and description only.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0020] Embodiment one Please refer to Figure 1 , the flowchart of the earphone control method based on AR glasses in the first embodiment of the present application is shown in the figure, as shown in the figure, the method comprises the following steps: Step S100, obtain the earphone wearing state signal, and obtain the environment audio signal collected by the microphone. Specifically, in the embodiment, the environment audio signal is collected by the microphone on the earphone or the microphone on the AR glasses. In the present solution, the environment audio signal is preferably collected by the AR glasses. When the ear-in detection confirmation signal of the earphone, i.e. the earphone wearing state signal, is received, the environment audio signal is collected based on the microphone on the AR glasses.

[0021] Preferably, in the embodiment, the method further comprises: obtaining a capacitance change rate based on the capacitance sensor of the earphone; when the capacitance change rate is greater than a preset change value, obtaining an ear image based on the image sensor on the AR glasses to calculate a displacement value of the earphone relative to the pinna based on the ear image; if the displacement value is less than a preset displacement range, determining that the wearing signal state is worn; if the displacement value is greater than the preset displacement range, determining that the wearing signal state is not worn. Specifically, compared with the single capacitance ear-in detection method in the prior art, the capacitance change rate is used as an activation condition, and then the ear image is used to determine the earphone wearing state, which prevents false determination caused by loose earphone, and prevents temperature or sweat from affecting the capacitance value determination, thereby achieving accurate ear-in detection, improving the use experience as a basis for subsequent earphone mode control.

[0022] Step S200, pre-process the environment audio signal and extract a feature parameter sequence, and calculate a matching degree value of the feature parameter sequence and a pre-stored reference scene sequence.

[0023] Preferably, in the embodiment, the step of pre-processing the environment audio signal specifically comprises: obtaining current output audio data of the AR glasses to filter the environment audio signal based on the output audio data to obtain environment audio data. Specifically, by filtering the environment audio signal based on the current output audio data of the AR glasses, the influence of the output audio of the AR glasses is solved, and the scene recognition accuracy is improved.

[0024] Further, in the embodiment, the feature parameter sequence comprises an audio frequency distribution parameter and an energy parameter, the calculation expression of the audio frequency distribution parameter is: ; wherein C is a frequency spectrum centroid value corresponding to the audio frequency distribution parameter, is a corresponding frequency value of the frequency point k, is the power spectrum value of frequency point k, m is the total number of spectrum analysis points. The spectrum centroid is a fingerprint feature for distinguishing the environment type, which can be judged for the low frequency, medium frequency and high frequency of the noise, so as to classify the scene, for example, the low frequency dominant mechanical noise such as fan, the medium frequency dominant human voice conversation such as office, and the high frequency dominant alarm sound or wind noise such as street, construction site, etc.

[0025] The calculation expression of the energy parameter is: ; In the formula, E is the energy parameter, n is the total number of frames, is the amplitude value of the i-th frame of the audio signal. The energy parameter represents the total intensity of the audio signal in the short-time frame, which is used to determine the noise intensity to drive the noise reduction intensity grading and avoid the risk of environmental isolation caused by excessive noise reduction.

[0026] Specifically, in the embodiment, the step of calculating the matching degree value of the feature parameter sequence and the pre-stored reference scene sequence specifically includes: receiving the feature parameter sequence corresponding to the current environment, and calculating the dynamic weight value of each feature point based on the short-term energy parameter in the feature parameter sequence; outputting the weighted matching degree value of the feature parameter sequence and the reference scene sequence based on the dynamic weight value of each feature point and the feature value of the reference scene sequence.

[0027] Preferably, in the embodiment, the calculation expression of the weighted matching degree value is: ; In the formula, is the weighted matching degree value, i is the frame index, is the dynamic weight of the i-th frame, , respectively, the energy value and the spectrum centroid value of the current feature sequence, , respectively, the energy value and the spectrum centroid value of the reference scene sequence, is the energy mean value of the current feature sequence, is the spectrum centroid mean value of the current feature sequence, is the energy mean value of the reference scene sequence, is the spectrum centroid mean value of the reference scene sequence, , respectively, the weight coefficient of the energy value and the spectrum centroid value.

[0028] The calculation expression of the dynamic weight value is: ; In the formula, It is the maximum value of the short-term energy of all frames in the current sequence.

[0029] Step S300: When the wearing status signal indicates that the headset is worn and the matching value is lower than a preset threshold, the headset is controlled to switch to a noise reduction output mode. Preferably, the intensity and mode of the noise reduction output mode can be adjusted according to the audio frequency distribution parameter and the energy parameter.

[0030] Step S400: When the wearing status signal indicates that the headset is worn and the matching degree is higher than a preset threshold, the headset is controlled to switch to a transparent output module.

[0031] It can be understood that in this embodiment, when the wearing status signal is not worn, the speaker of the AR glasses is used for audio output by default.

[0032] In summary, the headphone control method based on AR glasses in the above-mentioned embodiment of the present invention processes the ambient audio signal and calculates the matching degree between the characteristic parameter sequence and the pre-stored reference scene sequence, thereby automatically switching the output mode of the headphone to improve the user experience. On the basis of the existing capacitive in-ear detection, the image sensor on the AR glasses performs a secondary judgment to realize in-ear detection, thereby preventing high temperature or sweat from affecting the capacitance value judgment, so as to further improve the user experience.

[0033] Example 2 The second embodiment of the present application also provides an AR glasses-based headset control system, which is used to implement the embodiments and preferred implementations, and will not be repeated here. As used below, the terms "module", "unit", "sub-unit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0034] like Figure 2 As shown, the system includes: an acquisition module 100 , a matching module 200 , a first switching module 300 and a second switching module 400 .

[0035] The acquisition module 100 is used to obtain the earphone wearing status signal and the ambient audio signal collected by the microphone; A matching module 200 is configured to pre-process the ambient audio signal and extract a characteristic parameter sequence, and calculate a matching value between the characteristic parameter sequence and a pre-stored reference scene sequence; The first switching module 300 is configured to control the earphone to switch to a noise reduction output mode when the wearing status signal indicates that the earphone is worn and the matching value is lower than a preset threshold; The second switching module 400 is configured to control the earphone to switch to the transparent output module when the wearing state signal is worn and the matching degree is higher than the preset threshold.

[0036] Preferably, in the embodiment, the system further comprises: The wearing module is configured to acquire a capacitance change rate based on a capacitance sensor of the earphone. When the capacitance change rate is greater than a preset change value, an ear image is acquired based on an image sensor on the AR glasses to calculate a displacement value of the earphone relative to an auricle according to the ear image. If the displacement value is less than a preset displacement range, it is determined that the wearing signal state is worn. If the displacement value is greater than the preset displacement range, it is determined that the wearing signal state is not worn.

[0037] Preferably, in the embodiment, the matching module 200 is specifically configured to: Acquire current output audio data of the AR glasses to filter the ambient audio signal according to the output audio data to obtain ambient audio data.

[0038] Preferably, the matching module 200 is specifically further configured to: Receive a feature parameter sequence corresponding to the current environment, and calculate a dynamic weight value of each feature point based on a short-term energy parameter in the feature parameter sequence. Output a weighted matching degree value of the feature parameter sequence and the reference scene sequence based on the dynamic weight value of each feature point and the feature value of the reference scene sequence.

[0039] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the embodiments are described, however, as long as the combinations of the technical features do not contradict, they should be considered within the scope of the present disclosure.

[0040] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that, for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A headset control method based on AR glasses, characterized in that: The following steps are involved: Obtain the earphone wearing status signal and the ambient audio signal collected by the microphone; Preprocessing the ambient audio signal and extracting a characteristic parameter sequence, and calculating a matching value between the characteristic parameter sequence and a pre-stored reference scene sequence; When the wearing status signal indicates that the headset is worn and the matching value is lower than a preset threshold, the headset is controlled to switch to a noise reduction output mode; When the wearing status signal indicates that the headset is worn and the matching degree is higher than a preset threshold, the headset is controlled to switch to the transparent output module.

2. The headphone control method based on AR glasses according to claim 1, characterized in that: The method further comprises: Capacitance change rate is obtained based on the capacitive sensor of the earphone; When the capacitance change rate is greater than a preset change value, acquiring an ear image based on an image sensor on the AR glasses, and calculating a displacement value of the earphone relative to the auricle according to the ear image; If the displacement value is less than the preset displacement range, the wearing signal state is determined to be worn; If the displacement value is greater than a preset displacement range, it is determined that the wearing signal state is not worn.

3. The headphone control method based on AR glasses according to claim 1, characterized in that: The step of preprocessing the ambient audio signal specifically includes: The current output audio data of the AR glasses is obtained, and the ambient audio signal is filtered according to the output audio data to obtain the ambient audio data.

4. The headphone control method based on AR glasses according to claim 1, characterized in that: The characteristic parameter sequence includes audio frequency distribution parameters and energy parameters. The calculation expression of the audio frequency distribution parameters is: ； Where C is the spectrum centroid value corresponding to the audio frequency distribution parameter, is the corresponding frequency value of frequency point k, is the power spectrum value of frequency point k, and m is the total number of spectrum analysis points; The calculation expression of the energy parameter is: ； Where E is the energy parameter, n is the total number of frames, is the amplitude value of the audio signal in the i-th frame.

5. The headphone control method based on AR glasses according to claim 4, characterized in that: The step of calculating the matching value between the feature parameter sequence and the pre-stored reference scene sequence specifically includes: Receive the feature parameter sequence corresponding to the current environment, and calculate the dynamic weight value of each feature point based on the short-term energy parameter in the feature parameter sequence; Based on the dynamic weight value of each feature point and the feature value of the reference scene sequence, the weighted matching value between the feature parameter sequence and the reference scene sequence is output.

6. The headphone control method based on AR glasses according to claim 5, characterized in that: The calculation expression of the weighted matching value is: ； Where, is the weighted matching value, i is the frame index, is the dynamic weight of the i-th frame, , are the energy value and spectrum centroid value of the current feature sequence respectively, , are the energy value and spectrum centroid value of the reference scene sequence respectively, is the energy mean of the current feature sequence, is the mean of the spectrum centroid of the current feature sequence, is the energy mean of the reference scene sequence, is the mean of the spectral centroid of the reference scene sequence, , are the weight coefficients of energy value and spectrum centroid value respectively.

7. The headphone control method based on AR glasses according to claim 6, characterized in that: The calculation expression of the dynamic weight value is: ； Where, It is the maximum value of the short-term energy of all frames in the current sequence.

8. An earphone control system based on AR glasses, characterized in that: include: The acquisition module is used to obtain the earphone wearing status signal and the ambient audio signal collected by the microphone; A matching module, configured to pre-process the ambient audio signal and extract a characteristic parameter sequence, and simultaneously calculate a matching value between the characteristic parameter sequence and a pre-stored reference scene sequence; A first switching module is configured to control the earphone to switch to a noise reduction output mode when the wearing status signal indicates that the earphone is worn and the matching value is lower than a preset threshold; The second switching module is used to control the earphone to switch to the transparent output module when the wearing status signal is worn and the matching degree is higher than a preset threshold.

9. The headset control system based on AR glasses according to claim 8, characterized in that: The system further comprises: Wearable module, used to obtain capacitance change rate based on the capacitive sensor of the headset; When the capacitance change rate is greater than a preset change value, acquiring an ear image based on an image sensor on the AR glasses, and calculating a displacement value of the earphone relative to the auricle according to the ear image; If the displacement value is less than the preset displacement range, the wearing signal state is determined to be worn; If the displacement value is greater than a preset displacement range, it is determined that the wearing signal state is not worn.

10. The headset control system based on AR glasses according to claim 8, characterized in that: The matching module is specifically used for: The current output audio data of the AR glasses is obtained, and the ambient audio signal is filtered according to the output audio data to obtain the ambient audio data.

Images