Wearable device offline scene interaction method and system based on sound wave carrier information

Abstract

The invention is suitable for the technical field of data communication between a wearable device and an offline scene, and provides a wearable device offline scene interaction method and system based on sound wave carrier information, and the method comprises the steps: firstly, based on a microphone array, rapidly obtaining environment sound wave information, then carrying out the modulation processing of the environment sound wave information, and carrying out the processing of the environment sound wave information; according to the invention, carrier data information is effectively generated, multi-dimensional scene information is accurately determined based on the carrier data information, and finally the multi-dimensional scene information is sent to a receiving device (such as a loudspeaker or a display device) of the wearable device. According to the invention, a non-contact and code-scanning-free real-time scene interaction scheme can be provided, intelligent information interaction between the wearable device and the physical environment is realized through sound wave carrier waves, the overall interaction process is simplified, the dependency on visual identification is reduced, a user can interact in a more natural and visual manner, and the user experience is improved. And the use experience of the user is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of data communication between wearable devices and offline scenes, and more specifically, to a wearable device offline scene interaction method and system based on acoustic wave carrier information. Background Art

[0002] Currently, in the data communication field between smart hardware such as mobile phones and offline scenarios, users generally scan QR codes or barcodes. However, traditional barcode scanning technology mainly relies on cameras to capture images, which is easily affected by recognition distance, ambient light, and recognition angle. QR codes are also easily damaged or tampered with. In addition, for users of wearable devices such as smart headphones, rings, and AR glasses, there is a problem of being unable to scan codes or scanning codes inconveniently. Summary of the Invention

[0003] Based on this, the embodiments of the present application provide a wearable device offline scene interaction method and system based on acoustic wave carrier information to solve the problem of poor user experience in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a wearable device offline scene interaction method based on acoustic wave carrier information, which is applied to a wearable device, and the method includes:

[0005] Based on the preset microphone array, obtain environmental sound wave information;

[0006] Based on differential phase shift keying technology, the environmental sound wave information is modulated to generate carrier data information;

[0007] Determining multi-dimensional scene information based on the carrier data information;

[0008] The multi-dimensional scene information is sent to a receiving device of the wearable device, wherein the receiving device includes a speaker or a display screen.

[0009] Compared with the existing technology, the beneficial effect is: the offline scene interaction method of wearable devices based on sound wave carrier information provided by the embodiment of the present application, the terminal device can first quickly obtain environmental sound wave information based on the microphone array, and then modulate and process the environmental sound wave information to effectively generate carrier data information, and then accurately determine the multi-dimensional scene information based on the carrier data information, and finally send the multi-dimensional scene information to the receiving device of the wearable device, thereby realizing a simplified overall interaction process, reducing dependence on visual recognition, effectively improving user experience, and to a certain extent solving the problem of poor user experience.

[0010] In a second aspect, an embodiment of the present application provides an offline scene interaction system for a wearable device based on acoustic wave carrier information, which is applied to a wearable device, and the system includes:

[0011] Ambient sound wave information acquisition module: used to obtain ambient sound wave information based on a preset microphone array;

[0012] Carrier data information generation module: used to modulate the environmental sound wave information based on differential phase shift keying technology to generate carrier data information;

[0013] A multi-dimensional scene information determination module: configured to determine multi-dimensional scene information based on the carrier data information;

[0014] Multi-dimensional scene information sending module: used to send the multi-dimensional scene information to the receiving device of the wearable device, wherein the receiving device includes a speaker or a display screen.

[0015] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect described above when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method of the first aspect described above are implemented.

[0017] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art.

[0019] Figure 1 This is a flowchart of an offline scene interaction method for a wearable device provided by an embodiment of the present application;

[0020] Figure 2 This is a flow chart of the process after step S101 in the offline scene interaction method for a wearable device provided in one embodiment of the present application;

[0021] Figure 3 This is a module block diagram of a wearable device offline scene interaction system provided by an embodiment of the present application;

[0022] Figure 4 This is a schematic diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0024] In the description of this application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0026] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0027] See also Figure 1 , Figure 1 : This is a flow chart of the offline scene interaction method of a wearable device based on acoustic carrier information provided in an embodiment of the present application. In this embodiment, the execution subject of the offline scene interaction method of a wearable device is a terminal device. It is understandable that the types of terminal devices include but are not limited to wearable smart devices such as AR glasses, smart headphones, bracelets, rings, as well as mobile phones, tablet computers, laptops, ultra-mobile personal computers (UMPC), netbooks, and personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0028] See also Figure 1 The wearable device offline scene interaction method provided in the embodiment of the present application includes but is not limited to the following steps:

[0029] In S100 , ambient sound wave information is acquired based on a preset microphone array.

[0030] Without loss of generality, the wearable device offline scene interaction method can be applied to wearable devices, which can be AR glasses, smart headphones, and other smart wearable devices. The wearable device can be integrated with a high-fidelity microphone array, such as a MEMS microphone, and the frequency response range of the microphone array can be 15kHz to 19kHz. In one possible implementation, the terminal device can control the microphone array to dynamically switch to the 18kHz or 19kHz frequency band based on the ambient noise.

[0031] For example, in an offline interaction scenario, a sound wave emission system can be pre-deployed. The sound wave emission system can be a miniaturized sound wave transmitter with a power consumption of less than 0.5 watts and an effective radius of less than 5 meters. The miniaturized sound wave transmitter can continuously emit high-frequency sound waves in the range of 15kHz to 19kHz.

[0032] Specifically, the terminal device can first obtain ambient sound wave information based on a preset microphone array, wherein the ambient sound wave information is used to describe the ambient sound waves in the offline interaction scenario.

[0033] In S200 , the ambient sound wave information is modulated based on the differential phase shift keying technology to generate carrier data information.

[0034] Specifically, after the terminal device obtains the ambient sound wave information, the terminal device can modulate the ambient sound wave information based on the Differential Phase Shift Keying (DPSK) technology to generate carrier data information, thereby realizing the extraction of the carrier data using the demodulation technology.

[0035] In some possible implementations, in order to reduce the adverse effects of environmental interference on user experience, after step S100, the method further includes but is not limited to the following steps:

[0036] In S101 , noise removal is performed on the ambient sound wave information based on a preset adaptive filtering algorithm to generate noise-reduced sound wave information.

[0037] Specifically, the terminal device can first perform noise removal processing on the ambient sound wave information based on a preset adaptive filtering algorithm to generate noise reduction sound wave information, thereby eliminating ambient noise interference. The adaptive filtering algorithm can be a least mean square (LMS) algorithm, a fast adaptive algorithm (NLMS), a recursive least squares (RLS) algorithm, or an adaptive linear prediction (ALP) algorithm.

[0038] Accordingly, the above step S200 includes but is not limited to the following steps:

[0039] In S201, based on the differential phase shift keying technology, the noise reduction carrier information is modulated to generate carrier data information.

[0040] Specifically, after the terminal device generates the noise reduction signal information, the terminal device can modulate the noise reduction signal information based on differential phase shift keying technology to generate carrier data information. The specific operation process is similar to the relevant content in the above step S200, so it is not repeated here.

[0041] In some possible implementations, to further reduce the adverse effects of environmental interference, see Figure 2 After step S101, the method further includes but is not limited to the following steps:

[0042] In S102, the noise-reduced acoustic wave information is subjected to Hanning window Fourier transform processing to obtain acoustic wave spectrum information.

[0043] Specifically, the terminal device may first perform Hanning window Fourier transform processing on the noise reduction acoustic wave information to obtain acoustic wave spectrum information, wherein the acoustic wave spectrum information is used to describe the spectrum of the noise reduction acoustic wave information.

[0044] In S103 , based on a preset Gezil algorithm, characteristic frequency recognition processing is performed on the sound wave spectrum information to determine target frequency information.

[0045] Specifically, after the terminal device obtains the sound wave spectrum information, the terminal device can perform characteristic frequency recognition processing on the sound wave spectrum information based on a preset Goertzel algorithm to determine the target frequency information, wherein the target frequency information is used to describe the characteristic frequency in the sound wave spectrum information.

[0046] In S104, channel equalization processing is performed on the target frequency information based on a preset least mean square adaptive filter to generate optimized signal information.

[0047] Specifically, after the terminal device determines the target frequency information, the terminal device can perform channel equalization on the target frequency information based on a preset least mean square (LMS) adaptive filter to generate optimized signal information, wherein the optimized signal information is used to describe the target frequency information after channel equalization.

[0048] In S105 , based on a preset Reed-Solomon decoding algorithm, error correction processing is performed on the optimized signal information to generate corrected signal information.

[0049] Specifically, after the terminal device generates the optimized signal information, the terminal device can perform error correction processing on the optimized signal information based on a preset Reed-Solomon (RS) decoding algorithm to generate corrected signal information, wherein the corrected signal information is used to describe the optimized signal information after the error correction processing.

[0050] In S106, a CRC check is performed on the correction signal information.

[0051] Specifically, after the terminal device generates the correction signal information, the terminal device may perform CRC check processing on the correction signal information.

[0052] In S107 , if the CRC check passes, the differential phase shift keying technology is continued to be used to modulate the noise reduction signal information to generate carrier data information.

[0053] Specifically, if the CRC check passes, the terminal device may continue to execute the above step S201, thereby implementing modulation processing on the optimized signal information after error correction processing.

[0054] In S300 , multi-dimensional scene information is determined based on carrier data information.

[0055] Specifically, after the terminal device generates the carrier data information, the terminal device can effectively determine the multi-dimensional scene information based on the carrier data information, wherein the multi-dimensional scene information is used to describe the multi-dimensional scene interaction information,

[0056] In some possible implementations, in order to effectively determine the multi-dimensional scene information, step S300 includes but is not limited to the following steps:

[0057] In S310 , multi-dimensional scene information is determined according to the unique ID code information in the carrier data information.

[0058] Without loss of generality, the high-frequency sound waves emitted by the miniaturized sound wave transmitter may carry a unique ID code, so that the ambient sound wave information may include unique ID code information, wherein the unique ID code information is used to describe the unique ID code corresponding to the ambient sound wave information.

[0059] Specifically, the terminal device can accurately determine the multi-dimensional scene information based on the unique ID coding information in the carrier data information, where the multi-dimensional scene information includes three-dimensional display model information, price discount information, product inventory information, navigation data information or cultural relics explanation audio and video information, etc. The three-dimensional display model information is used to describe data about the three-dimensional display model, the price discount information is used to describe data about the price discount situation, the product inventory information is used to describe data about the product inventory situation, the navigation data information is used to describe data about the navigation route, and the audio and video explanation presents digital information of the corresponding cultural relics, etc.

[0060] In S400, multi-dimensional scene information is sent to a receiving device of the wearable device.

[0061] Specifically, the receiving device includes a speaker or a display screen; after the terminal device determines the multi-dimensional scene information, the terminal device can send the multi-dimensional scene information to the receiving device of the wearable device. For example, when the receiving device is a display screen, the terminal device can enable the AR display engine to superimpose the multi-dimensional scene information on the virtual information layer in real time.

[0062] It should be noted that the offline scene interaction method of wearable devices has a high interaction efficiency, and the information acquisition time is less than 300 mm, while the traditional code scanning takes an average of 2.5 seconds; the offline scene interaction method of wearable devices has good environmental adaptability and still maintains a 95% recognition rate in an 85 decibel noise environment; the offline scene interaction method of wearable devices has a power consumption advantage and can reduce energy consumption by 60% compared to Bluetooth communication; the offline scene interaction method of wearable devices is also highly secure and can prevent man-in-the-middle attacks through dynamic keys.

[0063] Without loss of generality, the offline scene interaction method of the wearable device can be applied to the application scenario of shopping mall shopping guide, such as delivering promotional information through sound wave tags on commodity shelves; the offline scene interaction method of the wearable device can also be applied to the application scenario of museum cultural relics guidance, such as automatically pushing AR commentary content to exhibits; the offline scene interaction method of the wearable device can also be applied to the application scenario of smart home, such as feeding back the status of home appliances to glasses through sound waves; the offline scene interaction method of the wearable device can also be applied to the application scenario of transportation hubs, such as platform broadcasting synchronized AR navigation paths.

[0064] For example, taking the smart supermarket scenario as an example, acoustic beacons are deployed on the product shelves, and the code contains the shelf ID and / or product category. Then the user wears AR glasses and enters the detection range. The AR glasses automatically receive and parse the 19kHz carrier signal, and then return the product price, inventory and / or 3D display model to the AR glasses through the cloud. The AR interface of the AR glasses displays the virtual label of the product in real time, and the voice price comparison function can be triggered when the user looks at a specific product.

[0065] The implementation principle of the offline scene interaction method of wearable devices based on sound wave carrier information in the embodiment of the present application is: the terminal device can first quickly obtain environmental sound wave information based on the microphone array, and then modulate and process the environmental sound wave information to effectively generate carrier data information, and then accurately determine the multi-dimensional scene information based on the carrier data information, and finally send the multi-dimensional scene information to the receiving device of the wearable device, thereby realizing intelligent information interaction between the wearable device and the physical environment through the sound wave carrier, simplifying the overall interaction process, reducing the dependence on visual recognition, and effectively improving the user experience.

[0066] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0067] The embodiment of the present application also provides a wearable device offline scene interaction system based on acoustic carrier information. For ease of description, only the parts related to the present application are shown, such as Figure 3 As shown, the system 30 includes:

[0068] Ambient sound wave information acquisition module 31: used to acquire ambient sound wave information based on a preset microphone array;

[0069] Carrier data information generating module 32: used to modulate the ambient sound wave information based on differential phase shift keying technology to generate carrier data information;

[0070] Multi-dimensional scene information determination module 33: used to determine multi-dimensional scene information based on carrier data information;

[0071] The multi-dimensional scene information sending module 34 is used to send the multi-dimensional scene information to a receiving device of the wearable device, wherein the receiving device includes a speaker or a display screen.

[0072] Optionally, the system 30 further includes:

[0073] The noise reduction wave information generation module is used to remove noise from the ambient sound wave information based on a preset adaptive filtering algorithm to generate noise reduction wave information.

[0074] Accordingly, the carrier data information generating module 32 includes:

[0075] Carrier data information generation submodule: used to modulate the noise reduction wave information based on differential phase shift keying technology to generate carrier data information.

[0076] Optionally, the system 30 further includes:

[0077] Acoustic spectrum information acquisition module: used to perform Hanning window Fourier transform processing on the noise-reduced acoustic information to obtain acoustic spectrum information;

[0078] Target frequency information determination module: used to perform characteristic frequency recognition processing on the sound wave spectrum information based on the preset Gezil algorithm to determine the target frequency information;

[0079] Optimized signal information generation module: used to perform channel equalization processing on the target frequency information based on a preset minimum mean square adaptive filter to generate optimized signal information;

[0080] Correction signal information generation module: used to perform error correction processing on the optimized signal information based on a preset Reed-Solomon decoding algorithm to generate corrected signal information;

[0081] CRC check module: used to perform CRC check on the correction signal information;

[0082] Carrier data information generation module: used to continue to perform modulation processing on the noise reduction wave information based on differential phase shift keying technology to generate carrier data information if the CRC check passes.

[0083] Optionally, the ambient sound wave information includes unique ID code information; the multi-dimensional scene information determination module 33 includes:

[0084] Multi-dimensional scene information determination submodule: used to determine multi-dimensional scene information based on the unique ID coding information in the carrier data information, wherein the multi-dimensional scene information includes three-dimensional display model information, price discount information, product inventory information or navigation data information.

[0085] Optionally, the microphone array has a frequency response range of 15kHz to 19kHz.

[0086] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0087] The present application also provides a terminal device, such as Figure 4As shown, the terminal device 40 of this embodiment includes: a processor 41, a memory 42, and a computer program 43 stored in the memory 42 and executable on the processor 41. When the processor 41 executes the computer program 43, the steps in the above-mentioned wearable device offline scene interaction method embodiment are implemented, such as Figure 1 In the steps S100 to S400 shown, or when the processor 41 executes the computer program 43, the functions of each module in the above-mentioned device are realized, for example Figure 3 The functions of modules 31 to 34 are shown.

[0088] The terminal device 40 can be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server, and the terminal device 40 includes but is not limited to a processor 41 and a memory 42. Those skilled in the art will understand that Figure 4 It is merely an example of the terminal device 40 and does not constitute a limitation on the terminal device 40. The terminal device 40 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 40 may also include input and output devices, network access devices, buses, etc.

[0089] Among them, the processor 41 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.; the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0090] The memory 42 can be an internal storage unit of the terminal device 40, such as a hard disk or memory of the terminal device 40, or the memory 42 can be an external storage device of the terminal device 40, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 40; further, the memory 42 can also include both the internal storage unit of the terminal device 40 and the external storage device, and the memory 42 can also store the computer program 43 and other programs and data required by the terminal device 40, and the memory 42 can also be used to temporarily store data that has been output or is to be output.

[0091] One embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, which, when executed by a processor, can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form; the computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium.

[0092] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the methods, principles, and structures of the present application should be included in the scope of protection of the present application.

Claims

1. A wearable device offline scene interaction method based on acoustic wave carrier information, characterized in that: Applied to a wearable device, the method includes: Based on the preset microphone array, obtain environmental sound wave information; Based on differential phase shift keying technology, the environmental sound wave information is modulated to generate carrier data information; Determining multi-dimensional scene information based on the carrier data information; The multi-dimensional scene information is sent to a receiving device of the wearable device, wherein the receiving device includes a speaker or a display screen.

2. The method according to claim 1, characterized in that After obtaining the ambient sound wave information based on the preset microphone array, the method further includes: Based on a preset adaptive filtering algorithm, the ambient sound wave information is subjected to noise removal processing to generate noise-reduced sound wave information; Accordingly, the ambient sound wave information is modulated based on the differential phase shift keying technology to generate carrier data information, including: Based on the differential phase shift keying technology, the noise reduction carrier wave information is modulated to generate carrier wave data information.

3. The method according to claim 2, characterized in that After performing noise removal processing on the ambient sound wave information based on the preset adaptive filtering algorithm to generate noise-reduced sound wave information, the method further includes: Performing Hanning window Fourier transform processing on the noise-reduced acoustic wave information to obtain acoustic wave spectrum information; Based on a preset Gezil algorithm, characteristic frequency identification processing is performed on the sound wave spectrum information to determine target frequency information; Based on a preset least mean square adaptive filter, channel equalization processing is performed on the target frequency information to generate optimized signal information; Based on a preset Reed-Solomon decoding algorithm, error correction processing is performed on the optimized signal information to generate corrected signal information; Performing a CRC check on the correction signal information; If the CRC check passes, the differential phase shift keying technology is continued to be executed to modulate the noise reduction wave information to generate carrier data information.

4. The method according to claim 1, wherein The environmental sound wave information includes unique ID code information; and determining multi-dimensional scene information based on the carrier data information includes: Multi-dimensional scene information is determined based on the unique ID code information in the carrier data information, wherein the multi-dimensional scene information includes three-dimensional display model information, price discount information, product inventory information, navigation data information and cultural relics explanation audio and video information.

5. The method according to claim 1, wherein The frequency response range of the microphone array is 15kHz to 19kHz.

6. A wearable device offline scene interaction system based on acoustic carrier information, characterized by Applied to a wearable device, the system includes: Ambient sound wave information acquisition module: used to obtain ambient sound wave information based on a preset microphone array; Carrier data information generation module: used to modulate the environmental sound wave information based on differential phase shift keying technology to generate carrier data information; A multi-dimensional scene information determination module: configured to determine multi-dimensional scene information based on the carrier data information; Multi-dimensional scene information sending module: used to send the multi-dimensional scene information to the receiving device of the wearable device, wherein the receiving device includes a speaker or a display screen.

7. The system according to claim 6, characterized in that The system further comprises: The noise reduction wave information generation module is used to perform noise removal processing on the environmental sound wave information based on a preset adaptive filtering algorithm to generate noise reduction wave information.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images