Space positioning method based on audio time delay, electronic device and storage medium

Abstract

A spatial positioning method based on audio time delay is applied to an electronic device, and comprises the following steps: enabling a virtual reality (VR) device to be connected with a plurality of loudspeakers in a pairing manner; calculating a plurality of audio time delays from the VR device to the loudspeakers; according to the audio time delays, current positions of the loudspeakers relative to the VR device are calculated, and a virtual boundary is set; defining a plurality of critical values of the loudspeakers relative to the virtual boundary; judging whether the VR device approaches or exceeds the virtual boundary according to the critical values; and if the VR device approaches or exceeds the virtual boundary, making the VR device give an alarm.

Description

Technical Field

[0001] The present invention relates to time delay, and in particular to a spatial positioning method, an electronic device, and a storage medium based on audio time delay. Background Art

[0002] Data Transmission technology has been quite mature in audio transmission. However, with the development of different audio applications, the need for audio synchronization has become more and more. Therefore, the accuracy of audio latency has become quite important.

[0003] In applications of virtual reality (VR), users cannot know their positions in the real space, which may cause collisions with audio devices when moving. Summary of the Invention

[0004] In view of the above, it is necessary to provide a spatial positioning method, an electronic device, and a storage medium based on audio time delay, which use active sound signals to calculate so that the VR device can calculate the user's position by the audio device, and dynamically remind the user of approaching the boundary of the VR space to avoid collisions with the audio device.

[0005] An embodiment of the present invention provides a spatial positioning method based on audio time delay, which is applied to an electronic device and includes: pairing and connecting a virtual reality (VR) device with a plurality of speakers; calculating a plurality of audio time delays from the VR device to the speakers; calculating the current positions of the speakers relative to the VR device according to the audio time delays and setting virtual boundaries; defining a plurality of critical values of the speakers relative to the virtual boundaries; determining whether the VR device is approaching or exceeding the virtual boundaries according to the critical values; and if the VR device is approaching or exceeding the virtual boundaries, causing the VR device to issue a warning.

[0006] An embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a spatial positioning program based on audio time delay stored in the memory and executable on the processor. The electronic device further comprises a pairing module, a calculation and control module, and an alarm module. When the spatial positioning program based on audio time delay is executed by the processor, the following steps are implemented: pairing a VR device with a plurality of speakers; calculating a plurality of audio time delays from the VR device to the speakers; deducing the current positions of the speakers relative to the VR device based on the audio time delays and setting a virtual boundary; defining a plurality of critical values of the speakers relative to the virtual boundary; determining whether the VR device is approaching or exceeding the virtual boundary based on the critical values; and causing the VR device to issue an alarm if the VR device is approaching or exceeding the virtual boundary.

[0007] An embodiment of the present invention further provides a storage medium having a computer program stored thereon. When the computer program is executed, the steps of the aforementioned spatial positioning method based on audio time delay are implemented.

[0008] The present invention discloses a spatial positioning method, electronic device, and storage medium based on audio time delay. The method calculates the transmission latency and sound arrival time from a virtual reality (VR) device to a home speaker via an audio path. This method can be used with wireless speakers of various brands. Using a VR equipped with a microphone, the method can calculate the transmission delay between wireless speakers of different brands and the sound path length at the current wireless speaker location, allowing the VR's precise spatial location to be determined using triangulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 FIG. 4 is a flowchart of a spatial positioning method based on audio time delay according to an embodiment of the present invention.

[0010] Figure 2 FIG. 4 is a schematic diagram of a correction system based on audio time delay according to an embodiment of the present invention.

[0011] Figure 3 FIG. 4 is a schematic diagram of a spatial positioning system based on audio time delay according to an embodiment of the present invention.

[0012] Figure 4 FIG. 4 is a schematic diagram of a spatial positioning application based on audio time delay according to a first embodiment of the present invention.

[0013] Figure 5 FIG. 4 is a schematic diagram of a spatial positioning application based on audio time delay according to a second embodiment of the present invention.

[0014] Figure 6 FIG. 4 is a schematic diagram of the hardware architecture of an electronic device according to an embodiment of the present invention.

[0015] Figure 7 It is a functional block diagram of the electronic device according to an embodiment of the present invention.

[0016] Description of main element symbols

[0017]

[0018] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. Specific embodiments

[0019] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.

[0020] In the following description, many specific details are set forth in order to fully understand the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0022] It should be noted that the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the said features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement it. When the combination of technical solutions appears to be contradictory or unable to be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0023] The spatial positioning method based on audio time delay according to an embodiment of the present invention calculates the transmission delay (latency) from a virtual reality (VR) device to a home speaker and the sound arrival time through a sound path, and can be used with wireless speakers of different brands. Through the VR equipped with a microphone, the transmission delay between wireless speakers of different brands and the sound path length at the current position of the wireless speaker can be calculated, and the accurate spatial position of the VR can be calculated by triangulation.

[0024] Figure 1 FIG. 4 is a flowchart of steps of the spatial positioning method based on audio time delay according to an embodiment of the present invention, which is applied to an electronic device, and the electronic device is a VR device. According to different requirements, the order of steps in the flowchart can be changed, and some steps can be omitted.

[0025] Step S10, the VR device is paired and connected with multiple speakers.

[0026] Step S20, calculate multiple audio time delays from the VR device to the speakers.

[0027] Figure 2 FIG. 5 is a schematic diagram of an audio time delay correction system according to an embodiment of the present invention. Refer to Figure 2 , the audio time delay correction system 100 according to an embodiment of the present invention includes a VR device 110, a Wi-Fi router 130, and a speaker 150 (for example, Figure 3 in S1 of FIG. 4). The VR device 110 further includes a Wi-Fi chip 111, a micro control unit (MCU) 113, a coder-decoder (Codec) 115, a microphone 117, and a speaker 119. The speaker 150 further includes a Wi-Fi chip 151, a micro control unit 153, a coder-decoder 155, and a speaker 159.

[0028] T1 refers to the time when the micro control unit 113 transmits an audio signal to the Wi-Fi router 130. T2 refers to the time when the Wi-Fi router 130 transmits the audio signal to the speaker 150 and plays it with the speaker 159. T3 refers to the time when the speaker 159 transmits the audio signal to the microphone 117. T3' refers to the time when the speaker 119 transmits the audio signal to the microphone 117. T4 refers to the time when the micro control unit 113 obtains the audio signal from the microphone 117. T5 refers to the time when the micro control unit 113 transmits the audio signal to the speaker 119 through the coder-decoder 115 for playback.

[0029] The VR device 110 transmits an audio signal to the speaker 150 via the Wi-Fi router 130, and the time delay T1+T2 for the VR device 110 to transmit to the speaker 150 can be obtained. The speaker 150 sends the audio signal to the microphone 117 of the VR device 110, and the audio time delay T3+T4 can be obtained. The audio time delay from when the VR device 110 sends out the audio signal to the speaker 150, through the speaker 150's speaker 159 and received by the codec 115 of the VR device 110 is TL = T1+T2+T3+T4, where the speaker is a known brand, and TL, T1, T2, and T4 are all known. The VR device 110 sends the audio signal to its own microphone 117 for reception, and the audio time delay of the VR device 110 itself can be obtained as TL int = T3'+T4+T5, where T5 is known.

[0030] When the VR device 110 and the speaker 150 are placed at a preset interval distance, it means ., and TL - TLint = (T1+T2+T3+T4)-(T3'+T4+T5) = T1+T2+T3 - T3' - T5 can be obtained. T3 - T3' = TL - TLint - (T1+T2) - T5, where T3' is extremely small and can be ignored. Therefore, the audio time delay T3 between the VR device 110 and the speaker 150 can be obtained.

[0031] Figure 3 is a schematic diagram of the spatial positioning system based on audio time delay according to an embodiment of the present invention, which includes 1 VR device and 4 speakers (S1, S2, S3, S4), and the speakers S1, S2, S3, and S4 can form a spatial area (for example, spatial area 10). Through the above operations, the audio time delays T3-1, T3-2, T3-3, and T3-4 between the VR device (VR) and the speakers S1, S2, S3, and S4 can be obtained.

[0032] Step S30, refer to Figure 4 ., and calculate the current positions of the speakers S1, S2, S3, and S4 relative to the VR device according to the audio time delays T3-1, T3-2, T3-3, and T3-4 and set a virtual boundary 20. Additionally, when the VR device moves, new audio time delays T3-1', T3-2', T3-3', and T3-4' can be calculated, as shown in Figure 3 shown.

[0033] Refer to Figure 4 ., the spatial positioning method based on audio time delay according to an embodiment of the present invention uses the audio time delay T3 to calculate the position change in the VR space, through a set of observation values (Observation Value) (TOV1 , T OV2 , T OV3 , T OV4 ) respectively correspond to audio time delays T3-1, T3-2, T3-3, and T3-4 and represent the moving distances of the VR device relative to speakers S1, S2, S3, and S4.

[0034] For example, when the VR device moves from the center point of the virtual boundary 20 to the front of speaker S2, the observed values (T3L2, 0, T3L3, T3L1) can be obtained. T OV1 = T3L2, indicating that the VR device moves a length of L2 to reach speaker S2. T OV2 = 0, indicating that the distance between the VR device and speaker S2 is close to 0. T OV3 = T3L3, indicating that the VR device moves a length of L3 from the center point of the virtual boundary 20 to reach speaker S2. T OV4 = T3L1, indicating that the VR device moves a length of L1 to reach speaker S2.

[0035] Step S40, refer to Figure 4 , define the threshold values (Threshold Value, TV) of speakers S1, S2, S3, and S4 relative to the virtual boundary 20, including T TV1 , T TV2 , T TV3 and T TV4 .

[0036] Step S50, according to the threshold values T TV1 , T TV2 , T TV3 and T TV4 judge whether the VR device is close to or exceeds the virtual boundary 20.

[0037] Step S60, if the VR device is close to or exceeds the virtual boundary 20, the VR device issues a warning to let the user know that the current position is close to or exceeds the virtual boundary 20, improving the safety of VR users.

[0038] In addition, refer to Figure 5 , by setting the threshold values T TV1 , T TV2 , T TV3 and T TV4 the range of the virtual boundary can be reduced. For example, from the virtual boundary 20 to the virtual boundary 30.

[0039] Figure 6It is a schematic diagram of the hardware architecture of the electronic device according to an embodiment of the present invention. The electronic device 200, for example, a VR device, but not limited to, can communicate with each other through a system bus to connect a processor 210, a memory 220, and a spatial positioning system 230 based on audio time delay. Figure 6 Only the electronic device 200 with components 210 - 230 is shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0040] The memory 220 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card - type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read - only memory (ROM), electrically erasable programmable read - only memory (EEPROM), programmable read - only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 220 can be an internal storage unit of the electronic device 200, such as the hard disk or memory of the electronic device 200. In other embodiments, the memory can also be an external storage device of the electronic device 200, such as a plug - in hard disk equipped on the electronic device 200, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Of course, the memory 220 can also include both the internal storage unit and the external storage device of the electronic device 200. In this embodiment, the memory 220 is generally used to store the operating system and various application software installed on the electronic device 200, such as the program code of the spatial positioning system 230 based on audio time delay. In addition, the memory 220 can also be used to temporarily store various data that have been output or will be output.

[0041] In some embodiments, the processor 210 can be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data - processing chips. The processor 210 is generally used to control the overall operation of the electronic device 200. In this embodiment, the processor 210 is used to run the program code stored in the memory 220 or process data, for example, run the spatial positioning system 230 based on audio time delay.

[0042] It should be noted that Figure 6 This is only an example to illustrate the electronic device 200. In other embodiments, the electronic device 200 can also include more or fewer components, or have a different component configuration.

[0043] If the modules / units integrated in the electronic device 200 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0044] Figure 7 It is a functional block diagram of the electronic device according to an embodiment of the present invention, which is used to execute a spatial positioning method based on audio time delay. The spatial positioning method based on audio time delay according to an embodiment of the present invention can be implemented by a computer program in a storage medium, for example, the memory 220 in the electronic device 200. When the computer program implementing the method of the present invention is loaded by the processor 210 into the memory 220, it drives the processor 210 of the line device 200 to execute the spatial positioning method based on audio time delay according to an embodiment of the present invention.

[0045] The electronic device 200 according to an embodiment of the present invention, for example, a VR device, includes a pairing module 310, a calculation and control module 320, and a warning module 330.

[0046] The pairing module 310 pairs and connects the VR device with a plurality of speakers (for example, S1, S2, S3, and S4).

[0047] The calculation and control module 320 calculates a plurality of audio time delays (for example, T3-1, T3-2, T3-3, and T3-4) from the VR device to the speakers, as Figure 3 shown.

[0048] The calculation and control module 320 calculates the current positions of the speakers S1, S2, S3, and S4 relative to the VR device based on the audio time delays T3-1, T3-2, T3-3, and T3-4, and sets a virtual boundary (e.g., virtual boundary 20). Additionally, when the VR device moves, the calculation and control module 320 can calculate new audio time delays, for example, T3-1′, T3-2′, T3-3′, and T3-4′, as Figure 3 shown.

[0049] The calculation and control module 320 defines threshold values of the speakers S1, S2, S3, and S4 relative to the virtual boundary 20, for example, T TV1 , T TV2 , T TV3 and T TV4 , as Figure 4 shown.

[0050] The calculation and control module 320 determines whether the VR device is approaching or exceeding the virtual boundary 20 based on the threshold values T TV1 , T TV2 , T TV3 and T TV4 .

[0051] If the VR device is approaching or exceeding the virtual boundary 20, the warning module 330 issues a warning to let the user know that the current position is approaching or exceeding the virtual boundary 20, enhancing the safety of VR users.

[0052] It can be understood that the above-described module division is only a logical function division, and there may be other division methods in actual implementation. Additionally, in each embodiment of the present application, the functional modules can be integrated in the same processing unit, or each module can exist physically alone, or two or more modules can be integrated in the same unit. The above-integrated modules can be implemented in the form of hardware or in the form of a combination of hardware and software functional modules.

[0053] For those of ordinary skill in the art, other corresponding changes or adjustments can be made according to the actual needs generated by combining the technical solutions and technical concepts provided by the embodiments of the present invention, and these changes and adjustments should fall within the protection scope of the claims of the present invention.

Claims

1. A spatial positioning method based on audio time delay, which is applied to an electronic device, and is characterized in that The method includes: Pairing and connecting a virtual reality (VR) device with a plurality of speakers; Calculating a plurality of audio time delays from the VR device to the speakers; Deducing the current positions of the speakers relative to the VR device based on the audio time delays and setting a virtual boundary; Defining a plurality of threshold values of the speakers relative to the virtual boundary; Judging whether the VR device approaches or exceeds the virtual boundary according to the threshold values; and If the VR device approaches or exceeds the virtual boundary, causing the VR device to issue a warning.

2. The spatial positioning method based on audio time delay according to claim 1, wherein, It further includes: Changing the setting of the threshold values to adjust the size of the range of the virtual boundary.

3. An electronic device, characterized in that, The electronic device includes a memory, a processor, and a spatial positioning program based on audio time delay stored on the memory and executable on the processor. The electronic device further includes a pairing module, a calculation and control module, and a warning module. When the spatial positioning program based on audio time delay is executed by the processor, the following steps are implemented: Pairing and connecting a VR device with a plurality of speakers; Calculating a plurality of audio time delays from the VR device to the speakers; Deducing the current positions of the speakers relative to the VR device based on the audio time delays and setting a virtual boundary; Defining a plurality of threshold values of the speakers relative to the virtual boundary; Judging whether the VR device approaches or exceeds the virtual boundary according to the threshold values; and If the VR device approaches or exceeds the virtual boundary, causing the VR device to issue a warning.

4. The electronic device according to claim 5, characterized in that, When the spatial positioning program based on audio time delay is executed by the processor, the following steps are further implemented:: Changing the setting of the threshold values to adjust the size of the range of the virtual boundary.

5. A storage medium having at least one computer instruction stored thereon, characterized in that, The instruction is executed by the processor and loaded to execute the spatial positioning method based on audio time delay as described in any one of claims 1-2.

Images