Information interaction method, device, system, wearable device and readable storage medium

Abstract

The present disclosure relates to the field of computer processing technology, and specifically to information interaction methods, devices, systems, wearable devices and readable storage media, the method comprising receiving beam parameters sent by a paired Bluetooth device; obtaining an audio signal to be processed in the current environment; performing beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal; and sending the target audio signal to the paired Bluetooth device for processing to obtain and output the processing result of the target audio signal. Since the beam parameters are sent before the audio signal to be processed in the current environment is processed, they can adapt to the current environment, making the method applicable to different application scenarios; at the same time, beam screening of the audio signal to be processed can reduce the amount of transmitted data, realize information interaction using only Bluetooth, without introducing other communication units, and reduce the weight of the wearable device to which the method is applied.

Description

Technical Field

[0001] The present disclosure relates to the field of computer processing technology, and in particular to an information interaction method, apparatus, system, wearable device, and readable storage medium. Background Art

[0002] For various types of wearable devices, it is usually necessary to upload the collected data to the terminal for analysis and processing. In order to ensure the normal transmission of data, wearable devices are generally connected to the terminal using wired or WIFI methods. In this connection mode, the wearable device uploads all the collected data to the terminal to meet the data requirements in different application scenarios.

[0003] If all data is uploaded, it will inevitably lead to greater power consumption. Therefore, a large-capacity battery is usually required in the wearable device to ensure normal working requirements. However, the setting of a large-capacity battery will make the wearable device heavier. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide an information interaction method, apparatus, system, wearable device and readable storage medium to solve the problem of heavy weight of wearable devices caused by data requirements in different application scenarios.

[0005] According to a first aspect, an embodiment of the present disclosure provides an information interaction method, including:

[0006] Receive beam parameters sent by paired Bluetooth devices;

[0007] Get the audio signal to be processed in the current environment;

[0008] Performing beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal;

[0009] The target audio signal is sent to the paired Bluetooth device for processing to obtain and output a processing result of the target audio signal.

[0010] According to a second aspect, an embodiment of the present disclosure provides an information interaction device, including:

[0011] A receiving module, used to receive beam parameters sent by a paired Bluetooth device;

[0012] An acquisition module is used to acquire the audio signal to be processed in the current environment;

[0013] A screening module, configured to perform beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal;

[0014] The output module is used to send the target audio signal to the paired Bluetooth device for processing, so as to obtain and output the processing result of the target audio signal.

[0015] According to a third aspect, an embodiment of the present disclosure provides a wearable device, including:

[0016] A Bluetooth communication unit, used to receive beam parameters sent by a paired Bluetooth device and send them to a processor;

[0017] An audio acquisition unit, used for acquiring the audio signal to be processed in the current environment and sending it to the processor;

[0018] The processor is used to perform beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal, and the Bluetooth communication unit is further used to send the target audio signal to the paired Bluetooth device for processing to obtain a processing result of the target audio signal;

[0019] An output unit is used to output the processing result of the target audio signal.

[0020] According to a fourth aspect, an embodiment of the present disclosure further provides an information interaction system, including:

[0021] The wearable device described in the third aspect of the present disclosure or any embodiment of the third aspect;

[0022] A Bluetooth device is used to pair with the wearable device, the Bluetooth device is used to send beam parameters to the wearable device, and receive the target audio signal sent by the wearable device to obtain a processing result of the target audio signal, and send the processing result to the wearable device.

[0023] According to the fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the information interaction method described in the first aspect or any one of the embodiments of the first aspect.

[0024] The information interaction method provided by the embodiment of the present disclosure first receives the beam parameters sent by the paired Bluetooth device before processing the audio signal to be processed. Since the beam parameters are sent before the audio signal to be processed in the current environment is processed, they can adapt to the current environment, so that the method can be applicable to different application scenarios. At the same time, beam screening is performed on the audio signal to be processed, which can reduce the amount of transmitted data, thereby reducing power consumption and realizing information interaction under low power consumption. Therefore, information interaction can be carried out only by Bluetooth without introducing other communication units, thereby reducing the weight of the wearable device using the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 is a structural diagram of an information interaction system according to an embodiment of the present disclosure;

[0027] Figure 2 is a structural block diagram of a wearable device according to an embodiment of the present disclosure;

[0028] Figure 3 is a schematic diagram of the hardware structure of a wearable device according to an embodiment of the present disclosure;

[0029] Figure 4 is a hardware structure block diagram of AR glasses according to an embodiment of the present disclosure;

[0030] Figure 5 is a flow chart of an information interaction method according to an embodiment of the present disclosure;

[0031] Figure 6 is a flow chart of an information interaction method according to an embodiment of the present disclosure;

[0032] Figure 7 is a schematic diagram of an interface of a paired Bluetooth device according to an embodiment of the present disclosure;

[0033] Figure 8 is a flow chart of an information interaction method according to an embodiment of the present disclosure;

[0034] Fig. 9 is a schematic diagram of multi-path signal separation according to an embodiment of the present disclosure;

[0035] Fig.10 is a schematic diagram of an application scenario according to an embodiment of the present disclosure;

[0036] Fig.11 It is a structural block diagram of an information interaction device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0038] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0039] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0040] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0041] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0043] In order to adapt to the data requirements in different application scenarios, wearable devices will upload all collected data to the terminal so that the terminal can analyze the data according to the requirements. Taking virtual reality (VR) glasses as an example, VR glasses will send all audio data collected in the current environment to the mobile phone, so that the mobile phone can analyze the audio data and feed back the analysis results to the VR glasses. Due to the large amount of data that needs to be sent, VR glasses generally use WIFI wireless connection or USB wired connection; at the same time, in order to support the high power consumption caused by the transmission of large amounts of data, a large-capacity battery needs to be set in the VR glasses. Therefore, this setting method will cause the VR glasses to be very heavy.

[0044] Based on this, the information interaction method provided by the embodiment of the present disclosure, in order to adapt to the data requirements in different scenarios, first uses the paired Bluetooth device to set the beam parameters, and the wearable device then performs beam screening on the acquired audio signal to be processed based on the beam parameters to obtain the target audio signal; and then sends the target audio signal to the paired Bluetooth device to obtain the processing result. In this method, the beam signal required in the current environment is screened out by setting the beam parameters, so that the method can be applied to different application scenarios; and because the beam parameters are used to screen the beam, the amount of data of the target audio signal obtained after screening is reduced, and stable data transmission can be performed via Bluetooth. Since the amount of data transmitted is reduced, the power consumption caused by data interaction is reduced. Therefore, a smaller capacity battery can be set in the wearable device, thereby reducing the weight of the wearable device.

[0045] The present disclosure provides an information interaction system. Figure 1 As shown, the system includes a wearable device 1 and a Bluetooth device 2, wherein the wearable device 1 is used to connect to the Bluetooth device 2 via Bluetooth pairing. The wearable device 1 may be a device with Bluetooth transmission function such as AR glasses and smart bracelets, and its specific form is not limited here, and it is set according to actual needs. The Bluetooth device 2 may be a mobile phone, a tablet or other terminal with Bluetooth function. After the wearable device 1 starts working, it is paired with the Bluetooth device 2. If the pairing is successful, the information is exchanged with the Bluetooth device 2. For example, after the pairing is successful, the Bluetooth device 2 sets the beam parameters of the wearable device 1 so that the wearable device 1 performs beam filtering on the acquired audio signal to be processed based on the beam parameters to obtain the demand for the audio signal of the target beam in the current environment. After the wearable device 1 performs beam screening on the audio signal to be processed, the target audio signal is obtained, and the target audio signal is sent to the paired Bluetooth device 2 via Bluetooth for processing. The Bluetooth device 2 may directly analyze and process the target audio signal, or send the target audio signal to the background server for processing by the background server, etc., and there is no limitation on it here. The Bluetooth device 2 sends the processing result of the target audio signal to the wearable device 1 via Bluetooth, and accordingly, the wearable device 1 outputs the processing result. The output method includes but is not limited to voice playback or image display.

[0046] In some optional embodiments, the information interaction system also includes a server connected to the Bluetooth device 2, the server is used to receive the target audio signal sent by the Bluetooth device, identify the target audio signal to determine the processing result of the target audio signal, and send the processing result to the Bluetooth device 2. Among them, the recognition method of the target audio signal includes but is not limited to automatic speech recognition, natural language processing, from text to speech, etc. For example, the user sends a voice "Today's weather" to the wearable device, and the wearable device may also collect other audio in the current environment when collecting the voice. The wearable device processes the collected audio signal using the received beam parameters to obtain the target audio signal, and sends the target audio signal to the Bluetooth device, and the Bluetooth device sends the target audio signal to the server. Among them, the processing of the collected audio signal by the wearable device includes but is not limited to multi-microphone noise reduction, beamforming, sound source localization, speaker separation or dynamic gain control, etc. After receiving the target audio signal, the server first converts the audio signal into text, then performs natural language processing to understand its meaning, and then converts the retrieved result into an audio signal as the processing result of the target audio signal and sends it to the Bluetooth device. The Bluetooth device sends the processing result to the wearable device, and the wearable device plays the processing result of "Today's Weather" through the playback unit.

[0047] By placing the analysis and processing of the target audio signal on the server, the Bluetooth device only plays the role of intermediate data transmission and beam parameter setting. For the Bluetooth device, the amount of data transmitted is very small, and the data transmission will not lead to more power consumption reduction. Therefore, this method can also reduce the power consumption of Bluetooth devices and improve the battery life of Bluetooth devices.

[0048] The present disclosure also provides a wearable device, such as Figure 2 As shown, the wearable device includes a Bluetooth communication unit 11, an audio acquisition unit 12, a processor 13 and an output unit 14. Among them, the Bluetooth device that is successfully paired with the wearable device is referred to as a paired Bluetooth device. The Bluetooth communication unit 11 is used to receive the beam parameters sent by the paired Bluetooth device and send them to the processor 13, the audio acquisition unit 12 is used to collect the audio signal to be processed in the current environment and send it to the processor 13, the processor is used to perform beam screening on the audio signal to be processed based on the beam parameters to obtain the target audio signal, the target audio signal is sent to the paired Bluetooth device through the Bluetooth communication unit to obtain the processing result of the target audio signal, and the output unit is used to output the processing result of the target audio signal. Furthermore, the communication unit in the wearable device only includes the Bluetooth communication unit 11, that is, the wearable device only interacts with information through Bluetooth.

[0049] Specifically, the audio acquisition unit 12 may be a microphone array or other audio acquisition devices. When the audio acquisition unit 12 is a microphone array, there is no limitation on the arrangement of the microphones in the microphone array. The output unit 14 includes audio output, video output, and the like. Taking AR glasses as an example, the output unit 14 includes the lenses of the AR glasses. Of course, a microphone output unit may also be provided on the AR glasses.

[0050] In some embodiments, the wearable device further includes a power supply unit, which includes a battery and a power conversion chip, wherein the power conversion chip is used to convert the battery voltage into a voltage required for each processor to work, or is also used to charge the battery.

[0051] See also Figure 3 , Figure 3 is a schematic diagram of the hardware structure of a wearable device provided by an optional embodiment of the present disclosure, Figure 3 Only the processor 13, the communication bus 15, the communication interface 16 and the memory 17 of the wearable device are shown. Specifically, Figure 3 As shown, the wearable device may include: at least one processor 13, such as a CPU (Central Processing Unit), at least one communication interface 16, a memory 17, and at least one communication bus 15. The communication bus 15 is used to realize the connection and communication between these components. The communication interface 16 may include a display screen (Display) and a keyboard (Keyboard), and the optional communication interface 16 may also include a standard wired interface and a wireless interface. The memory 17 may be a high-speed RAM memory (Random Access Memory) or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 17 may optionally be at least one storage device located away from the aforementioned processor 13. The processor 13 may be combined with Fig.11 In the described device, the memory 17 stores an application program, and the processor 13 calls the program code stored in the memory 17 to execute any of the above method steps.

[0052] The communication bus 15 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 15 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0053] Among them, the memory 17 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 17 may also include a combination of the above types of memory.

[0054] The processor 13 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0055] The processor 13 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0056] Optionally, the memory 17 is also used to store program instructions. The processor 13 can call the program instructions to implement the information interaction method shown in any embodiment of the present application.

[0057] Taking AR glasses as an example, Figure 4The hardware structure block diagram of the AR glasses is shown. The AR glasses include a processor, an audio acquisition unit, a Bluetooth communication unit, a display unit, a playback unit, and a power supply unit. Among them, the display unit and the playback unit are both output units, the display unit is used to output images or videos; the playback unit is used to play audio; the Bluetooth communication unit is used to realize information interaction between the AR glasses and the paired Bluetooth devices. The control unit includes a switch or a case, etc., which is used to realize the functions of turning on and off the AR glasses, turning on and off the display, or turning on and off the playback. The Bluetooth communication unit can be a Bluetooth low-power unit or a Bluetooth unit.

[0058] The AR glasses achieve end-to-end information interaction through a Bluetooth communication unit. Since the beam parameters of the AR glasses are set using a paired Bluetooth device, the amount of data transmitted can be reduced, and the power consumption is low, so the demand for battery capacity is small, which reduces the weight of the AR glasses. Since the AR glasses are paired with Bluetooth devices using Bluetooth, that is, wireless connection is used, the AR glasses are easy to wear and carry. And because Bluetooth connection is highly universal and simple to operate, it uses a universal interface for communication. For Bluetooth devices, the application development workload is small, and the interface can be universal.

[0059] According to an embodiment of the present disclosure, an information interaction method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] In this embodiment, an information interaction method is provided, which can be used for the above-mentioned wearable device. Figure 5 is a flow chart of an information interaction method according to an embodiment of the present disclosure, such as Figure 5 As shown, the process includes the following steps:

[0061] S11, receiving beam parameters sent by the paired Bluetooth device.

[0062] The beam parameters include the target beam direction or the gain of each beam direction, etc., which are used to filter out the audio signal in the target beam direction. The specific parameters are set according to actual needs. As mentioned above, the paired Bluetooth device is a Bluetooth device that is successfully paired with the wearable device. After the pairing is successful, the paired Bluetooth device sets the beam parameters based on the needs of the current environment and sends the beam parameters to the wearable device.

[0063] The setting of the beam parameters may be determined by the paired Bluetooth device after analyzing the audio signal of the current environment, or may be set by the paired Bluetooth device through interaction with the user, etc. The specific method for determining the beam parameters is set according to actual needs.

[0064] S12, obtaining an audio signal to be processed in the current environment.

[0065] The audio acquisition unit of the wearable device acquires the audio signal in the current environment, wherein the acquired original audio signal can be used as the audio signal to be processed, or the original audio signal can be first subjected to speaker separation, dynamic gain control, etc., to separate the original audio signal into multiple audio signals, which are used as the audio signal to be processed.

[0066] For example, the audio acquisition unit in the wearable device first uses a microphone array to locate the sound source and collect the audio of the speaker. Then, based on the results of the sound source localization, it performs gain processing on the audio signal in the direction of the speaker to enhance the audio signal in the direction of the speaker and reduce the audio signals in other directions.

[0067] S13, performing beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal.

[0068] The wearable device uses the beam parameters to perform beam screening on the audio signal to be processed, and screens out the audio signal in the target beam direction. Before beam screening, the wearable device performs beamforming on the audio signal to be processed, and then uses the beam parameters to perform beam screening on the beamformed audio signal to obtain the target audio signal. For example, the audio signal in the main lobe direction is screened out from the audio signal to be processed to obtain the target audio signal.

[0069] The wearable device is provided with a beamforming model, which is constructed based on a neural network model and is used to extract the audio signal in a specified direction to obtain the target audio signal. For example, the masks of speech and noise are estimated by the nearest neighbor of the discontinuous density function, and then the power spectrum of the noise is estimated based on the mask. Then, based on this, the noise is suppressed by using beamforming algorithms such as minimum mean square distortion-free response or generalized sidelobe clipping based on the beam parameters, thereby obtaining the target audio signal. Alternatively, the weights of the beamforming are directly predicted by the neural network, and the weights predicted by the neural network are multiplied by the complex spectra of multiple channels and then summed to obtain the beamforming result in the frequency domain. The beamforming result is screened by the beam parameters, and the time domain waveform is obtained by the short-time inverse Fourier transform, that is, the target audio signal is obtained.

[0070] S14, sending the target audio signal to the paired Bluetooth device for processing, so as to obtain and output the processing result of the target audio signal.

[0071] The wearable device sends the target audio signal to the paired Bluetooth device for processing by the paired Bluetooth device. As described above, the processing of the target audio signal can be directly processed by the paired Bluetooth device, or sent by the paired Bluetooth device to a server, which is then processed and sent to the paired Bluetooth device. The paired Bluetooth device sends the processing result of the target audio signal to the wearable device, and the wearable device outputs the processing result of the target audio signal using an output unit.

[0072] The information interaction method provided in this embodiment first receives the beam parameters sent by the paired Bluetooth device before processing the audio signal to be processed. Since the beam parameters are sent before the audio signal to be processed in the current environment is processed, they can adapt to the current environment, so that the method can be applicable to different application scenarios; at the same time, beam screening of the audio signal to be processed can reduce the amount of transmitted data, thereby reducing power consumption and realizing information interaction with low power consumption. Therefore, information interaction can be carried out only by Bluetooth without introducing other communication units, thereby reducing the weight of the wearable device to which the method is applied.

[0073] In this embodiment, an information interaction method is provided, which can be used for the above-mentioned wearable device. Figure 6 is a flow chart of an information interaction method according to an embodiment of the present disclosure, such as Figure 6 As shown, the process includes the following steps:

[0074] S21, receiving beam parameters sent by the paired Bluetooth device.

[0075] Specifically, the above S21 includes:

[0076] S211, receiving a start signal for information interaction, and searching for a pairable Bluetooth device.

[0077] The wearable device is provided with a control unit, for example, a switch button. When the switch button is activated, the wearable device receives a start signal for information interaction and searches for a pairable Bluetooth device within a Bluetooth search range.

[0078] S212: Establish a connection with a pairable Bluetooth device and determine that the Bluetooth device has been paired.

[0079] When a pairable Bluetooth device is searched, pairing is performed with the Bluetooth device, and a Bluetooth connection with the pairable Bluetooth device is established to determine the paired Bluetooth device.

[0080] S213: Receive beam parameters sent by the paired Bluetooth device.

[0081] After pairing is successful, the paired Bluetooth device can send the beam parameters to the wearable device, or the paired Bluetooth device can obtain the beam parameters based on the test audio signal collected by the wearable device in the current environment.

[0082] In some implementations, the above S21 may further include:

[0083] (1) Obtain the test audio signal in the current environment.

[0084] (2) Sending a test audio signal to a paired Bluetooth device to obtain beam parameters determined by the paired Bluetooth device based on the test audio signal.

[0085] After the pairing is successful, the wearable device collects the test audio signal in the current environment to determine the application scenario in the current environment. The wearable device sends the test audio signal to the paired Bluetooth device to use the paired Bluetooth device to determine the beam parameters. The test audio signal is a real-time audio signal in the current environment.

[0086] For example, in the current environment where the wearable device user is located, multiple people are outputting audio, which may be playing music, chatting, playing audio, etc. The wearable device collects the real-time audio signal in the current environment to obtain a test audio signal, which is used to characterize the audio environment in the current environment.

[0087] Before processing the audio signal to be processed, a test audio signal in the current environment is first collected and sent to a paired Bluetooth device, so that the paired Bluetooth device can be used to analyze the test audio signal to accurately obtain the beam parameters in the current environment, thereby improving the matching degree between the beam parameters and the current environment, thereby better obtaining the speaker's audio signal and improving the accuracy of recognizing the audio signal to be processed.

[0088] In some optional implementations, the beam parameters in step (2) of S21 are determined by the paired Bluetooth device performing beam analysis on the test audio signal. Specifically, the paired Bluetooth device adaptively analyzes the test audio signal, for example, performs beamforming on the test audio signal, and the paired Bluetooth device analyzes the result of the beamforming to determine the main lobe direction, and uses it as the beam parameter to filter out the audio signal of the main speaker.

[0089] In some other optional implementations, the test audio signal is used to be displayed on an interface of a paired Bluetooth device, and the beam parameter is determined by the paired Bluetooth device in response to a beam selection operation on the test audio signal.

[0090] For example, the paired Bluetooth device performs beamforming on the test audio signal, and displays the beamforming result on the interface. Figure 7The result of beamforming is shown. The pairable Bluetooth device provides an interactive function. The user selects the main lobe and determines the direction of the target beam by interacting with the pairable Bluetooth device, which will be used as the beam parameter.

[0091] S22, obtaining an audio signal to be processed in the current environment.

[0092] For details, please see Figure 5 S12 of the illustrated embodiment will not be described in detail here.

[0093] S23, performing beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal.

[0094] For details, please see Figure 5 S13 of the illustrated embodiment will not be described in detail here.

[0095] S24, sending the target audio signal to the paired Bluetooth device for processing, so as to obtain and output the processing result of the target audio signal.

[0096] For details, please see Figure 5 S14 of the illustrated embodiment will not be described in detail here.

[0097] The information interaction method provided in this embodiment can obtain the beam parameters sent by the paired Bluetooth device after the information interaction is started, and realize the setting of beam parameters in different application scenarios.

[0098] In this embodiment, an information interaction method is provided, which can be used for the above-mentioned wearable device. Figure 8 is a flow chart of an information interaction method according to an embodiment of the present disclosure, such as Figure 8 As shown, the process includes the following steps:

[0099] S31, receiving beam parameters sent by a paired Bluetooth device.

[0100] For details, please see Figure 6 S21 of the illustrated embodiment will not be described in detail here.

[0101] S32, obtaining an audio signal to be processed in the current environment.

[0102] For details, please see Figure 5 S12 of the illustrated embodiment will not be described in detail here.

[0103] S33, performing beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal.

[0104] Specifically, the above S33 includes:

[0105] S331, performing speaker separation on the audio signal to be processed to obtain an audio signal corresponding to the speaker.

[0106] The wearable device determines the location of the speaker based on the sound source localization function in the audio acquisition unit. Accordingly, after the speaker is located, the speaker separation can be performed on the collected original audio signal, and the audio signal output by the speaker can be separated from the background audio signal to obtain the audio signal corresponding to the speaker. Among them, the audio signal corresponding to the speaker can be one channel, two channels or three channels, etc., depending on the number of speakers within a certain range of the wearable device in the current environment.

[0107] like Fig. 9 As shown, the wearable device uses a microphone array to collect three channels of original audio signals, and obtains the wearer's audio signal and other audio signals by performing speaker separation on them.

[0108] S332: Perform beam screening on the audio signal corresponding to the speaker based on the beam parameters to obtain a preset audio signal of the target beam.

[0109] Since the audio signal corresponding to the speaker has been separated in the above steps and the background audio has been removed, the wearable device uses the acquired beam parameters to perform beam screening on the audio signal corresponding to the speaker to obtain the preset audio signal of the target beam. In the above processing, since the speaker separation is performed first and then the beam screening is performed, the beam screened audio signal only includes the speaker's audio signal, thereby ensuring the accuracy of the preset audio signal obtained subsequently.

[0110] S333: compress the preset audio signal to obtain a target audio signal.

[0111] The wearable device compresses the preset audio signal of the target beam, for example, compresses it N times to obtain the target audio signal. The compression multiple can also be sent by the paired Bluetooth device to the wearable device. For example, after the paired Bluetooth device is successfully paired with the wearable device, it detects the current signal strength. If the signal strength is low, a larger compression multiple is set; if the signal strength is large, a smaller compression multiple is set. By adaptively adjusting the compression multiple, it is possible to ensure lossless transmission of data without loss, that is, after the preset audio signal is compressed N times, it can be stably transmitted through the Bluetooth communication unit without frame loss.

[0112] S34, sending the target audio signal to the paired Bluetooth device for processing, so as to obtain and output the processing result of the target audio signal.

[0113] When the processing result of the target audio signal includes an image or text, the image or text is displayed; when the processing result of the target audio signal includes audio information, the audio information is played.

[0114] For more details, please see Figure 5 S14 of the illustrated embodiment will not be described in detail here.

[0115] The information interaction method provided in this embodiment first separates the speakers of the audio signal to be processed to obtain the audio signals corresponding to each speaker, and then uses the beam parameters to perform beam screening to obtain the desired preset audio signal. On this basis, the preset audio signal is compressed, which can further reduce the amount of data transmitted using Bluetooth and reduce power consumption.

[0116] As a specific application example of the embodiment of the present disclosure, Fig.10 As shown, the wearable device is an AR glasses, and the Bluetooth device is a mobile phone. After the user puts on the AR glasses, Bluetooth pairing is started to determine the mobile phone that is successfully paired with the AR glasses. The AR glasses collect the test audio signal in the current environment, send the test audio signal to the mobile phone, and display the beamforming result corresponding to the test audio signal on the interface of the mobile phone. The user determines the target beam through interactive selection with the mobile phone, and accordingly determines the beam parameters. The mobile phone sends the determined beam parameters to the AR glasses. If the user outputs the voice "Query flight information from A to B" at this time, the AR glasses collect the voice information to obtain the audio signal to be processed, and process the audio signal to be processed based on the beam parameters to obtain the preset audio signal of the target beam; then compress the preset audio signal to obtain the target audio signal. The AR glasses send the target audio signal to the mobile phone, and the mobile phone sends it to the server after receiving the target audio signal, and the server analyzes the target audio signal. The server first converts the target audio signal into text, then performs natural language processing on the text to understand its meaning, searches for corresponding content based on the understood meaning, and finally represents the search results in the form of an image, that is, the processing result of the target audio signal is an image of "flight information from A to B". The server sends the image to the mobile phone, which transmits it to the AR glasses via Bluetooth, and the image of "flight information from A to B" is displayed on the lenses of the AR glasses.

[0117] As another specific application example of the embodiment of the present disclosure, the wearable device is a smart watch. The smart watch starts Bluetooth pairing with the mobile phone. If the pairing is successful, the mobile phone sets the beam parameters in the current environment, determines the compression multiple based on the signal strength of the Bluetooth connection, and sends the beam parameters and the compression multiple to the smart watch. The user outputs the voice "play today's news", and the smart watch collects the voice to obtain the audio signal to be processed. The smart watch performs speaker separation on the audio signal to be processed, and then performs beam filtering based on the beam parameters to obtain the preset audio signal of the target beam; then the preset audio signal is compressed using the compression multiple to obtain the target audio signal. The smart watch transmits the target audio signal to the mobile phone via Bluetooth, and the mobile phone sends it to the server. The server analyzes the target audio signal and understands its meaning as "play today's news". The server searches for today's news, compresses it to obtain the target audio, sends the target audio to the mobile phone, and the mobile phone sends it to the smart watch. Correspondingly, the smart watch plays today's news through voice.

[0118] In this embodiment, an information interaction device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0119] This embodiment provides an information interaction device, such as Fig.11 As shown, including:

[0120] A receiving module 41 is used to receive beam parameters sent by a paired Bluetooth device;

[0121] An acquisition module 42 is used to acquire an audio signal to be processed in a current environment;

[0122] A screening module 43, configured to perform beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal;

[0123] The output module 44 is used to send the target audio signal to the paired Bluetooth device for processing, so as to obtain and output the processing result of the target audio signal.

[0124] In some embodiments, the receiving module 41 includes:

[0125] A first receiving unit, configured to receive a start signal for information interaction and search for a pairable Bluetooth device;

[0126] A connecting unit, configured to establish a connection with the pairable Bluetooth device and determine the paired Bluetooth device;

[0127] The second receiving unit is used to receive the beam parameters sent by the paired Bluetooth device.

[0128] In some embodiments, the receiving module 41 includes:

[0129] An acquisition unit, used to acquire a test audio signal in the current environment;

[0130] A sending unit is used to send the test audio signal to the paired Bluetooth device to obtain the beam parameter determined by the paired Bluetooth device based on the test audio signal.

[0131] In some implementations, the beam parameters are determined by the paired Bluetooth device performing beam analysis on the test audio signal.

[0132] In some implementations, the test audio signal is used to be displayed on an interface of the paired Bluetooth device, and the beam parameter is determined by the paired Bluetooth device in response to a beam selection operation on the test audio signal.

[0133] In some embodiments, the screening module 43 includes:

[0134] A separation unit, configured to perform speaker separation on the audio signal to be processed to obtain an audio signal corresponding to the speaker;

[0135] A screening unit, configured to perform beam screening on the audio signal corresponding to the speaker based on the beam parameters to obtain a preset audio signal of the target beam;

[0136] The compression unit is used to compress the preset audio signal to obtain the target audio signal.

[0137] In some embodiments, the output module 44 includes:

[0138] A display unit, configured to display an image or text when the processing result of the target audio signal includes the image or text;

[0139] The playing unit is configured to play the audio information when the processing result of the target audio signal includes the audio information.

[0140] The information interaction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0141] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0142] The disclosed embodiment also provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the information interaction processing method in any of the above method embodiments. The storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0143] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0144] It can be understood that in the specific implementation of the present disclosure, related data such as user information and audio information are involved. When the above embodiments of the present disclosure are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0145] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An information interaction method, It is characterized in that include: Receive beam parameters sent by paired Bluetooth devices; Get the audio signal to be processed in the current environment; Performing beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal, including: performing beamforming on the audio signal to be processed, and then performing beam screening on the beamformed audio signal using the beam parameters to obtain the target audio signal; Sending the target audio signal to the paired Bluetooth device for processing to obtain and output a processing result of the target audio signal; The receiving of the beam parameters sent by the paired Bluetooth device includes: acquiring a test audio signal in the current environment; and sending the test audio signal to the paired Bluetooth device to obtain the beam parameters determined by the paired Bluetooth device based on the test audio signal.

2. The method according to claim 1, It is characterized in that The receiving of beam parameters sent by the paired Bluetooth device includes: Receive the start signal of information interaction and search for pairable Bluetooth devices; Establishing a connection with the pairable Bluetooth device and determining the paired Bluetooth device; Receive the beam parameters sent by the paired Bluetooth device.

3. The method according to claim 1, It is characterized in that The beam parameters are determined by the paired Bluetooth device performing beam analysis on the test audio signal.

4. The method according to claim 1, It is characterized in that The test audio signal is used to be displayed on the interface of the paired Bluetooth device, and the beam parameter is determined by the paired Bluetooth device in response to a beam selection operation on the test audio signal.

5. The method according to claim 1, It is characterized in that The step of performing beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal includes: Performing speaker separation on the audio signal to be processed to obtain an audio signal corresponding to the speaker; Performing beam screening on the audio signal corresponding to the speaker based on the beam parameters to obtain a preset audio signal of a target beam; The preset audio signal is compressed to obtain the target audio signal.

6. The method according to claim 1, It is characterized in that The obtaining and outputting the processing result of the target audio signal comprises: When the processing result of the target audio signal includes an image or text, displaying the image or text; When the processing result of the target audio signal includes audio information, the audio information is played.

7. An information interaction device, It is characterized in that include: A receiving module, used to receive beam parameters sent by a paired Bluetooth device; An acquisition module is used to acquire the audio signal to be processed in the current environment; A screening module, configured to perform beam screening on the audio signal to be processed based on the beam parameters to obtain a target audio signal, comprising: performing beamforming on the audio signal to be processed, and then performing beam screening on the beamformed audio signal using the beam parameters to obtain the target audio signal; An output module, used for sending the target audio signal to the paired Bluetooth device for processing, so as to obtain and output a processing result of the target audio signal; The receiving module includes: An acquisition unit, used to acquire a test audio signal in the current environment; A sending unit is used to send the test audio signal to the paired Bluetooth device to obtain the beam parameter determined by the paired Bluetooth device based on the test audio signal.

8. A wearable device, It is characterized in that include: A Bluetooth communication unit, used to receive beam parameters sent by a paired Bluetooth device and send them to a processor; An audio acquisition unit, used for acquiring the audio signal to be processed in the current environment and sending it to the processor; The processor is used to perform beam screening on the audio signal to be processed based on the beam parameters to obtain the target audio signal, including: beamforming the audio signal to be processed, and then using the beam parameters to perform beam screening on the beamformed audio signal to obtain the target audio signal, and the Bluetooth communication unit is also used to send the target audio signal to the paired Bluetooth device for processing to obtain the processing result of the target audio signal; An output unit is used to output the processing result of the target audio signal.

9. The wearable device according to claim 8, It is characterized in that The wearable device is augmented reality glasses.

10. An information interaction system, It is characterized in that include: The wearable device according to claim 8 or 9; A Bluetooth device is used to pair with the wearable device, the Bluetooth device is used to send beam parameters to the wearable device, and receive the target audio signal sent by the wearable device to obtain a processing result of the target audio signal, and send the processing result to the wearable device.

11. The information interaction system according to claim 10, It is characterized in that The system further comprises: The server is connected to the Bluetooth device, and is used to receive the target audio signal sent by the Bluetooth device, identify the target audio signal, determine the processing result of the target audio signal, and send the processing result to the Bluetooth device.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the information interaction method according to any one of claims 1 to 6.

Images