Space recording method and device, true wireless earphone and storage medium

Abstract

The invention discloses a space recording method and device, a true wireless earphone and a storage medium. Relates to the technical field of recording, and the method comprises the steps that under the condition that a target earphone and terminal equipment are in a Bluetooth connection state, the target earphone collects a target recording audio and target pose data; and the target earphone sends the target recorded audio and the target pose data to the terminal device, so that the terminal device generates spatial audio information according to the target recorded audio and the target pose data. According to the method, the posture data of the earphone can be collected and reported while the earphone collects the recording audio, so that the recording information with a space effect is generated, and better recording experience is brought to a user.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of recording technology, and are related to but not limited to a spatial recording method and device, a true wireless headset, and a storage medium. Background Art

[0002] The existing technology can achieve the effect of binaural recording technology of "binaural head stereo recording technology" by using the microphones (Microphone, MIC) on both sides of the True Wirless Stereo (TWS) headset to collect sound. "Binaural head stereo recording technology" is a recording technology that highly restores the real hearing sense. By equipping the two ears of the artificial head with microphones to capture the sound reaching each ear, the two sound signals can be reproduced on each ear respectively. The microphone is usually set on the headset to completely isolate the sound signals received by the two ears, so that the artificial head can completely reproduce what the audience hears.

[0003] In the prior art, TWS headphones only collect recording data and provide it to the terminal device. When the user wears the headphones and turns his head, the terminal device cannot change the recording effect according to the head rotation, so the recorded audio lacks a sense of space and the effect is not realistic enough. Summary of the invention

[0004] In view of this, the spatial recording method and apparatus, terminal device, and storage medium provided in the embodiments of the present application can collect and report the posture data of the headphones while the headphones are collecting the recorded audio, so as to generate recording information with spatial effects, thereby bringing users a better recording experience.

[0005] In a first aspect, a spatial recording method provided in an embodiment of the present application is applied to a true wireless TWS headset, wherein the TWS headset includes a target headset, and the target headset includes at least one of a master headset and a slave headset, and the method includes:

[0006] When the target headset is in a Bluetooth connection state with the terminal device, the target headset collects target recorded audio and target posture data, where the target posture data is used to indicate the spatial posture of the target headset corresponding to the target recorded audio;

[0007] The target earphone sends the target recorded audio and the target posture data to the terminal device, so that the terminal device generates spatial audio information according to the target recorded audio and the target posture data.

[0008] In a second aspect, the spatial recording device provided by the embodiment of the present application is applied to a true wireless TWS headset, wherein the TWS headset includes a target headset, wherein the target headset includes at least one of a master headset and a slave headset, and the device includes:

[0009] A data acquisition module, configured to enable the target headset to acquire target recorded audio and target posture data when the target headset is in a Bluetooth connection state with the terminal device, wherein the target posture data is used to indicate a spatial posture of the target headset corresponding to the target recorded audio;

[0010] A data sending module is used for the target earphone to send the target recorded audio and the target posture data to the terminal device, so that the terminal device generates spatial audio information according to the target recorded audio and the target posture data.

[0011] In a third aspect, the true wireless headset provided in an embodiment of the present application includes a memory and a processor, the memory stores a computer program that can be run on the processor, and the processor implements the steps of the spatial recording method provided in the first aspect of the embodiment of the present application when executing the program.

[0012] In a fourth aspect, the computer-readable storage medium provided in an embodiment of the present application stores a computer program thereon, and when the computer program is executed by a processor, the steps of the spatial recording method provided in the first aspect of the embodiment of the present application are implemented.

[0013] The spatial recording method, device, true wireless headset and computer-readable storage medium provided in the embodiments of the present application can collect and report the posture data of the headset while the headset collects the recorded audio, so as to generate recording information with spatial effects, bring users a better recording experience, and thus solve the technical problems raised in the background technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0015] Figure 1 A schematic diagram of a communication system provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of a process flow for implementing a spatial recording method provided in an embodiment of the present application;

[0017] Figure 3 A schematic diagram of the implementation flow of another spatial recording method provided in an embodiment of the present application;

[0018] Figure 4A schematic diagram of a process flow for implementing yet another spatial recording method provided in an embodiment of the present application;

[0019] Figure 5 A schematic diagram of a method for establishing a Bluetooth connection according to an embodiment of the present application;

[0020] Figure 6 A schematic diagram of a process flow for implementing another spatial recording method provided in an embodiment of the present application;

[0021] Figure 7 A schematic diagram of the overall process of a spatial recording method provided in an embodiment of the present application;

[0022] Figure 8 A schematic diagram of the structure of a spatial recording device provided in an embodiment of the present application;

[0023] Fig. 9 A schematic diagram of the structure of a true wireless TWS headset provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0026] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0027] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present application are used to distinguish similar or different objects, and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0028] The existing technology can achieve the effect of binaural recording technology in "binaural human head stereo recording technology" by using the microphones MIC on both sides of the TWS headset to collect sound. "Binaural human head stereo recording technology" is a recording technology that highly restores the real sense of hearing. By equipping the two ears of the artificial head with microphones to capture the sound reaching each ear, the two sound signals can be reproduced on each ear separately. The microphone is usually set on the headset to completely isolate the sound signals received by the two ears, so that the artificial head can completely reproduce what the audience hears.

[0029] Figure 1 A schematic diagram of a communication system scenario provided in an embodiment of the present application. Figure 1 As shown, the communication system includes a true wireless TWS headset 10 and a terminal device 20, wherein the TWS headset 10 includes a left headset 11 and a right headset 12. The left headset 11 and the right headset 12 can communicate with the terminal device 20 respectively. Exemplarily, the left headset 11 and the right headset 12 can establish a connection with the terminal device 20 via a Bluetooth connection and send recorded audio data.

[0030] In the prior art, TWS headphones only collect recording data and provide it to the terminal device. When the user wears the headphones and turns his head, the terminal device cannot identify from which direction the left and right ears pick up the sound, that is, the terminal device cannot change the recording effect according to the rotation of the head, so the recorded audio lacks a sense of space and the effect is not realistic enough.

[0031] In view of this, an embodiment of the present application provides a spatial recording method, in which the target headset collects target recording audio and target posture data when the target headset is in a Bluetooth connection state with the terminal device; the target headset sends the target recording audio and the target posture data to the terminal device, so that the terminal device generates spatial audio information according to the target recording audio and the target posture data. This method collects and reports the posture data of the headset while the headset collects the recording audio, which is used to generate recording information with spatial effects, giving users a better recording experience.

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0033] Figure 2 A schematic diagram of the implementation flow of a spatial recording method provided in an embodiment of the present application. The spatial recording method can be applied to a true wireless TWS headset, wherein the TWS headset includes a target headset, and the target headset includes at least one of a master headset and a slave headset. During the implementation process, the true wireless TWS headset can be various types of TWS headsets with information processing capabilities. Figure 2As shown, the method may include the following steps 101 to 104:

[0034] Step 101: When the target headset is in a Bluetooth connection state with a terminal device, the target headset collects target recorded audio and target posture data, where the target posture data is used to indicate the spatial posture of the target headset corresponding to the target recorded audio.

[0035] The target headset may collect the target recorded audio by an audio collection module provided on the target headset, such as a microphone. The embodiment of the present application does not limit the method for the target headset to collect the target recorded audio. For example, when the target headset includes a master headset and / or a master headset, the master headset may collect the corresponding recorded audio by a microphone provided by itself, and the slave headset may also collect the corresponding recorded audio by a microphone of its own device.

[0036] The target earphone may collect the target posture data by collecting the posture data through a posture collection module provided on the target earphone. The posture collection module may be an inertial measurement unit IMU, etc. The embodiment of the present application does not limit the manner in which the target earphone collects the target posture data. Exemplarily, when the target earphone includes a master earphone and / or a slave earphone, the master earphone or the slave earphone may collect the corresponding posture data through the IMU module provided by itself. Among them, the target posture data corresponds to the target recorded audio, that is, the target posture data can reflect the spatial posture of the target earphone during the process of collecting the target recorded audio.

[0037] Step 102: The target earphone sends the target recorded audio and the target posture data to the terminal device, so that the terminal device generates spatial audio information according to the target recorded audio and the target posture data.

[0038] Among them, there are many ways for the target earphone to send the target recorded audio and the target posture data to the terminal device. For example, when the target earphone includes a master earphone or a slave earphone, the target earphone can send the collected target recorded audio and target posture data to the terminal device through the master earphone or the slave earphone. When the target earphone includes a master earphone and a slave earphone, the master earphone and the slave earphone can each transmit their own collected recorded audio and posture data, or transmit the recorded audio and posture data collected by both earphones through one of the earphones. The embodiment of the present application does not limit the way in which the target earphone sends the target recorded audio and the target posture data to the terminal device.

[0039] It can be understood that after the terminal device receives the target recorded audio and the target posture data corresponding to the target headset, the target recorded audio and the target posture data are given to the audio module of the terminal device for spatial rendering processing, and then the processed data is sent to the recording application, thereby realizing the spatial recording function of the TWS headset.

[0040] The control method of the application function provided in the embodiment of the present application can, when the TWS headset collects recorded audio through the headset, simultaneously collect posture data and report the posture data together to the terminal device, thereby generating recording information with spatial effects, giving users a better recording experience.

[0041] In some embodiments, the target headset can collect target recorded audio and target posture data according to the instruction information sent by the terminal device.

[0042] Figure 3 This is a schematic diagram of another implementation flow of a spatial recording method provided in an embodiment of the present application. Figure 3 As shown, in Figure 2 Based on the illustrated embodiment, the target earphone collecting the target recorded audio and target posture data in step 101 may include:

[0043] Step 201: At the collection start time indicated by the terminal device, the target headset starts collecting the target recorded audio and the target posture data.

[0044] It should be noted that after the target headset establishes a Bluetooth connection with the terminal device, the terminal device can send an indication message to the target headset to instruct the target headset to collect the target recorded audio and target posture data, and the indication message may include the collection start time. The target headset can start collecting the target recorded audio and target posture data at the collection start time. Exemplarily, when the target headset includes a master headset and a slave headset, the terminal device sends the collection start time to the master headset and the slave headset so that the master headset and the slave headset start collecting recorded audio and posture data at the same time, and the terminal device can correspond the master and slave headset data when performing data processing.

[0045] It is understandable that the terminal device can also send the data segmentation method to the master earphone and the slave earphone. For example, the terminal device instructs the master earphone and the slave earphone to collect every 20ms of recorded audio as a frame, and determine the same collection start time and data segmentation method, so that the terminal device can align each frame of recorded data collected by the left and right ears.

[0046] In some embodiments, before the target earphone sends the target posture data to the terminal device, it can first determine whether the spatial posture of the target earphone has changed, and if it has changed, send the target posture data.

[0047] In an embodiment of the present application, the target recorded audio and the target posture data are collected within a target collection time period. Before the target earphone sends the target posture data to the terminal device in step 102, the method may further include: determining whether the spatial posture of the target earphone has changed within the target collection time period.

[0048] It should be noted that there are many ways to determine whether the spatial posture of the target earphone has changed during the target acquisition time period. For example, the currently acquired posture data can be compared with the posture data acquired last time. When the posture data difference reaches a preset threshold, it is considered that the spatial posture of the target earphone has changed during the target acquisition time period. It is also possible to determine that the spatial posture of the current earphone has also changed when it is determined that the spatial posture of the other earphone has changed. The embodiment of the present application does not limit the method for determining the change in the spatial posture of the target earphone during the target acquisition time period.

[0049] In some embodiments, the target headset may send the target recorded audio and target posture data to the terminal device in two ways: Method 1: the target recorded audio and target posture data are sent through the same Bluetooth connection, and Method 2: the target recorded audio and target posture data are sent through different Bluetooth connections.

[0050] In an embodiment of the present application, the target earphone sends the target recorded audio and the target posture data to the terminal device, which may include: the target earphone sends the target recorded audio and the target posture data to the terminal device through the same Bluetooth connection; or, the target earphone sends the target recorded audio and the target posture data to the terminal device respectively through different Bluetooth connections.

[0051] It should be noted that the target headset sends the target recorded audio and target posture data to the terminal device through the same Bluetooth connection, and only one Bluetooth connection can be established, and the device connection speed is faster. The target headset sends the target recorded audio and target posture data to the terminal device through different Bluetooth connections, and two Bluetooth connections need to be established, and the connection speed is slower, but when transmitting data, both data can be transmitted at the same time, and the data transmission speed is faster.

[0052] Furthermore, the different Bluetooth connections may include a Bluetooth connection established based on the Bluetooth HOGP protocol and a Bluetooth connection established based on the Bluetooth audio protocol. The target headset sends the target recorded audio and the target posture data to the terminal device through different Bluetooth connections, which may include: the target headset sends the target posture data to the terminal device through the Bluetooth connection established based on the Bluetooth HOGP protocol; and the target headset sends the target recorded audio to the terminal device through the Bluetooth connection established based on the Bluetooth audio protocol.

[0053] It should be noted that the target headset sends the target recorded audio and target posture data to the terminal device through different Bluetooth connections respectively. HOGP is the abbreviation of HID Over GATT Profile, that is, the Bluetooth HID device implements the Human Interface Device (HID) protocol through the Generic Attribute Profile (GATT) of Bluetooth Low Energy (BLE). The target posture data can be transmitted through the Bluetooth connection established by the HOGP protocol. The Bluetooth connection established based on the Bluetooth audio protocol can be a CIS connection. The CIS connection is a connection under Bluetooth low energy audio Bluetooth LE Audio. In this connection mode, each audio stream is called CIS (Connected Isochronous Stream); each CIS is connected to a CIG (Connected IsochronousGroup) for audio synchronization (for example, the left and right earphones play audio synchronously). The target recorded audio can be transmitted through a Bluetooth connection established by the low energy Bluetooth audio BLE Audio protocol.

[0054] Furthermore, before the target headset sends the target recorded audio to the terminal device via the Bluetooth connection established based on the Bluetooth audio protocol, the method may also include: the target headset determines a time sequence, and the time sequence is used to indicate the time sequence of the target headset sending the target recorded audio to the terminal device via the Bluetooth connection established based on the Bluetooth audio protocol.

[0055] Among them, after the target headset and the terminal device establish a Bluetooth connection based on the Bluetooth audio protocol, before sending the target recorded audio to the terminal device, it is necessary to negotiate with the terminal device and specify multiple time points for sending the target recorded audio, or a starting time point and a fixed time interval, that is, a time sequence. On the one hand, it is convenient for the terminal device to receive the target recorded audio sent by the target headset. On the other hand, when the target headset includes a master headset and a slave headset, the sending time of the master headset and the slave headset can be coordinated to avoid conflicts, so that the terminal device can better receive the target recorded audio.

[0056] For example, when starting binaural recording, the terminal device can establish an independent connection synchronization stream CIS connection with the master earphone and the slave earphone respectively (but these two CIS connections belong to the same combination), and specify the time sequence of the data exchange between the master and slave ears and the terminal device when establishing the CIS connection. When the terminal device is ready to receive data, it notifies the master and slave earphones of the TWS earphones to start recording and send the recording data to the terminal device, and the master and slave earphones report their respective posture data to the terminal device through the HOGP connection.

[0057] It can be understood that there are many methods for the two ears to report their respective posture data in the embodiments of the present application, which are not limited to the use of the HOGP protocol.

[0058] Furthermore, when the target headset sends the target recorded audio and the target posture data to the terminal device respectively through different Bluetooth connections, the target recorded audio and the target posture data both include corresponding tag information, and the tag information is used to indicate the collection time.

[0059] It is understandable that when the target recorded audio and target posture data are sent to the terminal device via different Bluetooth connections, marking information needs to be added to indicate the data collection time so that the terminal device can identify the recorded audio and posture data collected at the same time for data processing.

[0060] In some embodiments, the target earphone may send the target recorded audio and the target position data to the terminal device in such a manner that the master earphone and the slave earphone each send the data collected by themselves to the terminal device.

[0061] In an embodiment of the present application, the target earphone includes a master earphone and a slave earphone, and the target earphone sends the target recorded audio and the target posture data to the terminal device, which may include: the master earphone and the slave earphone respectively send their respectively collected recorded audio and posture data to the terminal device, the target recorded audio includes the recorded audio collected by the master earphone and the slave earphone respectively, and the target posture data includes the posture data collected by the master earphone and the slave earphone respectively.

[0062] It should be noted that, when the target headset includes a master headset and a slave headset, there are two ways for the target headset to send the target recorded audio and target posture data to the terminal device. One is that the master headset is responsible for both data transmission with the terminal device and transfer of the slave headset data to the terminal device. The disadvantage of this method is that the master headset consumes more power, so the battery life of the two headsets is quite different, and the master headset needs to transmit the recording data and posture data of the two headsets to the terminal device. The data volume is large and will occupy a large Bluetooth bandwidth. When the interference in the environment is slightly larger, it is easy to cause the master headset data to fail to be sent to the terminal device in time, resulting in jams and affecting the user experience. The other is that the master headset and the slave headset send their collected recording data and posture data to the terminal device respectively. This method can keep the battery life of the master headset and the slave headset close, and neither headset will occupy a large Bluetooth bandwidth, ensuring the stability of data transmission and improving the user experience.

[0063] It can be understood that the solution provided in the embodiment of the present application enables the dual earphones of the TWS earphone to independently interact with the terminal device for recording data. The recording data of the slave earphone does not need to be forwarded through the master earphone, and the recording power consumption of the dual earphones is consistent. Compared with the existing solution of transmitting all data of the main earphone and the slave earphone to the terminal device through the main earphone, the amount of data sent by the main earphone to the terminal device is smaller, and the occupied Bluetooth bandwidth is smaller, thereby having a stronger anti-interference ability, which can bring a better recording experience to the user.

[0064] In some embodiments, before the target headset transmits the target recorded audio and target posture data to the terminal device, the target headset needs to establish a Bluetooth connection with the terminal device.

[0065] Figure 4 A schematic diagram of the implementation flow of another spatial recording method provided in the embodiment of the present application. Figure 4 As shown, in Figure 2 On the basis of the illustrated embodiment, the target headset includes a master headset and a slave headset. When the target headset and the terminal device are in a Bluetooth connection state in step 101, before the target headset collects the target recorded audio and target posture data, the method further includes:

[0066] Step 301: The master headset and the slave headset respectively establish Bluetooth connections with the terminal device.

[0067] It should be noted that only when the master headset and the slave headset respectively establish Bluetooth connections with the terminal device, can the master headset and the slave headset respectively send their collected recorded audio and posture data to the terminal device.

[0068] Figure 5The following is a schematic diagram of a method for establishing a Bluetooth connection according to an embodiment of the present application. Figure 5 As shown, in step 301, the master headset and the slave headset respectively establish a Bluetooth connection with the terminal device, which may include:

[0069] Step 401: the master headset establishes a first Bluetooth connection with the terminal device through a first Bluetooth broadcast, and sends first verification information to the terminal device.

[0070] Step 402: The slave headset sends second verification information and combined verification information to the terminal device via a second Bluetooth broadcast, so that the terminal device obtains calculated verification information based on the first verification information and the second verification information, and establishes a second Bluetooth connection with the slave headset when the combined verification information matches the calculated verification information.

[0071] It should be noted that when the TWS headset is opened, the master and slave headsets respectively send Bluetooth low-power audio BLE Audio broadcasts. When the user chooses to connect the TWS headset on the terminal device, it will first establish a low-power audio LE Audio connection with the master headset of the TWS headset, so as to obtain the first verification information of the master headset, that is, the combined identification resolution value SIRK. Then, when the terminal device receives the low-power audio LE Audio broadcast sent by the slave headset of the TWS headset, it is parsed according to the SIRK of the master headset and the second verification information and the combined verification information in the broadcast to determine that they belong to the same combination, that is, the calculated verification information is obtained according to the first verification information and the second verification information, and then determined according to the calculated verification information and the combined verification information. Then, the terminal device automatically initiates a connection with the slave headset, so that the terminal device respectively establishes BLE Audio connections with the master and slave headsets of the TWS headset. At the same time, the terminal device can also establish a connection with the two headsets based on BLE (for example, establish a HOGP connection with each left and right headset respectively), which is used for the headset to report the headset posture data in real time.

[0072] The above method of establishing a Bluetooth connection between the master headset, the slave headset and the terminal device can prepare to establish a connection between the master headset, the slave headset in the combination and the terminal device, thereby improving the accuracy of establishing the Bluetooth connection.

[0073] In some embodiments, before the target earphone collects the target recorded audio and the target posture data, the target earphone may also send the initial posture data to the terminal device.

[0074] Figure 6 A schematic diagram of the implementation flow of another spatial recording method provided in an embodiment of the present application. Figure 6 As shown, in Figure 2Based on the illustrated embodiment, before the target earphone collects the target recorded audio and target posture data in step 101, the method further includes:

[0075] Step 501: The target earphone collects initial posture data, and sends the initial posture data to the terminal device, so that the terminal device generates the spatial audio information according to the initial posture data, the target recorded audio and the target posture data.

[0076] It should be noted that before the target headset starts recording, the target headset may first collect initial posture data and send it to the terminal device, so that the terminal device can determine whether the posture of the target headset changes during the recording collection process. When the target headset includes a master headset and a slave headset, the master headset and the slave headset respectively send their collected initial posture data to the terminal device.

[0077] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0078] Figure 7 The overall flow chart of a spatial recording method provided in an embodiment of the present application is as follows. Figure 7 As shown, the true wireless TWS headset includes headset 1 and headset 2, wherein headset 1 is a master headset and headset 2 is a slave headset. The method includes the following steps 601 to 612:

[0079] Step 601: When the earphone box of the TWS earphone is opened, earphone 1 sends a Bluetooth low energy audio BLE Audio broadcast 1, and earphone 2 sends a Bluetooth low energy audio BLE Audio broadcast 2;

[0080] Step 602: The mobile phone scans the headset 1 through inquiry scanning;

[0081] Step 603: When the user clicks to establish a connection with the TWS headset, the mobile phone connects to the BLE Audio of headset 1, and the mobile phone establishes a BLE Audio connection with headset 1;

[0082] Step 604: The mobile phone obtains the combined identification resolution value SIRK of the headset 1 through the headset 1;

[0083] Step 605: The mobile phone receives the BLE Audio broadcast 2 sent by the headset 2, and calculates the local value Local Value in a specific manner according to the SIRK of the headset 1 and the second verification information in the broadcast 2;

[0084] Step 606: The mobile phone determines whether the local value Local Value is equal to the specific field value in broadcast 2;

[0085] Step 607: If they are equal, the mobile phone automatically initiates a connection to earphone 2, and the mobile phone and earphone 2 establish a BLE Audio connection;

[0086] Step 608: The mobile phone establishes HOGP connections with the two earphones respectively. Earphone 1 establishes HOGP connection with the mobile phone and uploads initial posture data. Earphone 2 establishes HOGP connection with the mobile phone and uploads initial posture data.

[0087] Step 609: The mobile phone controls binaural recording, earphone 1 establishes a CIS connection with the mobile phone and specifies a time sequence, earphone 2 establishes a CIS connection with the mobile phone and specifies a time sequence. The time sequence includes the start time of acquisition;

[0088] Step 610: Headphone 1 and Headphone 2 record the audio respectively and transmit the recorded audio data to the mobile phone at a specified time;

[0089] Step 611: Headphone 1 and Headphone 2 respectively determine whether their collected posture data are updated;

[0090] Step 612: if there is an update, the updated posture data is reported to the mobile phone, and the mobile phone processes the binaural data according to the recording data and the posture data;

[0091] Step 613: The mobile phone, earphone 1 and earphone 2 determine whether the recording is finished. If not, the process returns to step 609. If finished, the process stops collecting the recording and gesture data.

[0092] In the above solution, based on the Bluetooth low energy audio BLE Audio function of the TWS headset, the dual earphones of the TWS headset can record independently and transmit them to the mobile phone. At the same time, the posture data of the dual earphones is reported to the mobile phone in real time, and then the mobile phone performs spatial rendering processing on the dual earphone data. In this solution, not only the power consumption of the master earphone and the slave earphone is consistent during recording, and the anti-interference ability is stronger, but also the recorded sound has a spatial effect, and the effect of recording with dual earphones is better, thus providing users with a better experience.

[0093] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0094] Based on the foregoing embodiments, the embodiments of the present application provide a spatial recording device, and the modules included in the device and the units included in each module can be implemented by a processor; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0095] Figure 8 This is a schematic diagram of the structure of a spatial recording device provided in an embodiment of the present application. Figure 8 As shown, the device 700 includes a data acquisition module 701 and a data sending module 702, wherein:

[0096] The data collection module 701 is used for collecting target recorded audio and target posture data by the target earphone when the target earphone is in a Bluetooth connection state with the terminal device, wherein the target posture data is used to indicate the spatial posture of the target earphone corresponding to the target recorded audio;

[0097] The data sending module 702 is used for the target earphone to send the target recorded audio and the target posture data to the terminal device, so that the terminal device generates spatial audio information according to the target recorded audio and the target posture data.

[0098] In some embodiments, the data collection module 701 is specifically used for: at the collection start time indicated by the terminal device, the target headset starts collecting the target recorded audio and the target posture data.

[0099] In some embodiments, the target recorded audio and the target posture data are collected within a target collection time period, and the device also includes a posture determination module, which is used to determine whether the spatial posture of the target headset has changed within the target collection time period.

[0100] In some embodiments, the data sending module 702 includes a first sending unit and a second sending unit, wherein the first sending unit is used for the target headset to send the target recorded audio and the target posture data to the terminal device through the same Bluetooth connection; or, the second sending unit is used for the target headset to send the target recorded audio and the target posture data to the terminal device respectively through different Bluetooth connections.

[0101] In some embodiments, the different Bluetooth connections include a Bluetooth connection established based on the Bluetooth HOGP protocol and a Bluetooth connection established based on the Bluetooth audio protocol; the second sending unit is specifically used for: the target headset sends the target posture data to the terminal device through the Bluetooth connection established based on the Bluetooth HOGP protocol; and the target headset sends the target recorded audio to the terminal device through the Bluetooth connection established based on the Bluetooth audio protocol.

[0102] In some embodiments, the data sending module 702 also includes a time determination unit, which is used to determine a time sequence before the target headset sends the target recorded audio to the terminal device through the Bluetooth connection established based on the Bluetooth audio protocol. The time sequence is used to indicate the time sequence of the target headset sending the target recorded audio to the terminal device through the Bluetooth connection established based on the Bluetooth audio protocol.

[0103] In some embodiments, the second sending unit is specifically used for: when the target headset sends the target recorded audio and the target posture data to the terminal device respectively through different Bluetooth connections, the target recorded audio and the target posture data both include corresponding tag information, and the tag information is used to indicate the collection time.

[0104] In some embodiments, the target earphone includes a master earphone and a slave earphone, and the data sending module 702 is specifically used for: the master earphone and the slave earphone respectively send their collected recorded audio and posture data to the terminal device, the target recorded audio includes the recorded audio collected by the master earphone and the slave earphone, and the target posture data includes the posture data collected by the master earphone and the slave earphone.

[0105] In some embodiments, the target headset includes a master headset and a slave headset, and the apparatus further includes: a Bluetooth connection module, and the Bluetooth connection module is used for the master headset and the slave headset to establish Bluetooth connections with the terminal device respectively.

[0106] In some embodiments, the Bluetooth connection module is specifically used for: the master earphone establishes a first Bluetooth connection with the terminal device through a first Bluetooth broadcast, and sends first verification information to the terminal device; the slave earphone sends second verification information and combined verification information to the terminal device through a second Bluetooth broadcast, so that the terminal device obtains calculated verification information based on the first verification information and the second verification information, and establishes a second Bluetooth connection with the slave earphone when the combined verification information matches the calculated verification information.

[0107] In some embodiments, the apparatus further includes an initial sending module, which is used for the target earphone to collect initial posture data and send the initial posture data to the terminal device, so that the terminal device generates the spatial audio information based on the initial posture data, the target recorded audio and the target posture data.

[0108] In an embodiment of the present application, the headset can collect and report posture data while collecting recorded audio, which is used to generate recording information with spatial effects, giving users a better recording experience.

[0109] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0110] It should be noted that in the embodiments of this application Figure 8 The division of modules in the spatial recording device shown is schematic and is only a logical functional division. There may be other division methods in actual implementation. In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. It may also be implemented in the form of a combination of software and hardware.

[0111] It should be noted that in the embodiments of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or partly contribute to the relevant technology in the form of a software product, which is stored in a storage medium and includes several instructions for enabling the TWS headset to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0112] Fig. 9 This is a schematic diagram of the structure of a true wireless TWS headset provided in an embodiment of the present application. Fig. 9 As shown, the true wireless headset 100 may include a processor 110, a memory 120, a wireless communication module 130, a microphone 140, a speaker 150, an audio module 160, a sensor module 170, a power module 180, and the like.

[0113] The processor 110 may include one or more processing units. For example, the processor 110 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs). Different processing units may be independent devices or integrated into one or more processors.

[0114] The memory 120 can be used to store computer executable program codes, which include instructions. The internal memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data (such as audio data, video data, etc.) created during the use of the true wireless headset 100. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the true wireless headset 100 by running instructions stored in the memory 120 and / or instructions stored in a memory provided in the processor.

[0115] The wireless communication module 130 can provide solutions for wireless communication including WLAN, such as Wi-Fi network, Bluetooth, NFC, IR, etc., applied to the true wireless headset 100. The wireless communication module 130 can be one or more devices integrating at least one communication processing module. In some embodiments of the present application, the true wireless headset 100 can establish a wireless communication connection with other terminal devices through the wireless communication module 130.

[0116] The microphone 140 is used to convert the collected audio signal into an electrical signal and transmit it to the processor 110. For the microphone 140, its type may include an analog microphone and a digital microphone. Correspondingly, according to the different types of microphones, the audio signal collected and transmitted to the processor 110 may include an analog signal or a digital signal. In addition, it should be noted that the microphone 140 of a single true wireless headset can be set to one or more, that is, each true wireless headset can collect audio through a single microphone 140 or through multiple microphones 140. When multiple microphones 140 are used to collect audio on each true wireless headset, the audio signals collected by the multiple microphones 140 of each true wireless headset are converted to digital and encoded by their respective audio modules 160, and after the noise reduction filtering of the noise reduction filtering module and the echo cancellation of the echo cancellation module, the processor 110 merges the multiple audio signals collected by the multiple microphones into one channel as the audio signal finally output by each true wireless headset.

[0117] The speaker 150 is used to convert the audio signal into a sound signal and then output it. The user can listen to music, etc. through the speaker 150.

[0118] The audio module 160 is used to convert digital audio information into an analog audio signal output, or to convert an analog audio input into a digital audio signal. The audio module 160 can also be used to encode and decode audio signals. In some embodiments of the present application, the audio module 160 can be arranged in the processor 110, or some functional modules of the audio module 160 can be arranged in the processor 110. For the audio module 160, it includes but is not limited to an analog-to-digital converter, an encoder, and a decoder, etc. For the analog-to-digital converter, it converts the analog audio signal collected by a true wireless headset into a digital audio signal; for the encoder, it compresses and encodes the audio signal collected by the true wireless headset, and transmits the encoded audio signal to the other true wireless headset via Bluetooth transmission or other wireless transmission methods. The decoder arranged in the other true wireless headset decodes and restores the original audio signal collected by a true wireless headset.

[0119] The sensor module 170 may include a gyro sensor, which may be used to determine the motion posture of the TWS headset 100. In some embodiments, the angular velocity of the TWS headset 100 around three axes (ie, x, y, and z axes) may be determined by the gyro sensor.

[0120] The power module 180 may include a battery, a battery management component, etc. The power management component is used to manage the charging of the power supply and the power supply to other modules. The battery may be a rechargeable battery such as a lithium battery or a lead-acid battery. The shape of the battery may be, but is not limited to, a button type, a square type, etc.

[0121] It is understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the true wireless headset 100. In other embodiments of the present application, the true wireless headset 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0122] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the spatial recording method provided in the above embodiment are implemented.

[0123] The above-mentioned computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0124] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0125] The program code embodied on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0126] Computer program code for performing the operations of the present specification may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] An embodiment of the present application provides a computer program product comprising instructions, which, when run on a true wireless TWS headset, enables the true wireless TWS headset to execute the steps of the spatial recording method provided in the above method embodiment.

[0128] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center to another website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0129] It should be noted here that the description of the above storage medium, program product and device embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the storage medium, storage medium and device embodiments of this application, please refer to the description of the method embodiment of this application for understanding.

[0130] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0131] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0132] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0133] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0134] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0135] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0136] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0137] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or partly contribute to the relevant technology in the form of a software product, which is stored in a storage medium and includes several instructions for enabling the TWS headset to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0138] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain a new method embodiment. The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain a new product embodiment. The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain a new method embodiment or device embodiment.

[0139] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A spatial recording method, characterized in that: Applied to a true wireless TWS headset, the TWS headset includes a target headset, the target headset includes at least one of a master headset and a slave headset, and the method includes: When the target headset is in a Bluetooth connection state with the terminal device, the target headset collects target recorded audio and target posture data, where the target posture data is used to indicate the spatial posture of the target headset corresponding to the target recorded audio; The target earphone sends the target recorded audio and the target posture data to the terminal device, so that the terminal device generates spatial audio information according to the target recorded audio and the target posture data.

2. The method according to claim 1, characterized in that The target recorded audio and the target posture data are collected within a target collection time period. Before the target headset sends the target posture data to the terminal device, the method further includes: It is determined that the spatial posture of the target headset changes during the target acquisition time period.

3. The method according to claim 1, characterized in that The target earphone sends the target recorded audio and the target posture data to the terminal device, including: The target headset sends the target recorded audio and the target posture data to the terminal device via the same Bluetooth connection; or, The target earphone sends the target recorded audio and the target posture data to the terminal device respectively through different Bluetooth connections.

4. The method according to claim 3, characterized in that The different Bluetooth connections include a Bluetooth connection established based on a Bluetooth HOGP protocol and a Bluetooth connection established based on a Bluetooth audio protocol; the target headset sends the target recorded audio and the target posture data to the terminal device through different Bluetooth connections, including: The target headset sends the target posture data to the terminal device through the Bluetooth connection established based on the Bluetooth HOGP protocol; and The target headset sends the target recorded audio to the terminal device via the Bluetooth connection established based on the Bluetooth audio protocol.

5. The method according to claim 4, characterized in that Before the target headset sends the target recorded audio to the terminal device through the Bluetooth connection established based on the Bluetooth audio protocol, the method further includes: The target headset determines a time sequence, and the time sequence is used to indicate a time sequence in which the target headset sends the target recorded audio to the terminal device through the Bluetooth connection established based on the Bluetooth audio protocol.

6. The method according to any one of claims 1 to 5, characterized in that The target earphone includes a master earphone and a slave earphone, and the target earphone sends the target recorded audio and the target posture data to the terminal device, including: The master earphone and the slave earphone respectively send the recorded audio and posture data collected by each of them to the terminal device, the target recorded audio includes the recorded audio collected by each of the master earphone and the slave earphone, and the target posture data includes the posture data collected by each of the master earphone and the slave earphone.

7. The method according to claim 1, characterized in that The target headset includes a master headset and a slave headset. When the target headset is in a Bluetooth connection state with the terminal device, before the target headset collects the target recorded audio and target posture data, the method further includes: The master headset establishes a first Bluetooth connection with the terminal device through a first Bluetooth broadcast, and sends first verification information to the terminal device; The slave earphone sends the second verification information and the combined verification information to the terminal device via the second Bluetooth broadcast, so that the terminal device obtains the calculated verification information based on the first verification information and the second verification information, and establishes a second Bluetooth connection with the slave earphone when the combined verification information matches the calculated verification information.

8. The method according to claim 1, characterized in that: Before the target earphone collects the target recorded audio and the target posture data, the method further includes: The target earphone collects initial posture data and sends the initial posture data to the terminal device, so that the terminal device generates the spatial audio information according to the initial posture data, the target recorded audio and the target posture data.

9. A spatial recording device, characterized in that: Applied to a true wireless TWS headset, the TWS headset includes a target headset, the target headset includes at least one of a master headset and a slave headset, and the device includes: A data acquisition module, configured to enable the target headset to acquire target recorded audio and target posture data when the target headset is in a Bluetooth connection state with the terminal device, wherein the target posture data is used to indicate a spatial posture of the target headset corresponding to the target recorded audio; A data sending module is used for the target earphone to send the target recorded audio and the target posture data to the terminal device, so that the terminal device generates spatial audio information according to the target recorded audio and the target posture data.

10. A true wireless headset, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps of the spatial recording method according to any one of claims 1 to 8 are implemented.

Images