Charging box, true wireless earphone, earphone system and audio playing method

Abstract

The invention provides a charging box, a true wireless earphone, an earphone system and an audio playing method, and relates to the field of earphones. The charging box is used for charging the real wireless earphone and comprises a first wireless communication module, a camera device and a first processor; the first processor is respectively connected with the camera device and the first wireless communication module; the first processor is used for receiving an acquired image shot by the camera device; identifying user pose information in the acquired image; and sending the user pose information to a true wireless earphone through the first wireless communication module, so that the true wireless earphone adjusts different sound channel information in to-be-played audio data based on the user pose information, and plays the adjusted to-be-played audio data. The charging box can improve the direction sense and stereoscopic sense of audio data playing and improve the playing effect.

Description

Technical Field

[0001] The present application relates to the field of headphones, and specifically provides a charging box, true wireless headphones, a headphone system and an audio playback method. Background Art

[0002] True wireless headphones can play audio with a stereo effect, so that the sound heard by the user has directionality, such as music, game sounds, etc.

[0003] However, the sense of direction of the sound played by true wireless headphones currently usually comes from the audio data itself, for example, the sense of direction of the sound is reflected through different channel information in the audio data.

[0004] Because true wireless headphones are small in size and can communicate wirelessly, users can choose to wear the headphones in some more comfortable postures, such as lying flat, lying on their side, leaning, tilting their head, etc. When the user is in different postures, there will be a certain angle offset in the user's posture. Since the sense of direction of the audio data comes from the audio data itself, the sense of direction of the played sound will change with the user's posture, which cannot bring a good listening experience to the user.

[0005] For example, when a user listens to the sound corresponding to the audio data, if the direction of the sound is initially in front of the user, the sound heard by the left and right ears is the same. After the user turns his head to the left, the direction of the sound generally needs to remain unchanged and remain in the original direction, that is, to the right of the head after the user turns his head. In this case, the sound heard by the right ear will be louder than the sound heard by the left ear. However, since the sense of direction of the audio data comes from the audio data itself, the direction of the sound will still rotate together, so that the direction of the sound is in front of the head after the user turns his head, and the sound heard by the left and right ears is still the same. This method will cause the sense of direction of the sound to change with the change of the user's posture, resulting in an unstable and unreal sense of direction of the sound, which will make the user's listening experience poor. Summary of the Invention

[0006] In view of this, the present application aims to provide a charging box, a true wireless headset, an earphone system and an audio playback method to improve the stereo playback effect of the true wireless headset.

[0007] First, an embodiment of the present application provides a charging box, which is used to charge true wireless headphones; the charging box includes: a first wireless communication module, a camera device and a first processor; the first processor is connected to the camera device and the first wireless communication module, respectively; the first processor is used to: receive the captured image taken by the camera device; identify the user posture information in the captured image; send the user posture information to the true wireless headphones through the first wireless communication module, so that the true wireless headphones adjust the different channel information in the audio data to be played based on the user posture information, and play the adjusted audio data to be played.

[0008] In an embodiment of the present application, a camera device is provided in the charging box, so that the charging box can capture the image of the user and identify the user's posture. After the charging box sends the user's posture information to the true wireless headset, the true wireless headset can adjust the different channel information of the audio data to be played according to the user's posture information, so that the sense of direction of the sound corresponding to the audio data to be played does not change with the change of the user's posture, thereby maintaining the sense of direction and stereoscopic sense of the audio data playback, and bringing a better listening experience to the user. Among them, compared with the true wireless headset, the use of the charging box to identify the user's posture, on the one hand, the size of the charging box is larger, and a larger size can be set for the camera device, avoiding the camera device from occupying the size of the true wireless headset. On the other hand, the charging box can be designed with a larger battery, providing better battery life, and has lower power consumption requirements. Due to size limitations, the true wireless headset cannot be designed with a larger battery. Therefore, the true wireless headset has higher power consumption requirements. The charging box of the present application captures the collected image, identifies the posture and sends the user's posture information to the true wireless headset, which can avoid transmitting the collected image and avoid the true wireless headset from identifying the posture based on the collected image, thereby reducing the power consumption of the true wireless headset.

[0009] In one embodiment, the captured images taken by the camera device include multiple images; the first processor is used to: determine the posture of the charging box through synchronous positioning and map building technology and multiple captured images; and determine the user posture information in at least one of the captured images based on the posture of the charging box.

[0010] Posture recognition can be to determine the posture of the target through the relative posture relationship between itself and the target. Therefore, when the charging box recognizes the user's posture, it must first determine its own charging box posture, and then recognize the user's posture information through the charging box posture. In the embodiment of the present application, multiple captured images are identified through synchronous positioning and map construction technology to determine the charging box posture. The synchronous positioning and map construction technology can utilize the changes in the static object features in multiple captured images to determine a more accurate charging box posture. Therefore, through a more accurate charging box posture, more accurate user posture information can be determined, and then the audio data to be played can be accurately adjusted to improve the playback effect.

[0011] In one embodiment, the first processor is configured with a pre-trained posture recognition model; the first processor is used to determine the user posture information in the captured image through the posture recognition model.

[0012] In the embodiment of the present application, the posture recognition model is trained by a large number of training samples. Using the pre-trained posture recognition model for posture recognition can help improve the accuracy and recognition efficiency of determining the user posture information.

[0013] In one embodiment, the first processor is further used to configure first timestamp information in the user posture information in response to the generation of the user posture information; the true wireless headset includes an inertial measurement unit IMU (Inertial Measurement Unit); the true wireless headset is also used to: determine the headset posture information through the IMU and configure second timestamp information for the headset posture information, and, in response to receiving the user posture information sent by the first wireless communication module, fuse the user posture information and the headset posture information based on the first timestamp information and the second timestamp information to obtain fused posture information, and adjust different channel information in the audio data to be played based on the fused posture information, and play the adjusted audio data to be played.

[0014] In this embodiment, the true wireless headset has an IMU, so the true wireless headset can adjust the audio data to be played through the user posture information determined by the charging box and the headset posture information determined by the IMU. Compared with the single user posture information or headset posture information, the fused posture information obtained by fusing the two can combine the characteristics of the two, so that the fused posture feature can more accurately represent the actual posture of the user. Therefore, adjusting the audio data to be played based on the fused posture feature can make its sense of direction after being played match the actual posture of the user, thereby improving the playback effect. In addition, when fusing, the user posture information and the headset posture information should be fused at the same time. Therefore, configuring the first timestamp information and the second timestamp information can help improve the accuracy of the fusion of the two, and thus can more accurately represent the actual posture of the user.

[0015] In one embodiment, the charging box also includes a first trigger circuit, which is connected to the first wireless communication module; the first trigger circuit is configured to generate third timestamp information after the first wireless communication module sends a wireless frame to the true wireless headset, and the charging box sends the third timestamp information to the true wireless headset through the first wireless communication module; the true wireless headset is also configured to: generate fourth timestamp information after receiving the wireless frame; if the wireless frame includes the user posture information, the user posture information and the headset posture information are fused through the first difference and the second difference to obtain the fused posture information; the first difference is the difference between the first timestamp information and the third timestamp information, and the second difference is the difference between the second timestamp information and the fourth timestamp information.

[0016] The charging case and the true wireless earphones are different devices, and their clocks may not be synchronized or even have different frequencies. Based on this, it is impossible to accurately determine the posture at the same time based on the first and second timestamp information. Therefore, in an embodiment of the present application, third and fourth timestamp information are also generated. The third timestamp information represents the time when the wireless frame is sent, where the user posture information is sent through the wireless frame. The fourth timestamp information represents the time when the true wireless earphones receive the wireless frame. The fourth timestamp information may also include the time when the user posture information is received. Therefore, when the wireless frame includes user posture information, the third and fourth timestamp information can be combined for fusion processing. The two are synchronized or have a certain time delay. Therefore, the first difference actually represents the difference between the time when the user posture information is generated and a reference time. Similarly, the second difference represents the difference between the time when the earphone posture information is generated and the reference time. Therefore, calculating the first and second differences and using the differences between each and the reference time for synchronization can effectively reduce the impact of clock cycle and frequency, effectively improve the accuracy of the fused posture information, and thus improve the accuracy of the adjustment of the audio data to be played, thereby improving the playback effect.

[0017] Based on the same inventive concept, an embodiment of the present application provides a true wireless headset, comprising: a second wireless communication module, used to connect to the charging box as described in any one of the first aspects, and receive the user posture information; a second processor, connected to the second wireless communication module, used to adjust the different channel information in the audio data to be played based on the user posture information, and play the adjusted audio data to be played.

[0018] In one embodiment, the user posture also includes first timestamp information; the true wireless headset also includes an IMU, and the processor is connected to the IMU; the IMU is used to detect the headset posture information and configure second timestamp information for the headset posture information; the second processor is also used to: fuse the user posture information and the headset posture information based on the first timestamp information and the second timestamp information to obtain fused posture information, adjust the different channel information in the audio data to be played based on the fused posture information, and play the adjusted audio data to be played.

[0019] In one embodiment, the second processor is further used to: determine the target user posture and target headphone posture at the same moment from the user posture information and the headphone posture information based on the first timestamp information and the second timestamp information; calibrate the zero drift of the IMU based on the target user posture and the target headphone posture, so as to re-determine the headphone posture information based on the IMU after the zero drift is calibrated.

[0020] After working for a period of time, the IMU is prone to zero drift. In the embodiment of the present application, the zero drift of the IMU is corrected using the user posture information determined by the charging box, which helps to improve the accuracy of the headphone posture information determined by the IMU, thereby improving the accuracy of the fused posture information, and further improving the accuracy of the adjustment of the audio data to be played, thereby improving the playback effect.

[0021] In one embodiment, the second wireless communication module is also used to receive the wireless frame and third timestamp information sent by the charging box; the true wireless headset also includes a second trigger circuit, and the second trigger circuit is used to generate fourth timestamp information after the second wireless communication module receives the wireless frame; the second processor is also used to: if the wireless frame includes the user posture information, the user posture information and the headset posture information are fused through the first difference and the second difference to obtain the fused posture information; the first difference is the difference between the first timestamp information and the third timestamp information, and the second difference is the difference between the second timestamp information and the fourth timestamp information.

[0022] In a third aspect, an embodiment of the present application provides an earphone system, comprising: a charging box as described in any one of the first aspects; and a true wireless earphone as described in any one of the second aspects, wirelessly connected to the charging box.

[0023] In a fourth aspect, an embodiment of the present application provides an audio playback method, which is applied to the earphone system as described in the third aspect; the audio playback method includes: capturing an image through the charging box; identifying user posture information in the captured image through the charging box; sending the user posture information to the true wireless earphone through the charging box; adjusting different channel information in the audio data to be played based on the user posture information through the true wireless earphone; and playing the adjusted audio data to be played through the true wireless earphone. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a schematic diagram of the structure of the charging box provided in one embodiment of the present application; Figure 2 This is a schematic diagram of the structure of a true wireless headset provided in one embodiment of the present application; Figure 3This is a flowchart of an audio playback method provided in one embodiment of the present application.

[0026] Icon: charging box 100; first wireless communication module 110; camera device 120; first processor 130; true wireless headset 200; second processor 210; second wireless communication module 220. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0028] First, an embodiment of the present application provides a charging box 100 and a true wireless headset 200. The charging box 100 is used to store the true wireless headset 200 and can also charge the true wireless headset 200.

[0029] In the embodiments of the present application, the charging box and the true wireless earphones are a set, and are usually sold to users as a set.

[0030] In the embodiments of the present application, the existing charging box and true wireless earphones can also be developed and upgraded with software and hardware, so that the existing charging box 100 and true wireless earphones 200 can realize the functions of the charging box 100 and true wireless earphones 200 provided in the embodiments of the present application. The functions of the charging box 100 and true wireless earphones 200 can be referred to later and will not be described in detail here.

[0031] See also Figure 1 , Figure 1 This is a schematic diagram of the charging box 100 provided in one embodiment of the present application.

[0032] The charging box 100 includes: a first wireless communication module 110, a camera device 120 and a first processor 130. In addition, the charging box 100 may also include Figure 1 Other structures not shown, such as the battery, charging port, charging contacts, and the storage cavity of the true wireless headset, are not expanded here. For details, please refer to the charging box 100 of the existing true wireless headset.

[0033] The first wireless communication module 110 is used for wireless communication with a true wireless headset.

[0034] In an embodiment of the present application, the first wireless communication module 110 may include a Bluetooth module that can implement at least one of classic Bluetooth, Bluetooth Low Energy, and Bluetooth HDT (Higher Data Throughput). In some embodiments, the first wireless communication module 110 may also include a WiFi (Wireless Fidelity) module to communicate with a true wireless headset via WiFi. The first wireless communication module 110 may also include a cellular communication module that can perform cellular communications such as 2G (2nd Generation Wireless Telephone Technology), 3G (3rd Generation Mobile Communication Technology), 4G (4th Generation Mobile Communication Technology), and 5G (5th Generation Mobile Communication Technology). The above description is merely illustrative and should not be construed as limiting the present application.

[0035] In some embodiments, the charging box 100 can be connected to a sound source device through the first wireless communication module 110, for example, connected to a mobile phone, computer, tablet computer and other devices, receive audio data sent by the sound source device and forward it to the true wireless headset 200.

[0036] The camera device 120 includes one or more cameras for capturing images. The captured images may or may not include the user. The captured images including the user can be used to identify the user's posture.

[0037] The first processor 130 may be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an MCU (Microcontroller Unit), or the like. In an embodiment of the present application, the first processor 130 is configured to receive images captured by the camera 120, identify user posture information in the images, and transmit the user posture information to the true wireless headset via the first wireless communication module 110. This allows the true wireless headset to adjust different channel information in the audio data to be played based on the user posture information and play the adjusted audio data to be played.

[0038] In the embodiment of the present application, the first processor 130 can perform posture recognition on the user in the captured image to obtain user posture information. Posture recognition includes identifying the user's position and posture. Position can also be called coordinates, and posture can also be called orientation, including roll angle, pitch angle, yaw angle, etc.

[0039] There are many ways to recognize posture, such as posture recognition based on calibration objects, posture recognition based on point cloud matching, posture recognition based on three-dimensional features or key point matching, etc. The first processor 130 of the charging box 100 can be configured with one or more posture recognition methods. The method of user posture recognition based on the collected image can be referred to the existing technology and will not be elaborated here.

[0040] In some embodiments of the present application, the first processor 130 may be configured with a pre-trained posture recognition model; the first processor 130 is configured to determine the user posture information in the captured image using the posture recognition model. The posture recognition model may also use an existing model, which will not be described in detail here.

[0041] In the embodiments of the present application, the posture recognition model is a neural network model that uses a large number of training samples during training, enabling it to accurately recognize user postures. Therefore, using a pre-trained posture recognition model to determine the user's posture in captured images helps improve the accuracy and efficiency of user posture recognition.

[0042] In an embodiment of the present application, no matter whether the user is indoors or outdoors, sitting, lying, standing or moving, if the user needs to adjust the sense of direction of the audio data to be played, the user can turn the shooting direction of the camera on the charging box 100 toward the user himself to capture an image including the user, and then determine the user's posture information to adjust the different channel information in the audio data to be played.

[0043] Among them, if the user is indoors, the charging box 100 can be placed on a stable place, such as on a table, and the camera can be pointed towards the user. In this way, the charging box 100 can collect the user's posture information at all times and send it to the true wireless headset, so that the true wireless headset can adjust the different channel information in the audio data to be played according to the user's posture information.

[0044] Posture recognition is usually performed through relative position relationships. Therefore, during posture recognition, the charging box 100 needs to determine its own posture and then identify the user posture information through the charging box posture. Therefore, in some embodiments of the present application, the camera device 120 can take multiple captured images, and the first processor 130 can also determine the charging box posture through synchronous positioning and map building technology and multiple captured images. Accordingly, the first processor 130 can also determine the user posture information in at least one captured image based on the charging box posture.

[0045] In some scenarios, the user may carry the charging box while moving. For example, if the user is outdoors, the user is likely to be moving. In this case, the user posture information identified by a single captured image may not be accurate enough. Therefore, in some embodiments of the present application, SLAM (Simultaneous Localization and Mapping, or synchronous positioning and mapping) can be used to identify the charging box posture of the charging box itself, so as to determine the relative posture relationship between the charging box and the user through the charging box posture and the captured image, and then determine the user posture information through the relative posture relationship.

[0046] Among them, the synchronous positioning and mapping technology can use the static objects in the captured image as reference objects to identify the motion state and posture of the image acquisition device itself. The synchronous positioning and mapping technology can refer to the existing technology and will not be expanded here. In the embodiment of the present application, through the synchronous positioning and mapping technology, the charging box 100 can identify its own motion state and posture even when it is in a moving state, so that a more accurate charging box posture can be determined. Therefore, the use of synchronous positioning and mapping technology can also help improve the accuracy of the charging box 100 in recognizing its own posture during movement. A more accurate charging box posture also helps to improve the accuracy of user posture recognition and obtain more accurate user posture information.

[0047] Therefore, in an embodiment of the present application, the method of identifying user posture information may include: 1. If the charging box 100 is in a stationary state, the charging box 100 captures an image through the camera, and the first processor 130 identifies at least one of the user, the user's head, or the true wireless headset worn by the user in the captured image to obtain user posture information.

[0048] 2. If the charging box 100 is moving, the charging box 100 can take multiple captured images through the camera. The first processor 130 first identifies the posture of the charging box through synchronous positioning and map construction technology and multiple captured images, and then determines the user posture information through the relationship between the charging box posture and the relative posture.

[0049] 3. If the charging box is moving, the charging box 100 can also determine the user posture information only through a single captured image. For example, the user posture information can be identified by identifying stationary objects, reference objects, etc. in the captured image. The implementation method can refer to the existing technology and will not be expanded here. Accordingly, in the embodiment of the present application, the user posture information sent by the charging box to the true wireless headset includes but is not limited to: posture information of the user's head or other parts of the user, posture information of the true wireless headset worn by the user, etc.

[0050] True wireless earphones usually have high requirements for size and power consumption. In an embodiment of the present application, a camera device 120 is provided on the charging box 100, which can effectively avoid the camera device 120 occupying the space of the true wireless earphones. At the same time, the charging box 100 identifies the user posture information and sends the user posture information to the true wireless earphones, rather than sending the captured image to the true wireless earphones. The charging box 100 has smaller requirements for circuit size, and it can be designed with a larger battery, which makes the charging box 100 have lower power consumption requirements and longer battery life. Therefore, the user posture recognition is performed by the charging box 100, rather than by transmitting the captured image and performing posture recognition by the true wireless earphones, which can effectively reduce or even avoid the situation where the true wireless earphones have excessive power consumption and reduced battery life due to posture recognition.

[0051] In an embodiment of the present application, the true wireless headset can refer to the existing technology for adjusting the different channel information in the audio data to be played based on the posture. For example, the true wireless headset can convert the audio signals of N channels in the HRTF (Head Related Transfer Function) model into audio signals of the corresponding channels of the left and right earphones based on the user posture information. For another example, the true wireless headset can adjust the delay, size, etc. of different audio signals in the audio data to be played based on the user posture information, so that the sound corresponding to the audio data to be played presents different directions and spatial senses. The specific implementation method can refer to the existing technology and will not be expanded here.

[0052] In some embodiments of the present application, the true wireless headset itself may include an IMU, which may include sensor elements such as a gyroscope and / or an accelerometer, which can be used to detect acceleration, velocity, angular velocity, and thus determine posture, position, etc. The implementation method of the IMU can refer to the existing technology and will not be elaborated here.

[0053] In an embodiment of the present application, the IMU can measure the headphone posture information of the true wireless headset itself, so that the true wireless headset can adjust the audio data to be played according to the headphone posture information.

[0054] In an embodiment of the present application, when the true wireless headset cannot obtain the user posture information from the charging box 100, for example, the captured image taken by the charging box 100 does not include the user, and the charging box 100 is away from the true wireless headset so that the wireless communication connection between the two is disconnected, the true wireless headset can detect its own headset posture information through the IMU, and then adjust the audio data to be played through the headset posture information.

[0055] True wireless earphones are worn on the user's ears, so the earphone posture information can also be used to characterize the user's posture. Therefore, in an embodiment of the present application, if the true wireless earphones can receive the user posture information sent by the charging box 100 and can also detect the earphone posture information through the IMU, the user posture information and the earphone posture information can be fused to obtain fused posture information, so as to adjust the different channel information in the audio data to be played based on the fused posture information.

[0056] In the embodiments of the present application, the fusion processing can be weighted calculation, Kalman filtering, etc., or a neural network model can be used to achieve the fusion of the two. There are many ways to implement it, which are not limited here.

[0057] The user's posture may change in real time. The charging box 100 and the true wireless earphones are two different devices, and the time information used by the two is inconsistent. Therefore, in the embodiment of the present application, when the user posture information and the earphone posture information are fused, the posture corresponding to the same moment should be fused.

[0058] Therefore, in some embodiments of the present application, the first processor 130 may further be configured to configure first timestamp information in the user posture information in response to the generation of the user posture information. The true wireless headset may also be configured to: determine the headset posture information through the IMU and configure second timestamp information for the headset posture information.

[0059] Therefore, the true wireless headset can be used to: in response to receiving the user posture information sent by the first wireless communication module 110, fuse the user posture information and the headset posture information based on the first timestamp information and the second timestamp information to obtain the fused posture information, and adjust the different channel information in the audio data to be played based on the fused posture information, so as to play the adjusted audio data to be played.

[0060] In an embodiment of the present application, the first timestamp information can be used to represent the generation time of the user posture information, and the second timestamp information can be used to represent the generation time of the headphone posture information. Therefore, based on the first timestamp information and the second timestamp information, the user posture information and the headphone posture information at the same time can be determined, and then the user posture information and the headphone posture information at the same time can be fused to obtain more accurate fused posture information, and then the audio data to be played is adjusted based on the fused posture information, so that the sound corresponding to the audio data to be played can be more directional.

[0061] In some embodiments of the present application, the charging box 100 further includes a first trigger circuit, which is connected to the first wireless communication module 110. The first trigger circuit is configured to generate third timestamp information after the first wireless communication module 110 sends a wireless frame to the true wireless headset, and the charging box sends the third timestamp information to the true wireless headset through the first wireless communication module 110.

[0062] Wireless frames can include audio data frames, control data frames, etc. User posture information is also sent to the true wireless earphones via wireless frames. Therefore, the wireless frames sent by the charging box may be frames containing user posture information. It is difficult for the first trigger circuit to distinguish the data types included in the wireless frames. Therefore, the first trigger circuit can generate the third timestamp information after any wireless frame is sent.

[0063] At the same time, the true wireless headset is further configured to: generate fourth timestamp information after receiving a wireless frame, and if the wireless frame includes the user posture information, fuse the user posture information and the headset posture information using the first difference and the second difference to obtain fused posture information. The first difference is the difference between the first timestamp information and the third timestamp information, and the second difference is the difference between the second timestamp information and the fourth timestamp information.

[0064] The timestamp is used to synchronize the two devices, that is, the first timestamp information and the second timestamp information are used to synchronize the charging box 100 and the true wireless earphones. However, the charging box 100 and the true wireless earphones are two different devices, and they use different clocks. The clocks of the two are not synchronized, and may even have different frequencies. In this case, the two cannot be used for direct comparison, that is, they cannot be used to determine the posture at the same time.

[0065] If the wireless frame includes user posture information, the third timestamp information can represent the moment when the charging box sends the user posture information, and the fourth timestamp information can represent the moment when the true wireless headset receives the user posture information. The third timestamp information and the fourth timestamp information are obtained at the same time or have a fixed delay. The first difference is the first timestamp information minus the third timestamp information, which actually represents the difference between the moment when the user posture information is generated and the reference time 1. Similarly, the second difference represents the difference between the moment when the headset posture information is generated and the reference time 2. Reference time 1 and reference time 2 are the same or have a fixed delay. Therefore, the first difference and the second difference can be directly used for comparison to determine the same time.

[0066] Therefore, in an embodiment of the present application, the posture at the same moment is determined based on the first difference and the second difference for fusion, and the influence of different clocks and frequencies of the two devices is taken into consideration. The first difference and the second difference will be able to more accurately determine the same moment, thereby improving the accuracy of the fused posture information.

[0067] Among them, there may be many reasons why the charging box 100 and the true wireless headset have different clocks and frequencies, including but not limited to hardware performance differences, errors generated during various uses, etc. Therefore, in the embodiment of the present application, if the charging box 100 and the true wireless headset are synchronized on the clock, so that the clocks of the two are synchronized, only the first timestamp information and the second timestamp information can be used to determine the fused posture information. If the clocks of the two are not synchronized, the third timestamp information and the fourth timestamp information can be combined for fusion.

[0068] In an embodiment of the present application, a camera device 120 is provided in the charging box 100 so that the charging box 100 can capture the user's image and identify the user's posture. Thus, after the charging box 100 sends the user's posture information to the true wireless headset, the true wireless headset can adjust the different channel information of the audio data to be played according to the user's posture information, so that the direction of the sound corresponding to the audio data to be played can change with the user's posture, bringing a better listening experience to the user. Among them, compared with the true wireless headset, using the charging box 100 to identify the user's posture, on the one hand, the size of the charging box 100 is larger, and the camera device 120 can be set in a larger size, avoiding the camera device 120 occupying the size of the true wireless headset. On the other hand, the charging box 100 can be designed with a larger battery to provide better battery life and has lower power consumption requirements. However, due to size limitations, true wireless headphones cannot be designed with a larger battery. Therefore, true wireless headphones have higher power consumption requirements. The charging box 100 of this application captures and collects images, identifies postures, and sends user posture information to the true wireless headphones, which can avoid transmitting collected images and avoid the true wireless headphones from identifying postures based on collected images, thereby reducing the power consumption of the true wireless headphones.

[0069] Based on the same inventive concept, the embodiment of the present application further provides a true wireless headset 200. In the embodiment of the present application, the true wireless headset 200 can be any one of an in-ear headset, a semi-in-ear headset, and an open-ear headset. The true wireless headset 200 can include at least one of a left earphone and a right earphone.

[0070] See also Figure 2 , Figure 2 A schematic diagram of the structure of a true wireless headset provided in an embodiment of the present application. In the embodiment of the present application, the true wireless headset 200 may include: a second wireless communication module 220 and a second processor 210.

[0071] In an embodiment of the present application, the first wireless communication module 110 and the second wireless communication module 220 are distinguished by first and second, and the two may include the same type of wireless communication modules. For example, when the first wireless communication module 110 includes a Bluetooth module, the second wireless communication module 220 also includes a Bluetooth module. Thus, the two can communicate based on the same type of wireless communication modules.

[0072] Accordingly, the second processor 210 and the first processor 130 may be the same or different types of processors, which is not limited here.

[0073] Among them, if the true wireless headset 200 includes a left earphone and a right earphone, the left earphone and the right earphone can both include a second processor 210, or only one of the left earphone and the right earphone includes the second processor 210, and the audio data to be played, posture information, etc. are transmitted between the two through wireless communication.

[0074] The true wireless headset 200 may also include other structures and circuits, such as a housing, a speaker, a filtering circuit, etc., which will not be described here. For details, please refer to the existing true wireless headset 200.

[0075] In an embodiment of the present application, the second wireless communication module 220 is used to connect to the charging box 100 provided in the aforementioned embodiment and receive user posture information.

[0076] The second processor 210 is connected to the second wireless communication module 220, and is used to adjust different channel information in the audio data to be played based on the user posture information, and play the adjusted audio data to be played.

[0077] In one embodiment, the user posture information also includes first timestamp information. The true wireless earphones 200 also include an IMU, and the second processor 210 is connected to the IMU; the IMU is used to detect the earphone posture information and configure the second timestamp information for the earphone posture information. The second processor 210 is further used to: based on the first timestamp information and the second timestamp information, fuse the user posture information and the earphone posture information to obtain fused posture information; adjust the different channel information in the audio data to be played based on the fused posture information; and play the adjusted audio data to be played.

[0078] In one embodiment, the second processor 210 is further used to: determine the target user posture and target headphone posture at the same moment from the user posture information and the headphone posture information based on the first timestamp information and the second timestamp information; calibrate the zero drift of the IMU based on the target user posture and the target headphone posture, so as to re-determine the headphone posture information based on the IMU after the zero drift is calibrated.

[0079] IMU zero drift, also known as zero-point drift, refers to the error in the pose and / or position measured by the IMU when no external measurement signals are received. This error is caused by drift and noise introduced by the IMU sensor itself. For example, internal mechanical structure or circuit noise can cause the IMU sensor's output signal to shift. Zero drift is a major issue with IMUs, reducing the sensor's accuracy and, in turn, affecting the accuracy and reliability of pose and / or position measurements.

[0080] The zero drift parameters may include the zero drift parameters of the IMU sensor accelerometer, such as the DC or low-frequency offset of the acceleration in each direction and / or the DC or low-frequency offset of its integral and / or the DC or low-frequency offset of its quadratic integral. The zero drift parameters may also include the zero drift parameters of the gyroscope in the IMU sensor, such as the DC or low-frequency offset of the angular velocity in each rotation direction and / or the DC or low-frequency offset of its integral (azimuth angle, attitude angle, or heading angle).

[0081] The zero drift of inertial sensors causes the cumulative error in IMU attitude and position estimation results to increase over time. For example, when a user wears a wearable device for extended periods of time, such as tens of minutes or even hours, the user's posture determined by the inertial sensors may deviate significantly from the user's actual posture, such as an azimuth deviation of 20 or 30 degrees. When the user listens to spatial audio, the audio's sense of orientation may shift significantly, affecting the user's listening experience.

[0082] Therefore, the zero drift of the IMU usually needs to be corrected to determine the accurate headphone posture information.

[0083] The user posture information determined by the charging box 100 is obtained by image detection. Compared with the IMU, the posture determined based on the image will not have a large error. Therefore, in the embodiment of the present application, the user posture information determined by the charging box 100 can be used to correct the zero drift of the true wireless headset 200 IMU, so that the IMU outputs more accurate headset posture information.

[0084] When calibrating zero drift, the target user posture and target headphone posture at the same moment can be determined from the user posture information and the headphone posture information through the first timestamp information and the second timestamp information, and then the difference between the target user posture and the target headphone posture can be determined. The zero drift parameters of the IMU are adjusted with the purpose of adjusting the target headphone posture to match the target user posture.

[0085] The method of correcting the zero drift of the IMU can refer to the existing technology, for example, using Kalman filtering, Euler angle method, quaternion method, equivalent rotation vector method, Rodrigues parameter method, direction cosine matrix method, etc. to calibrate the zero drift. For details, please refer to the existing technology and will not be expanded here.

[0086] In one embodiment, the second wireless communication module 220 is further configured to receive wireless frames and third timestamp information sent by the charging box 100. The true wireless earphones 200 further include a second trigger circuit configured to generate fourth timestamp information after the second wireless communication module receives the wireless frame. The second processor 210 is further configured to: if the wireless frame includes the user posture information, fuse the user posture information and the earphone posture information using a first difference and a second difference to obtain fused posture information; the first difference is the difference between the first timestamp information and the third timestamp information, and the second difference is the difference between the second timestamp information and the fourth timestamp information.

[0087] It can be understood that the functions implemented by the true wireless headset 200 provided in the embodiment of the present application are consistent with the functions implemented by the true wireless headset 200 in the aforementioned charging box 100 embodiment. Please refer to the previous text for details and will not be elaborated here.

[0088] Based on the same inventive concept, an embodiment of the present application also provides an earphone system, which includes: a charging box 100 and a true wireless earphone 200.

[0089] The specific structure and function of the charging box 100 and the true wireless earphones 200 can refer to the charging box 100 and the true wireless earphones 200 provided in the aforementioned embodiments and will not be elaborated here.

[0090] Based on the same inventive concept, the present embodiment also provides an audio playback method, which can be applied to the earphone system provided in the above embodiment. Figure 3 , Figure 3This is a flowchart of an audio playback method provided by one embodiment of the present application.

[0091] Audio playback methods include: S310: Capture and collect images through the charging box.

[0092] S320: Identify user posture information in the captured image through the charging box.

[0093] S330: Send the user posture information to the true wireless headset through the charging box.

[0094] S340: Adjust different channel information in the audio data to be played based on the user posture information through the true wireless headset.

[0095] S350 plays the adjusted audio data to be played through the true wireless headset.

[0096] In one embodiment, capturing a captured image through the charging box 100 includes: capturing multiple captured images through the charging box 100. Identifying user posture information in the captured images through the charging box 100 includes: the charging box 100 determining the charging box posture using synchronous positioning and mapping technology and multiple captured images; and the charging box 100 determining the user posture information in at least one captured image based on the charging box posture.

[0097] In one embodiment, the first processor 130 of the charging box 100 is configured with a pre-trained posture recognition model, and recognizing the user posture information in the captured image through the charging box 100 includes: the charging box 100 determines the user posture information in the captured image through the posture recognition model.

[0098] In one embodiment, after the charging box 100 identifies the user posture information in the captured image, the method further includes: the charging box 100 configures first timestamp information in the user posture information in response to the generation of the user posture information.

[0099] The true wireless headset 200 includes an inertial measurement unit IMU; before adjusting the different channel information in the audio data to be played based on the user posture information through the true wireless headset 200, the method further includes: determining the headset posture information through the IMU of the true wireless headset 200 and configuring a second timestamp information for the headset posture information, and in response to receiving the user posture information sent by the first wireless communication module 110, fusing the user posture information and the headset posture information based on the first timestamp information and the second timestamp information to obtain fused posture information. Adjusting the different channel information in the audio data to be played based on the user posture information through the true wireless headset 200 includes: adjusting the different channel information in the audio data to be played based on the fused posture information through the true wireless headset 200, and playing the adjusted audio data to be played.

[0100] In one embodiment, the charging box 100 further includes a first trigger circuit connected to the first wireless communication module 110. After the user posture information is sent to the true wireless earphones 200 via the charging box 100, the method further includes: generating third timestamp information after the first wireless communication module sends a wireless frame to the true wireless earphones, and the charging box sends the third timestamp information to the true wireless earphones 200 via the first wireless communication module 110. The method includes: generating fourth timestamp information after receiving the wireless frame, and if the wireless frame includes the user posture information, fusing the user posture information and the earphone posture information using a first difference and a second difference to obtain the fused posture information; the first difference is the difference between the first timestamp information and the third timestamp information, and the second difference is the difference between the second timestamp information and the fourth timestamp information.

[0101] In one embodiment, before fusing the user posture information and the headphone posture information based on the first timestamp information and the second timestamp information to obtain the fused posture information, the method further includes: determining, by the true wireless headset 200, the target user posture and target headphone posture at the same time from the user posture information and the headphone posture information based on the first timestamp information and the second timestamp information; calibrating the zero drift of the IMU based on the target user posture and the target headphone posture by the true wireless headset 200, so as to re-determine the headphone posture information based on the IMU after the zero drift is calibrated.

[0102] In the embodiments provided herein, it should be understood that the disclosed methods and devices may also be implemented in other ways. The device embodiments described above are merely illustrative. The functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0103] The above embodiments can be freely combined without conflict, and the embodiments obtained by the combination are included in the protection scope of this application.

[0104] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0105] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A charging box, characterized in that: The charging box is used to charge the true wireless headset; the charging box includes: a first wireless communication module, a camera device and a first processor; the first processor is connected to the camera device and the first wireless communication module respectively; The first processor is configured to: Receiving a captured image captured by the camera device; Identifying user posture information in the captured image; The user posture information is sent to the true wireless headset through the first wireless communication module, so that the true wireless headset adjusts different channel information in the audio data to be played based on the user posture information, and plays the adjusted audio data to be played.

2. The charging box according to claim 1, characterized in that The captured images taken by the camera device include multiple images; The first processor is configured to: Determine the position of the charging box through synchronous positioning and mapping technology and multiple collected images; Determine the user posture information in at least one of the captured images based on the posture of the charging box.

3. The charging box according to claim 1, characterized in that The first processor is configured with a pre-trained posture recognition model; The first processor is used to determine the user posture information in the captured image through the posture recognition model.

4. The charging box according to any one of claims 1 to 3, characterized in that: The first processor is further configured to configure first timestamp information in the user posture information in response to the generation of the user posture information; The true wireless headset includes an inertial measurement unit IMU; The true wireless earphones are also used for: Determine the headphone posture information by using the IMU and configure second timestamp information for the headphone posture information, In addition, in response to receiving the user posture information sent by the first wireless communication module, the user posture information and the headphone posture information are fused based on the first timestamp information and the second timestamp information to obtain fused posture information, and different channel information in the audio data to be played is adjusted based on the fused posture information, and the adjusted audio data to be played is played.

5. The charging box according to claim 4, characterized in that: The charging box further includes a first trigger circuit, and the first trigger circuit is connected to the first wireless communication module; The first trigger circuit is configured to generate third timestamp information after the first wireless communication module sends a wireless frame to the true wireless headset, and the charging box sends the third timestamp information to the true wireless headset through the first wireless communication module; The true wireless headset is further configured to: generate fourth timestamp information after receiving the wireless frame; if the wireless frame includes the user posture information, fuse the user posture information and the headset posture information through the first difference and the second difference to obtain the fused posture information; The first difference is the difference between the first timestamp information and the third timestamp information, and the second difference is the difference between the second timestamp information and the fourth timestamp information.

6. A true wireless headset, characterized in that: include: a second wireless communication module, configured to connect to the charging box according to any one of claims 1 to 5 and receive the user posture information; The second processor is connected to the second wireless communication module, and is used to adjust different channel information in the audio data to be played based on the user posture information, and play the adjusted audio data to be played.

7. The true wireless headset according to claim 6, characterized in that The user posture information also includes first timestamp information; The true wireless headset also includes an IMU, and the second processor is connected to the IMU; the IMU is used to detect headset posture information and configure second timestamp information for the headset posture information; The second processor is also used to: fuse the user posture information and the headphone posture information based on the first timestamp information and the second timestamp information to obtain fused posture information, adjust the different channel information in the audio data to be played based on the fused posture information, and play the adjusted audio data to be played.

8. The true wireless headset according to claim 7, characterized in that: The second processor is further configured to: Determining a target user posture and a target headphone posture at the same moment from the user posture information and the headphone posture information based on the first timestamp information and the second timestamp information; The zero drift of the IMU is calibrated based on the target user posture and the target headphone posture, so as to re-determine the headphone posture information based on the IMU after the zero drift is calibrated.

9. The true wireless headset according to claim 7 or 8, characterized in that: The second wireless communication module is further used to receive the wireless frame and third timestamp information sent by the charging box; The true wireless headset further includes a second trigger circuit, which is configured to generate fourth timestamp information after the second wireless communication module receives the wireless frame sent by the charging box; The second processor is further configured to: if the wireless frame includes the user posture information, fuse the user posture information and the earphone posture information by using the first difference and the second difference to obtain the fused posture information; The first difference is the difference between the first timestamp information and the third timestamp information, and the second difference is the difference between the second timestamp information and the fourth timestamp information.

10. A headphone system, characterized in that: include: The charging box according to any one of claims 1 to 5; The true wireless headset according to any one of claims 6 to 9 is wirelessly connected to the charging box.

11. An audio playback method, characterized in that: Applicable to the earphone system according to claim 10; the audio playback method comprises: Capture and collect images through the charging box; Identifying user posture information in the captured image through the charging box; Sending the user posture information to the true wireless headset through the charging box; Adjusting different channel information in the audio data to be played based on the user posture information by the true wireless headset; The adjusted audio data to be played is played through the true wireless headset.

Images