Audio processing method, apparatus, system, and electronic device

Abstract

The present disclosure provides an audio processing method, device, system and electronic device. The method comprises: receiving an audio switching instruction, wherein the audio switching instruction is used to instruct to switch a playing effect of target audio data to a spatial sound effect; acquiring a motion parameter detected by a motion sensor; processing the target audio data according to a current playing thread of the target audio data and the motion parameter; obtaining spatial audio data corresponding to the target audio data, and playing the spatial audio data. When receiving an instruction of a user to switch an audio playing effect, the audio data is converted into a spatial sound effect based on a current audio playing thread of the user, so that switching between threads is avoided, and the fluency of audio playing is improved.

Description

Technical Field

[0001] This disclosure relates to the field of audio processing technology, and in particular to an audio processing method, apparatus, system and electronic device. Background Technology

[0002] In related technologies, when a user switches the audio playback effect to spatial sound, the CPU (Central Processing Unit) needs to switch from the current audio playback thread to a dedicated thread for spatial sound. Since switching between threads takes time, audio stuttering can easily occur during this process, especially noticeable under high CPU load. Summary of the Invention

[0003] In view of the above, this disclosure provides an audio processing method, apparatus, system and electronic device.

[0004] According to a first aspect of the present disclosure, an audio processing method is provided, the method comprising:

[0005] Receive an audio switching instruction, wherein the audio switching instruction is used to instruct the playback effect of the target audio data to be switched to spatial sound effects;

[0006] Acquire motion parameters detected by the motion sensor;

[0007] The target audio data is processed based on the current playback thread and the motion parameters.

[0008] Obtain the spatial audio data corresponding to the target audio data, and play the spatial audio data.

[0009] In one embodiment, processing the target audio data according to the current playback thread and the motion parameters includes:

[0010] The motion parameters are sent outside the current playback thread, and the target audio data is processed according to the motion parameters to obtain spatial audio data.

[0011] In one embodiment, processing the target audio data according to the motion parameters to obtain the spatial audio data includes:

[0012] Based on the motion parameters, determine the direction of the sound source in the target audio data relative to the head of the target object;

[0013] Based on the direction of the sound source in the target audio data relative to the head of the target object, determine the corresponding processing function;

[0014] The target audio data is processed according to the processing function to obtain the spatial audio data.

[0015] In one embodiment, playing the spatial audio data includes:

[0016] The spatial audio data is sent to an audio playback device so that the audio playback device can play the spatial audio data.

[0017] In one embodiment, acquiring the motion parameters detected by the motion sensor includes at least one of the following:

[0018] The motion parameters sent by the communication module in the audio playback device are obtained, wherein the motion parameters are collected by a motion sensor in the audio playback device and transmitted to the audio playback device;

[0019] The motion parameters sent by the motion sensor are acquired.

[0020] In one embodiment, the motion sensor includes a physical communication module and / or a wireless communication module, wherein the physical communication module is used to establish a communication connection with the audio playback device, and the wireless communication module is used to establish a communication connection with the audio playback device and / or the processing device.

[0021] According to a second aspect of the present disclosure, an audio processing apparatus is provided, the apparatus comprising:

[0022] An acquisition unit is used to receive an audio switching instruction, wherein the audio switching instruction is used to instruct the playback effect of the target audio data to be switched to spatial sound effects;

[0023] The detection unit is used to detect motion parameters via motion sensors.

[0024] The processing unit is configured to process the target audio data according to the current playback thread and the motion parameters.

[0025] The playback unit is used to obtain the spatial audio data corresponding to the target audio data and play the spatial audio data.

[0026] In one embodiment, the processing unit is configured to:

[0027] The motion parameters are sent outside the current playback thread, and the target audio data is processed according to the motion parameters to obtain spatial audio data.

[0028] In one embodiment, the processing unit is configured to:

[0029] Based on the motion parameters, determine the direction of the sound source in the target audio data relative to the head of the target object;

[0030] Based on the direction of the sound source in the target audio data relative to the head of the target object, determine the corresponding processing function;

[0031] The target audio data is processed according to the processing function to obtain spatial audio data.

[0032] In one embodiment, the device further includes:

[0033] A transmitting unit is configured to transmit the spatial audio data to the audio playback device, so that the audio playback device plays the spatial audio data.

[0034] In one embodiment, the acquisition unit is configured to:

[0035] The motion parameters sent by the communication module in the audio playback device are acquired, wherein the motion parameters are collected by a motion sensor in the audio playback device and transmitted to the audio playback device.

[0036] The motion parameters sent by the motion sensor are acquired.

[0037] In one embodiment, the external motion sensor includes a physical communication module and / or a wireless communication module. The physical communication module is used to establish a communication connection with the audio playback device, and the wireless communication module is used to establish a communication connection with the audio playback device and / or the processing device.

[0038] According to a third aspect of the present disclosure, an audio processing system is provided, the system comprising:

[0039] A motion sensor, used to collect motion parameters;

[0040] An audio playback device is used to play spatial audio data, and a motion sensor is disposed on the audio playback device, wherein the spatial audio data is obtained by processing the target audio data according to the motion parameters.

[0041] In some embodiments, the motion sensor is disposed inside the housing of the audio playback device, or the motion sensor is disposed outside the housing of the audio playback device.

[0042] In some embodiments, the motion sensor transmits the motion parameters to the audio playback device via a wired transmission module, or transmits the motion parameters to the audio playback device and / or processing device via a wireless transmission module.

[0043] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described in the first aspect above.

[0044] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising:

[0045] processor;

[0046] Memory used to store processor-executable instructions;

[0047] Wherein, the processor executes the steps of any of the methods described in the first aspect above.

[0048] According to a sixth aspect of the present disclosure, a computer program product is provided that, when executed by a processor, implements the steps of any of the methods described in the first aspect above.

[0049] The technical solutions provided in this disclosure may have the following beneficial effects:

[0050] When a user sends a command to switch audio playback effects, the system can acquire motion parameters of the user's head collected by a motion sensor in real time. Based on the user's current audio playback thread, the system processes the target audio data according to the motion parameters, converting the target audio data into spatial sound effects. This avoids switching between threads and improves the smoothness of audio playback.

[0051] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram illustrating spatial audio processing according to an embodiment of the present disclosure;

[0054] Figure 2 This is a flowchart illustrating an audio processing method according to an embodiment of the present disclosure;

[0055] Figure 3 This is a schematic diagram illustrating an audio playback device and a motion sensor according to an embodiment of this disclosure;

[0056] Figure 4 This is a structural diagram of a motion sensor shown in an embodiment of this disclosure;

[0057] Figure 5 This is a schematic diagram illustrating one method of acquiring motion parameters according to an embodiment of this disclosure. Figure 1 ;

[0058] Figure 6 This is a schematic diagram illustrating one method of acquiring motion parameters according to an embodiment of this disclosure. Figure 2 ;

[0059] Figure 7 This is a schematic diagram illustrating one method of acquiring motion parameters according to an embodiment of this disclosure. Figure 3 ;

[0060] Figure 8 This is a schematic diagram illustrating an embodiment of the present disclosure for acquiring motion parameters;

[0061] Figure 9 This is a flowchart illustrating an embodiment of generating spatial audio data;

[0062] Figure 10 This is a schematic diagram illustrating an embodiment of generating spatial audio data according to this disclosure;

[0063] Figure 11 This is a flowchart illustrating a spatial audio data processing method according to an embodiment of this disclosure;

[0064] Figure 12 This is a block diagram of an audio processing apparatus shown in an embodiment of the present disclosure;

[0065] Figure 13 This is a hardware structure diagram of an electronic device shown in an embodiment of this disclosure. Detailed Implementation

[0066] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. Furthermore, the technical features involved in the different embodiments of this disclosure described below can be combined with each other as long as they do not conflict with each other.

[0067] Spatial audio is a revolutionary audio technology designed to create an immersive listening experience by simulating a three-dimensional sound environment, allowing listeners to perceive sound coming from all directions, including in front, behind, above, below, and all around. Its core characteristics include directionality, a sense of distance, dynamic movement, and a sense of height. By simulating a three-dimensional sound environment, spatial audio enhances the sense of direction, distance, and dynamics of sound, thus creating an immersive listening experience. It combines advanced hardware and software technologies and is widely used in games, audio-visual media, and virtual reality, bringing users a more realistic audio experience.

[0068] Converting ordinary 2D audio data into spatial sound effects requires motion sensors to acquire the user's head position parameters in real time. Based on these parameters, the 2D audio data is then converted into spatial sound effects with a 3D effect. In related technologies, after acquiring the user's head position parameters from the motion sensor, the spatial audio synthesis is performed in the CPU. This audio data processing scheme consumes CPU resources. For CPUs with slightly lower performance, under high load conditions, audio stuttering, noise, and other issues may occur due to the CPU's slow processing speed, affecting the user's audio experience. For details on spatial audio processing, please refer to [link to relevant documentation]. Figure 1 The diagram illustrates that when a user selects to enable spatial audio, the electronic device switches the audio playback thread from the 2D sound effects playback thread to the 3D sound effects playback thread within the CPU. Audio data processing is then completed on the basis of the 3D sound effects playback thread. Furthermore, this switching of playback threads takes time, and the switching speed is affected by CPU performance. Under high load, if the playback thread switching process is lengthy, audio stuttering can easily occur, impacting the user experience.

[0069] Based on this, this disclosure provides an audio processing method that can be applied to electronic devices such as mobile phones, tablets, smartwatches, smart headphones, and smart speakers to improve the playback effect of spatial audio. Please refer to the following for details. Figure 2 The flowchart shown includes the following steps:

[0070] S201 receives audio switching commands.

[0071] The audio switching command is used to instruct the playback effect of the target audio data to be switched to spatial sound effects. The target audio data is the audio data to be played. After the user selects to switch the effect of the target audio data, electronic devices such as mobile phones can receive the audio switching command initiated by the user.

[0072] S202, acquire motion parameters detected by the motion sensor.

[0073] The motion sensor can be a gyroscope, angular velocity sensor, or other motion sensor. The motion parameters are the head motion parameters of the target object while wearing the audio playback device during audio playback. The target object is the user of the electronic device, and the target object is wearing an audio playback device, such as headphones. The motion parameters are the head motion parameters of the target object, used to determine the head position of the target object.

[0074] The limitations of related technologies in acquiring user head motion parameters are significant. Generally, only headphones with head tracking capabilities can collect user head position parameters in real time. Ordinary Bluetooth or wired headphones cannot collect user head position parameters, and therefore cannot send these parameters to the electronic device maintaining a communication connection with the headphones. Consequently, the headphones cannot acquire and play spatial audio with spatial sound effects. Therefore, this application provides a method for flexibly configuring motion sensors in headphones. The motion sensor is disposed inside the housing of the audio playback device, or the motion sensor is disposed outside the housing of the audio playback device.

[0075] When the motion sensor is installed inside the housing of the audio playback device, the motion sensor can transmit the collected motion parameters of the target object to the audio playback device, and the audio playback device can then send the motion parameters of the target object to the electronic device.

[0076] Some playback devices do not have built-in motion sensors, making it impossible for them to collect user head movement parameters. To address this, this disclosure provides a motion sensor that can be externally mounted to the audio playback device. The motion sensor is independent of the audio playback device. For example, the motion sensor can be attached to the outside of the audio playback device or clip-on to it.

[0077] In some embodiments, the configuration relationship between the audio playback device and the motion sensor can be as follows: Figure 3 As shown in the diagram. The audio playback device can be a Bluetooth headset, and the motion sensors are ring-shaped, with at least two sensors, one on the left earpiece and one on the right earpiece. To ensure that the motion sensors can transmit the collected motion parameters to other devices, the motion sensors are equipped with a data transmission module. In one embodiment, the structure of the motion sensor is as follows: Figure 4As shown, the motion sensor includes at least a detection module, a data transmission module, and a power supply module. The detection module is used to detect the motion parameters of the target object's head, the data transmission module is used to transmit the motion parameters to the audio playback device or the electronic device, and the power supply module is used to provide power to the motion sensor.

[0078] The data transmission module in the motion sensor may include a wired transmission module and / or a wireless transmission module. The wired transmission module is used to establish a communication connection with the audio playback device, and the wireless transmission module is used to establish a communication connection with the audio playback device and / or a processing device. The processing device is a device that processes target audio data and generates spatial audio data.

[0079] If the data transmission module is a wired transmission module, physical interfaces can be provided in the audio playback device and the motion sensor. For example, the audio playback device has a first physical interface, and the motion sensor has a second physical interface, with the first physical interface matching the second physical interface. The physical interface can be a communication interface such as a USB interface. The first physical interface can be provided on a Bluetooth headset, for example, on the earcups of an over-ear headphone, which can be a wireless or wired headset.

[0080] If the data transmission module is a wireless transmission module, the motion sensor can communicate with the audio playback device or the electronic device via wireless communication technology. When the motion sensor communicates wirelessly with the audio playback device, it can send the motion parameters of the target object to the audio playback device, which then sends them to the electronic device via wired or wireless communication. When the motion sensor communicates wirelessly with the electronic device, it can send the motion parameters of the target object to the electronic device.

[0081] In other words, electronic devices acquire user head motion parameters through at least one of the following methods:

[0082] Obtain the motion parameters sent by the communication module in the audio playback device;

[0083] The motion parameters sent by the motion sensor are acquired.

[0084] The electronic device may include a wireless communication module or a wired communication module. Several embodiments of the electronic device acquiring motion parameters will be described below.

[0085] like Figure 5As shown, the motion sensor can send motion parameters to the audio playback device via wired communication, and the audio playback device can also send motion parameters to the electronic device via wired communication. The electronic device can then obtain the motion parameters sent by the communication module in the audio playback device through its wired communication module.

[0086] Or such as Figure 6 As shown, the motion sensor can send motion parameters to the audio playback device via wired communication, and the audio playback device can wirelessly send the motion parameters to the electronic device. The electronic device can then obtain the motion parameters sent by the communication module in the audio playback device through its wireless communication module.

[0087] Or such as Figure 7 As shown, the motion sensor can transmit motion parameters to the audio playback device via wireless communication, and the audio playback device can also transmit motion parameters to the electronic device wirelessly. The electronic device can then obtain the motion parameters transmitted by the communication module in the audio playback device through its wireless communication module.

[0088] Or like Figure 8 As shown, the motion sensor can also directly send motion parameters to the electronic device via wireless communication. The electronic device can then obtain the motion parameters sent by the motion sensor in the audio playback device through the wireless communication module.

[0089] According to the above embodiments, when the user changes the playback effect of the target audio data from ordinary 2D sound effects to 3D sound effects, or when the user directly selects to play the target audio data with 3D sound effects, the electronic device can obtain the motion parameters sent by the audio playback device or motion sensor in real time.

[0090] S203, Process the target audio data according to the current playback thread of the target audio data and the motion parameters.

[0091] In this embodiment of the disclosure, after the electronic device acquires the motion parameters of the user's head, it does not need to switch audio playback threads. The playback effect can be converted based on the current playback thread of the target audio data. In some embodiments, the electronic device can send the target audio data to a dedicated chip for processing audio data within the current playback thread. This dedicated chip can be a system-on-a-chip (SoC) of the electronic device or a newly added audio processing chip. The motion parameters are then also sent to the dedicated chip, which converts the target audio data into spatial audio data based on the motion parameters.

[0092] In some embodiments, the motion parameters can be sent outside the current playback thread, and the target audio data can be processed according to the motion parameters to obtain spatial audio data.

[0093] Motion parameters and target audio data are sent outside the current playback thread; that is, the target audio data is processed based on the motion parameters outside the CPU to obtain spatial audio data. In one embodiment, a spatial audio data module can be set up, independent of the CPU, and can be set up in a dedicated chip in the electronic device. Processing the target audio data in the spatial audio data module to obtain spatial audio data can save CPU resource consumption, thereby reducing power consumption and improving device performance.

[0094] In one embodiment, the process of the electronic device generating spatial audio data can be found in [reference needed]. Figure 9 The flowchart shown includes:

[0095] S901, based on the current playback thread of the target audio data, the target audio data is transmitted to the spatial audio data module.

[0096] S902, based on the data transmission module, transmits the motion parameters to the spatial audio data module.

[0097] S903, Based on the spatial audio data module, the target audio data is processed according to the motion parameters to obtain the spatial audio data.

[0098] The spatial audio data module is independent of the playback thread and is located in a dedicated processing chip for the audio data. For example... Figure 10 As shown, the data transmission module for transmitting motion parameters includes the Sensor HAL interface in the hardware abstraction layer, the Sensor Service module in the application framework layer, the AudioManager module in the audio management layer, and the chip interface in the hardware abstraction layer. The current playback thread of the target audio data may include the audio presentation module playback in the application framework layer and the audio playback interface in the hardware abstraction layer.

[0099] Electronic devices can transmit target audio data to the spatial audio data module via the current playback thread of the target audio data, and can also transmit motion parameters to the spatial audio data module via the data transmission module. The spatial audio data module can convert the target audio data into spatial audio data based on the received motion parameters of the target object and the target audio data using spatial audio processing algorithms.

[0100] In one embodiment, the spatial audio data processing procedure is as follows: Figure 11 As shown, it specifically includes:

[0101] S1101, Based on the motion parameters, determine the direction of the sound source in the target audio data relative to the head of the target object.

[0102] S1102, Based on the direction of the sound source in the target audio data relative to the head of the target object, determine the corresponding processing function.

[0103] S1103, The target audio data is processed according to the processing function to obtain the spatial audio data.

[0104] In the spatial audio data module, the HRTF (Head Related Transfer Functions) algorithm can be used to process the target audio data. HRTF is a set of filters that utilizes techniques such as binaural time difference and auricular frequency vibration to create a surround sound effect when the generated audio data is transmitted to the auricle, ear canal, and tympanic membrane. Because HRTF is a set of filters, the spatial audio data module can be integrated into the DSP (Digital Signal Processor) module of the main processing chip. The DSP module can then perform convolution calculations and filtering on the audio data.

[0105] HRTF simulates the spatial orientation of sound by dynamically adjusting the spectral characteristics of the sound source and combining it with the user's real-time head position parameters. Therefore, the spatial audio data module first transforms the coordinate system of the motion parameters sent by motion sensors (gyroscopes, accelerometers), converting the head posture data into a three-dimensional rotation matrix or quaternion to update the direction of the sound source relative to the head. In spatial sound effects, each sound source has fixed coordinates (e.g., footsteps in a game are located at (x, y, z)). Based on the current head orientation, the sound source coordinates are transformed to a coordinate system centered on the head using a rotation matrix, resulting in a new azimuth angle (azimuth θ, elevation φ). Then, the HRTF database pre-measures acoustic filtering data in different directions (e.g., sampled every 5°), with each direction containing two sets of filters for the left and right ears. Based on the current relative direction of the sound source, the closest HRTF pair is selected. Finally, convolution processing is performed on the HRTF pairs, convolving the mono or stereo audio with the HRTF filters for the left and right ears respectively, generating binaural signals to obtain spatial audio data.

[0106] S204, obtain the spatial audio data corresponding to the target audio data, and play the spatial audio data.

[0107] After obtaining the spatial audio data corresponding to the target audio data, the spatial audio data can be sent to an audio playback device (e.g., headphones) so that the audio playback device can play the spatial audio data. By playing spatial audio data with time and intensity differences through the left and right earpieces of the headphones, the listener can achieve an immersive listening experience.

[0108] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should know that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps may be performed in other orders or simultaneously.

[0109] Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by this disclosure.

[0110] Corresponding to the aforementioned application function implementation method embodiments, this disclosure also provides embodiments of application function implementation apparatus and corresponding terminals.

[0111] Reference Figure 12 According to an exemplary embodiment, an audio processing apparatus block diagram is shown, the apparatus may include:

[0112] The acquisition unit 1201 is used to receive an audio switching instruction, wherein the audio switching instruction is used to instruct the playback effect of the target audio data to be switched to spatial sound effects;

[0113] Detection unit 1202 is used to detect motion parameters through a motion sensor;

[0114] Processing unit 1203 is used to process the target audio data according to the current playback thread and the motion parameters.

[0115] The playback unit 1204 is used to obtain the spatial audio data corresponding to the target audio data and play the spatial audio data.

[0116] In one embodiment, the processing unit 1203 is configured to:

[0117] The motion parameters are sent outside the current playback thread, and the target audio data is processed according to the motion parameters to obtain spatial audio data.

[0118] In one embodiment, the processing unit 1203 is configured to:

[0119] Based on the motion parameters, determine the direction of the sound source in the target audio data relative to the head of the target object;

[0120] Based on the direction of the sound source in the target audio data relative to the head of the target object, determine the corresponding processing function;

[0121] The target audio data is processed according to the processing function to obtain spatial audio data.

[0122] In one embodiment, the device further includes:

[0123] A sending unit is configured to send the spatial audio data to the audio playback device, so that the audio playback device plays the spatial audio data to the target object.

[0124] In one embodiment, the acquisition unit 1201 is configured to:

[0125] The motion parameters sent by the communication module in the audio playback device are acquired, wherein the motion parameters are collected by a motion sensor in the audio playback device and transmitted to the audio playback device.

[0126] The motion parameters sent by the motion sensor are acquired.

[0127] In one embodiment, the external motion sensor includes a physical communication module and / or a wireless communication module. The physical communication module is used to establish a communication connection with the audio playback device, and the wireless communication module is used to establish a communication connection with the audio playback device or the processing device.

[0128] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0129] This disclosure also provides an embodiment of an audio processing system, the system comprising:

[0130] A motion sensor, used to collect motion parameters;

[0131] An audio playback device is used to play spatial audio data. The motion sensor is disposed on the audio playback device. The spatial audio data is obtained by a processing device processing the target audio data according to the motion parameters.

[0132] The motion sensor can collect motion parameters of the user's head to determine the position of the user's head. The audio playback device can be headphones, used to play spatial audio data generated by processing devices such as mobile phones. The method for generating the spatial audio data can be found in the solutions provided in the audio processing method embodiments.

[0133] In some embodiments, the motion sensor is disposed inside the housing of the audio playback device, or the motion sensor is disposed outside the housing of the audio playback device.

[0134] The motion sensor can be built into the audio playback device or installed independently. When the motion sensor is installed independently outside the audio playback device, it can communicate with the audio playback device via a wired transmission module and / or a wireless transmission module, and can also communicate with an electronic device that processes the target audio data via a wireless transmission module.

[0135] In some embodiments, the motion sensor transmits the motion parameters to the audio playback device via a wired transmission module, or transmits the motion parameters to the audio playback device and / or processing device via a wireless transmission module.

[0136] When a motion sensor includes a wired transmission module, the motion sensor can send motion parameters to an audio playback device via the wired transmission module, and then the audio playback device can transmit the motion parameters to a processing device that processes the target audio data. For example... Figure 5 The diagram shows that the motion sensor can transmit motion parameters to the headphones via a wired communication module, and then the headphones can transmit the motion parameters to the electronic device via wired communication. For example... Figure 6 As shown, the motion sensor can transmit motion parameters to the headphones via wired communication using a wired transmission module, and then the headphones can transmit the motion parameters to the electronic device via wireless communication.

[0137] When a motion sensor includes a wireless transmission module, the motion sensor can transmit motion parameters to an audio playback device via the wireless transmission module, and then the audio playback device can transmit the motion parameters to a processing device that processes the target audio data. For example... Figure 7 As shown, the motion sensor can transmit motion parameters to the headphones via wireless communication using the wireless transmission module, and then the headphones can transmit the motion parameters to the electronic device via wireless communication.

[0138] Alternatively, motion sensors can wirelessly transmit motion parameters to a processing device, allowing the processing device to process the target audio data based on these parameters. For example... Figure 8As shown, the motion sensor can transmit motion parameters directly to the electronic device via wireless communication using a wireless transmission module.

[0139] Figure 13 The diagram shows a structural block diagram of an electronic device according to some embodiments of the present disclosure. The following is a description of the structure of the device in conjunction with the provided text. Figure 13 Some embodiments of the electronic device described herein will be explained.

[0140] Reference Figure 13 The electronic device 1300 may include one or more of the following components: processing component 1302, memory 1304, power supply component 1306, multimedia component 1308, audio data component 1310, input / output (I / O) interface 1312, sensor component 1316, and communication component 1313.

[0141] Processing component 1302 typically controls the overall operation of electronic device 1300, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1302 may include one or more processors 1320 to execute instructions. Furthermore, processing component 1302 may include one or more modules to facilitate interaction between processing component 1302 and other components. For example, processing component 1302 may include a multimedia module to facilitate interaction between multimedia component 1308 and processing component 1302. As another example, processing component 1302 may read executable instructions from memory to implement relevant functions of the electronic device.

[0142] Memory 1304 is configured to store various types of data to support the operation of electronic device 1300. Examples of such data include instructions for any application or method operating on electronic device 1300, contact data, phonebook data, messages, pictures, videos, etc. Memory 1304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0143] Power supply component 1306 provides power to various components of electronic device 1300. Power supply component 1306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1300.

[0144] The multimedia component 1308 includes a display screen that provides an output interface between the electronic device 1300 and the user. In some embodiments, the multimedia component 1308 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0145] Audio data component 1310 is configured to output and / or input audio data signals. For example, audio data component 1310 includes a microphone (MIC) configured to receive external audio data signals when electronic device 1300 is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio data signals may be further stored in memory 1304 or transmitted via communication component 1313. In some embodiments, audio data component 1310 also includes a speaker for outputting audio data signals.

[0146] I / O interface 1312 provides an interface between processing component 1302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0147] Sensor assembly 1316 includes one or more sensors for providing state assessments of various aspects of electronic device 1300. For example, sensor assembly 1316 may detect the on / off state of electronic device 1300, the relative positioning of components such as the display and keypad of electronic device 1300, changes in position of electronic device 1300 or a component of electronic device 1300, the presence or absence of user contact with electronic device 1300, the orientation or acceleration / deceleration of electronic device 1300, and temperature changes of electronic device 1300. Sensor assembly 1316 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1316 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1316 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0148] Communication component 1313 is configured to facilitate wired or wireless communication between electronic device 1300 and other devices. Electronic device 1300 can access wireless networks based on communication standards, such as Wi-Fi, 2G, 3G, 4G, 5G, or 6G, or combinations thereof. In one exemplary embodiment, communication component 1313 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1313 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0149] In an exemplary embodiment, the electronic device 1300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0150] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the embodiments. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this disclosure.

Claims

1. An audio processing method, characterized in that, The method includes: Receive an audio switching instruction, wherein the audio switching instruction is used to instruct the playback effect of the target audio data to be switched to spatial sound effects; Acquire motion parameters detected by the motion sensor; The target audio data is processed based on the current playback thread and the motion parameters. Obtain the spatial audio data corresponding to the target audio data, and play the spatial audio data.

2. The method according to claim 1, characterized in that, The step of processing the target audio data based on the current playback thread and the motion parameters includes: The motion parameters are sent outside the current playback thread, and the target audio data is processed according to the motion parameters to obtain spatial audio data.

3. The method according to claim 2, characterized in that, The step of processing the target audio data according to the motion parameters to obtain spatial audio data includes: Based on the motion parameters, determine the direction of the sound source in the target audio data relative to the head of the target object; Based on the direction of the sound source in the target audio data relative to the head of the target object, determine the corresponding processing function; The target audio data is processed according to the processing function to obtain spatial audio data.

4. The method according to claim 1, characterized in that, Playing the spatial audio data includes: The spatial audio data is sent to an audio playback device so that the audio playback device can play the spatial audio data.

5. The method according to claim 1, characterized in that, The acquisition of motion parameters detected by the motion sensor includes at least one of the following: The motion parameters sent by the communication module in the audio playback device are acquired, wherein the motion parameters are collected by a motion sensor in the audio playback device and transmitted to the audio playback device. The motion parameters sent by the motion sensor are acquired.

6. The method according to claim 1, characterized in that, The motion sensor includes a wired transmission module and / or a wireless transmission module. The wired transmission module is used to establish a communication connection with the audio playback device, and the wireless transmission module is used to establish a communication connection with the audio playback device and / or the processing device.

7. An audio processing device, characterized in that, The device includes: An acquisition unit is used to receive an audio switching instruction, wherein the audio switching instruction is used to instruct the playback effect of the target audio data to be switched to spatial sound effects; The detection unit is used to detect motion parameters through a motion sensor; The processing unit is configured to process the target audio data according to the current playback thread and the motion parameters. The playback unit is used to obtain the spatial audio data corresponding to the target audio data and play the spatial audio data.

8. The apparatus according to claim 7, characterized in that, The processing unit is used for: The motion parameters are sent outside the current playback thread, and the target audio data is processed according to the motion parameters to obtain spatial audio data.

9. An audio processing system, characterized in that, The system includes: A motion sensor, used to collect motion parameters; An audio playback device is used to play spatial audio data, and a motion sensor is disposed on the audio playback device, wherein the spatial audio data is obtained by processing target audio data according to the motion parameters.

10. The system according to claim 9, characterized in that, The motion sensor may be disposed inside the housing of the audio playback device, or the motion sensor may be disposed outside the housing of the audio playback device.

11. The system according to claim 9, characterized in that, The motion sensor transmits the motion parameters to the audio playback device via a wired transmission module, or transmits the motion parameters to the audio playback device and / or processing device via a wireless transmission module.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method described in any one of claims 1 to 6.

13. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor executes the steps of any one of the methods described in claims 1 to 6.

Images