A method, apparatus and electronic device for playing audio

Abstract

The application is suitable for the field of multimedia technology, and provides a playing method and device of audio and electronic equipment, the method comprises the following steps: controlling an analog sound source to move on a preset sound surround track, so as to prompt a user to move the neck according to the sound surround track; the analog sound source is simulated by loudspeakers located on both sides of the user's ears; in the process of moving the analog sound source on the sound surround track, collecting head movement information of the user; and generating a score report based on the head movement information and the sound surround track. The technical scheme provided by the application does not require the user to constantly focus on the screen, but can move according to the relative position relationship between the analog sound source and itself to a specified direction and a specified track, thereby reducing the operation difficulty of the user, so that the operation of the neck movement is more flexible, and the use experience of the user is improved.

Description

Technical Field

[0001] This application belongs to the field of multimedia technology, and in particular relates to an audio playback method, apparatus, and electronic device. Background Technology

[0002] As people's living standards continue to improve, users are also paying more attention to their physical health. To properly perform stretching exercises during rest periods, users can play demonstration videos of corresponding exercises on their mobile phones, such as neck exercise videos, and follow along with the content of the demonstration videos.

[0003] However, with existing multimedia playback technology, users often need to constantly pay attention to the demonstration video on the screen when they need to exercise. Especially when neck stretching exercises are required, constantly paying attention to the video content makes it impossible to fully exercise the neck, thus greatly reducing the effectiveness of neck stretching exercises and making the operation more difficult. Summary of the Invention

[0004] This application provides an audio playback method, apparatus, electronic device, and computer-readable storage medium, which can solve the problem that existing multimedia playback technologies often require users to constantly pay attention to the demonstration video on the screen when they need to exercise, especially when they need to stretch their necks. Constantly paying attention to the video content makes it impossible to fully exercise the neck, thus greatly reducing the effectiveness of neck stretching exercises and making the operation difficult.

[0005] In a first aspect, embodiments of this application provide an audio playback method, including:

[0006] The simulated sound source is controlled to move along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory; the simulated sound source is simulated by speakers located on both sides of the user's ears;

[0007] As the simulated sound source moves along the sound surround trajectory, the user's head movement information is collected;

[0008] A scoring report is generated based on the head movement information and the sound surround trajectory.

[0009] The implementation of this application's embodiments has the following beneficial effects: By configuring corresponding speakers on both sides of the user's ears, a simulated sound source is generated based on the binaural effect. By controlling the simulated sound source to move along a preset sound surround track, the purpose of prompting the user to perform neck movements is achieved. Furthermore, during the user's neck movements, i.e., as the simulated sound source moves along the sound surround track, the user's head movement information is collected. Finally, based on the head movement information and the aforementioned sound surround track, an evaluation score for the user's neck movement is obtained, achieving the goal of screenless guided activity. Compared with existing multimedia playback technologies, this application's embodiments can simulate a corresponding simulated sound source through the binaural effect. By controlling the simulated sound source to move around and guide the user's activities, the user does not need to constantly stare at the screen. Instead, the user can move in a specified direction and along a specified trajectory based on the relative positional relationship between the simulated sound source and themselves, reducing the difficulty of operation for the user, making neck movement operations more flexible, and improving the user experience.

[0010] In one possible implementation of the first aspect, the step of collecting the user's head movement information during the movement of the simulated sound source along the sound surround trajectory includes:

[0011] Control the simulated sound source to move to the Nth key position point on the sound surround trajectory; where N is a positive integer.

[0012] Collect the user's head posture when the simulated sound source moves to the Nth key position point;

[0013] Based on the head pose and the Nth key position point, the target audio of each of the speakers is determined; the target audio is used for the audio segment during the process of the simulated sound source moving from the Nth key position point to the N+1th key position point;

[0014] Increase the value of N. If the value of N after the increase is less than the total number of key position points on the sound surround trajectory, then return to the operation of collecting the user's head posture after the simulated sound source moves to the Nth key position point.

[0015] If the value of N after the increase is greater than or equal to the total number, then the head movement information is generated based on all the head poses.

[0016] In one possible implementation of the first aspect, determining the target audio for each of the speakers based on the head pose and the Nth key position point includes:

[0017] Based on the user's initial pose when the simulated sound source moves to the Nth key location point and the Nth key location point, determine the first distance value between the user's head and the Nth key location point;

[0018] Determine the second distance value between the head pose and the Nth key location point;

[0019] Based on the first distance value and the second distance value, a first audio segment is determined; the first audio segment is used to simulate the relative position change between the user's head and the simulated sound source;

[0020] Based on the Nth key position point, the (N+1)th key position point, and the head pose, a second audio segment is determined; the second audio segment is used to simulate the simulated audio moving from the Nth key position point to the (N+1)th key position point;

[0021] The target audio is generated based on the first audio segment and the second audio segment.

[0022] In one possible implementation of the first aspect, generating a scoring report based on the head movement information and the sound surround trajectory includes:

[0023] Based on multiple head poses within the head activity information, the user's head movement trajectory is generated during the process of the simulated sound source moving on the sound surround trajectory.

[0024] Calculate the trajectory similarity between the head movement trajectory and the sound surround trajectory;

[0025] The user's activity range is determined based on the head movement trajectory;

[0026] The scoring report is generated based on the trajectory similarity and the activity range.

[0027] In one possible implementation of the first aspect, selecting applications with a startup confidence level greater than a preset confidence threshold as candidate applications, and determining the set of background running applications based on the candidate applications, includes:

[0028] Import all the candidate applications and the runtime environment information into the decision algorithm corresponding to each runtime environment dimension, and calculate the first application corresponding to the runtime environment dimension;

[0029] Based on the first application corresponding to all runtime environment dimensions, the set of background running applications is generated.

[0030] In one possible implementation of the first aspect, after generating the scoring report based on the head movement information and the sound surround trajectory, the method further includes:

[0031] Based on the activity range, determine the actual range of the user's activity in multiple activity directions;

[0032] If the actual activity amplitude corresponding to any activity direction is less than a preset amplitude threshold, then the any activity direction is identified as an enhanced activity direction.

[0033] Neck exercise prompts are generated based on all the described directions of increased activity.

[0034] In one possible implementation of the first aspect, before the controlled simulated sound source moves along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory, the method further includes:

[0035] Based on a preset list of tutorial prompts, play the Mth tutorial prompt message; the initial value of M is 1;

[0036] Obtain the user's actual movement information based on the Mth tutorial prompt;

[0037] If the actual motion information matches the standard motion information associated with the Mth tutorial prompt, a preset success prompt sound is output, and the value of M is increased.

[0038] If the value of M is less than or equal to the total number of tutorial prompts in the tutorial prompt list, then return to the preset tutorial prompt list and play the Mth tutorial prompt.

[0039] If the value of M is greater than the total number of tutorial prompts, a preset completion prompt sound will be output.

[0040] In one possible implementation of the first aspect, it is applied to a head-mounted device; the head-mounted device includes an earphone component and an inertial sensor; the earphone component includes speakers on both sides of the ears; the inertial sensor is used to collect head movement information.

[0041] Secondly, embodiments of this application provide an audio playback device, including:

[0042] A simulated sound source movement unit is used to control the simulated sound source to move along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory; the simulated sound source is simulated by speakers located on both sides of the user's ears;

[0043] The head movement information acquisition unit is used to acquire the user's head movement information as the simulated sound source moves along the sound surround trajectory.

[0044] The scoring report output unit is used to generate a scoring report based on the head movement information and the sound surround trajectory.

[0045] In one possible implementation of the second aspect, the head activity information acquisition unit includes:

[0046] A key position point moving unit is used to control the simulated sound source to move to the Nth key position point on the sound surround trajectory; where N is a positive integer;

[0047] A head position and pose determination unit is used to collect the head position and pose of the user when the simulated sound source moves to the Nth key position point.

[0048] The target audio determination unit is used to determine the target audio of each of the speakers based on the head posture and the Nth key position point; the target audio is used for the audio segment of the simulated sound source during the process of moving from the Nth key position point to the N+1th key position point;

[0049] The loop unit is used to increase the value of N. If the value of N after the increase is less than the total number of key position points on the sound surround trajectory, then the operation of collecting the user's head posture after the simulated sound source moves to the Nth key position point is returned.

[0050] The head activity information encapsulation unit is used to generate the head activity information based on all the head poses if the value of N after the increase is greater than or equal to the total number.

[0051] In one possible implementation of the second aspect, the target audio determination unit includes:

[0052] The first distance value determination unit is used to determine the first distance value between the user's head and the Nth key position point based on the user's initial pose when the simulated sound source moves to the Nth key position point and the Nth key position point;

[0053] The second distance value determination unit is used to determine the second distance value between the head pose and the Nth key position point;

[0054] The first audio segment determination unit is used to determine a first audio segment based on the first distance value and the second distance value; the first audio segment is used to simulate the relative position change between the user's head and the simulated sound source;

[0055] The second audio segment determination unit is used to determine a second audio segment based on the Nth key position point, the (N+1)th key position point, and the head pose; the second audio segment is used to simulate the simulated audio moving from the Nth key position point to the (N+1)th key position point;

[0056] The target audio encapsulation unit is used to generate the target audio based on the first audio segment and the second audio segment.

[0057] In one possible implementation of the second aspect, the scoring report output unit includes:

[0058] The head motion trajectory generation unit is used to generate the user's head motion trajectory during the process of the simulated sound source moving on the sound surround trajectory, based on multiple head poses in the head activity information.

[0059] A trajectory similarity calculation unit is used to calculate the trajectory similarity between the head movement trajectory and the sound surround trajectory;

[0060] An activity range determination unit is used to determine the user's activity range based on the head movement trajectory.

[0061] The rating report generation unit is used to generate the rating report based on the trajectory similarity and the activity range.

[0062] In one possible implementation of the second aspect, the audio playback device further includes:

[0063] The actual activity range determination unit is used to determine the actual activity range of the user in multiple activity directions based on the activity range.

[0064] The enhanced activity direction determination unit is used to identify any activity direction as an enhanced activity direction if the actual activity amplitude corresponding to any activity direction is less than a preset amplitude threshold.

[0065] A neck exercise prompting unit is used to generate neck exercise prompting information based on all of the said directions of increased activity.

[0066] In one possible implementation of the second aspect, the audio playback device further includes:

[0067] The tutorial prompt unit is used to play the Mth tutorial prompt message based on a preset tutorial prompt list; the initial value of M is 1.

[0068] The actual motion information acquisition unit is used to acquire the actual motion information of the user based on the Mth tutorial prompt information;

[0069] A success prompt tone output unit is used to output a preset success prompt tone and increase the value of M if the actual motion information matches the standard motion information associated with the Mth tutorial prompt information.

[0070] The execution unit is used to return to the preset tutorial prompt list and play the Mth tutorial prompt if the value of M is less than or equal to the total number of tutorial prompts in the tutorial prompt list.

[0071] The completion prompt unit is used to output a preset completion prompt sound if the value of M is greater than the total number of tutorial prompt messages.

[0072] In one possible implementation of the second aspect, it is applied to a head-mounted device; the head-mounted device includes an earphone component and an inertial sensor; the earphone component includes speakers on both sides of the ears; the inertial sensor is used to collect head movement information.

[0073] Thirdly, embodiments of this application provide an electronic device, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the audio playback method described in any of the first aspects above.

[0074] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, characterized in that, when executed by a processor, the computer program implements the audio playback method described in any of the first aspects above.

[0075] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the audio playback method described in any of the first aspects above.

[0076] In a sixth aspect, embodiments of this application provide a chip system including a processor coupled to a memory, the processor executing a computer program stored in the memory to implement an audio playback method as described in any of the first aspects.

[0077] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0078] Figure 1 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0079] Figure 2 This is a software structure block diagram of an electronic device according to an embodiment of this application;

[0080] Figure 3 This is a schematic diagram of the existing neck movement process;

[0081] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0082] Figure 5 This is a flowchart illustrating the implementation of an audio playback method according to an embodiment of this application;

[0083] Figure 6 This is a schematic diagram illustrating the initiation of a neck relaxation command according to an embodiment of this application;

[0084] Figure 7 This is a schematic diagram illustrating the selection of audio files according to an embodiment of this application;

[0085] Figure 8 This is a schematic diagram illustrating the initiation of a neck relaxation command according to another embodiment of this application;

[0086] Figure 9 This is a flowchart illustrating the implementation of an audio playback method according to an embodiment of this application;

[0087] Figure 10 This is a schematic diagram illustrating a tutorial prompt provided in one embodiment of this application;

[0088] Figure 11 This is a flowchart illustrating the specific implementation of S502 in an audio playback method provided in another embodiment of this application;

[0089] Figure 12 This is a schematic diagram of the division of a sound surround track according to an embodiment of this application;

[0090] Figure 13 This is a schematic diagram illustrating the determination of an audio segment according to an embodiment of this application;

[0091] Figure 14 This is a schematic diagram showing a scoring report provided in an embodiment of this application;

[0092] Figure 15 This is a flowchart illustrating the specific implementation of S503 in an audio playback method provided in another embodiment of this application;

[0093] Figure 16 This is a schematic diagram of neck exercise prompts provided in an embodiment of this application;

[0094] Figure 17 This is a structural block diagram of the audio playback device provided in the embodiments of this application;

[0095] Figure 18 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0096] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0097] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0098] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0099] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0100] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0101] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0102] The audio playback method provided in this application embodiment can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). In particular, the audio playback method can be applied to electronic scales or electronic devices with body detection functions. This application embodiment does not impose any restrictions on the specific type of electronic device.

[0103] For example, the electronic device may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a computer, a laptop computer, a handheld communication device, a handheld computing device, and / or other devices for communicating over a wireless system, as well as next-generation communication systems, such as mobile terminals in 5G networks or mobile terminals in future evolved Public Land Mobile Network (PLMN) networks.

[0104] Figure 1 A schematic diagram of an electronic device 100 is shown.

[0105] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0106] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0107] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0108] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0109] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0110] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0111] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0112] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0113] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0114] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0115] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

[0116] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0117] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0118] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0119] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0120] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0121] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0122] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0123] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0124] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0125] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0126] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0127] Electronic device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information. Specifically, the display screen 194 can display the generated detection report, allowing the user to view the report.

[0128] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1. Display screen 194 may include a touch panel and other input devices.

[0129] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0130] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0131] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0132] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0133] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0134] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0135] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0136] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0137] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0138] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0139] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A. Specifically, the speaker 170A can be used to output prompts to notify the user of the part of the electronic scale that needs to be touched.

[0140] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0141] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0142] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0143] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194, for example, electronic devices can obtain the user's weight through pressure sensor 180A. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the intensity of the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with a touch operation intensity less than a first pressure threshold is applied to the SMS application icon, a command to view SMS messages is executed. When a touch operation with a strength greater than or equal to the first pressure threshold is applied to the SMS application icon, the instruction to create a new SMS message is executed.

[0144] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0145] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0146] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0147] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0148] A distance sensor 180F is used to measure distance. Electronic device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0149] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 may use the proximity sensor 180G to detect when a user holds the electronic device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0150] The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0151] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0152] Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs thermal protection by reducing the performance of a processor located near temperature sensor 180J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 100 heats battery 142 to prevent abnormal shutdown of electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0153] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0154] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0155] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0156] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0157] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0158] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0159] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0160] Figure 2 This is a software structure block diagram of an electronic device according to an embodiment of this application.

[0161] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime system layer, and the kernel layer.

[0162] The application layer can include a series of application packages.

[0163] like Figure 2 As shown, the application package can include applications such as camera, calendar, map, WLAN, Bluetooth, music, video, SMS, email, WeChat, and WPS.

[0164] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0165] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0166] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0167] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0168] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0169] A phone manager is used to provide communication functions for electronic devices. For example, it manages call status (including connection and disconnection).

[0170] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0171] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0172] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0173] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0174] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0175] The system layer can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0176] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0177] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0178] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0179] A 2D graphics engine is a graphics engine for 2D drawing.

[0180] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0181] The following example, using a scene of capturing a photograph, illustrates the workflow of the software and hardware of the electronic device 100.

[0182] When touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking a touch click as an example, where the corresponding control is the camera application icon, the camera application calls the application framework layer's interface to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through camera 193.

[0183] Nowadays, health issues are receiving increasing attention, especially with the rising pressures of modern life. Prolonged desk work, computer use, or long drives often lead to maintaining a single posture for extended periods, increasing the likelihood of cervical spondylosis. To help reduce the incidence of cervical spondylosis by regularly moving the neck, users can watch relevant videos and follow the instructions to move and relax their necks. For example, Figure 3 A schematic diagram of the existing neck movement process is shown. See also Figure 3As shown, when users watch guided videos on handheld electronic devices, they need to keep their eyes focused on the screen, which limits the range of motion in their necks and prevents them from achieving a good neck relaxation effect. On the other hand, users often hope to relax both mentally and physically while exercising their necks. If they need to constantly focus on the guided video on the screen, they need to concentrate, making it impossible to fully relax their minds and further reducing the user experience. Therefore, it is clear that existing methods for neck stretching exercises are difficult to operate and result in a poor user experience.

[0184] Example 1:

[0185] Therefore, to address the shortcomings of existing methods for guiding users in neck stretching exercises, which suffer from high operational difficulty and poor user experience, this application provides an audio playback method. The execution entity of this audio playback method can be an electronic device containing at least two speakers, with at least one speaker positioned on each side of the user's ears. This utilizes the binaural effect to simulate a sound source. The speakers and the electronic device can be connected via wired or wireless communication.

[0186] For example, the electronic device can be a smartphone, computer, tablet, etc. In this case, the electronic device can be connected to an external headphone component, which can be a wired headphone or a wireless headphone. Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of this application is shown. See also Figure 4 As shown in (a), the electronic device is a smartphone. This smartphone can establish a communication connection with an earphone component via a Bluetooth communication module, or alternatively, via a serial interface. The specific connection can be determined based on the actual situation. See also Figure 4 As shown in (b) in this embodiment, the speaker can be an external speaker, with at least one external speaker placed on both sides of the user's ears. Of course, in order to achieve a better simulation effect, two or more external speakers can be added. The smartphone can establish a communication connection with the external speaker device group composed of multiple external speakers via Bluetooth.

[0187] For example, the electronic device can also be a head-mounted device, such as wireless headphones, true wireless earbuds, or over-ear headphones, which has two internal speakers, i.e., two earpieces. In this case, the user can wear the head-mounted device to achieve neck movement guidance, such as... Figure 4 As shown in (c), the user does not need to hold a smartphone. The head-mounted device can be configured with a processing module to execute the audio playback method provided in this embodiment.

[0188] Figure 5 A flowchart illustrating the implementation of an audio playback method according to an embodiment of this application is shown, and is described in detail below:

[0189] In S501, the simulated sound source is controlled to move along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory; the simulated sound source is simulated by speakers located on both sides of the user's ears.

[0190] In this embodiment, to guide neck movement, the electronic device can simulate a sound source using two speakers. Due to the binaural effect, by controlling the loudness and frequency of the audio located on both sides of the ears, as well as filtering directional sounds, the perceived location of the sound source can be controlled. In this case, the electronic device can use the principle of the binaural effect to control the simulated sound source to move along a preset sound surround track.

[0191] In one possible implementation, the electronic device can respond to a user-initiated neck relaxation command and control a simulated sound source to move along a preset sound surround trajectory, prompting the user to move their neck according to the sound surround trajectory, i.e., initiating a neck relaxation process. For example, Figure 6 A schematic diagram illustrating the initiation of a neck relaxation command according to an embodiment of this application is shown. See also Figure 6 As shown in (a), the electronic device is a smartphone with a "Health" application 61 installed. If the user clicks on the application 61, the Health interface can be launched, such as... Figure 6 As shown in (b), the operation interface of this sports and health system includes various functional controls, including a neck relaxation control 62. If the user clicks on the above-mentioned functional control 62, it is recognized that the user has initiated a neck relaxation command, and the operation of S501 is executed. Preferably, in order to facilitate the user to select a suitable speaker for neck relaxation guidance, the smartphone can generate an audio device selection interface after detecting that the user clicks on the functional control 62. This selection interface includes multiple selectable audio playback methods, including local playback and external device playback. For external device playback, the corresponding device identifier can be displayed, such as... Figure 6 As shown in (c), the selection interface contains several options: local playback 63, Freebuds playback 64 (i.e., playback via wireless Bluetooth headphones), and external speaker playback 65. Users can select the appropriate audio device for audio playback according to their actual situation.

[0192] Furthermore, as another embodiment of this application, the electronic device may include multiple different audio files, each corresponding to a different neck movement guidance audio. For example, different audio files may correspond to different ranges of motion, durations, and exercise focuses. The electronic device may display multiple different audio files, allowing the user to select the appropriate target file for playback based on the actual activity situation. For example, Figure 7 This diagram illustrates the selection of an audio file according to an embodiment of this application. See also... Figure 7 As shown in (a), after the user selects the desired audio device, they can enter the audio file selection interface. This interface contains several selectable audio files: a light relaxation track 1, a deep relaxation track 2, a track with a focus on relaxation on the left side 3, and a track with a focus on relaxation on the right side 4. The user can select the corresponding track according to their actual situation. For example, in this embodiment, the user can click the control corresponding to the deep relaxation track 2 and play the audio file. Figure 7 (b) in the middle.

[0193] It should be noted that, in addition to allowing users to select playback files by default, users can also play audio files sequentially according to a preset playback order. For example, after detecting that the user has selected the corresponding audio device, the electronic device can directly enter the audio playback interface, i.e. Figure 7 (b) skips the audio file selection interface, i.e., skips... Figure 7 In (a) of this case, the audio playback interface includes controls for switching tracks, such as control 71 for switching to the previous track, control 72 for switching to the next track, and control 73 for viewing the track list. Of course, the audio playback interface also includes a switching mode selection control 74, which allows users to select from various switching modes such as playlist-based playback, shuffle playback, and single-track loop playback, and users can choose according to the actual situation.

[0194] Figure 8 This illustration shows a schematic diagram of initiating a neck relaxation command according to another embodiment of this application. The electronic device is a head-mounted device containing multiple operation buttons, such as a volume up button 81, a volume down button 82, and a function key 83. When the device detects that a user has pressed the function key 83, it enters a function selection mode. At this time, the user can initiate a neck relaxation command to the head-mounted device via voice command, such as saying "neck relaxation." The head-mounted device collects the user's voice command, extracts the keywords contained within the voice command, and determines the corresponding operation command based on the keywords. Alternatively, the user can select the corresponding function by continuously pressing the function key 83 a specified number of times.

[0195] In one possible implementation, the electronic device can be configured with multiple different playback modes. These playback modes include, but are not limited to, a normal audio playback mode and a neck relaxation playback mode. In normal audio playback mode, the electronic device plays the audio file according to the original data of each sound track within the audio file. In neck relaxation mode, the electronic device can adjust the original data of each sound track based on the aforementioned sound surround track, causing the simulated sound sources within each sound track to move according to a preset sound surround track, generating surround movement data, and executing the operation S501 based on the surround movement data.

[0196] Furthermore, as another embodiment of this application, Figure 9 A flowchart illustrating the implementation of an audio playback method according to an embodiment of this application is shown. See also... Figure 9 As shown, with Figure 5 Compared to the illustrated embodiment, the embodiment of this application further includes the following steps before S501, which are described in detail below:

[0197] Furthermore, before controlling the simulated sound source to move along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory, the method further includes:

[0198] In S901, based on a preset list of tutorial prompts, the Mth tutorial prompt message is played; the initial value of M is 1.

[0199] In this embodiment, since the neck movement guidance process is only guided by audio, users may not be able to directly understand how to perform neck movements when they first encounter it. In this case, the electronic device can output corresponding tutorial prompts according to a preset tutorial prompt list to prompt the user, so that the user can learn how to follow the simulated sound source to perform neck movements in the subsequent process.

[0200] For example, the tutorial prompts could be "Please move closer to the analog sound source during audio playback," "Increased volume indicates approaching the analog sound source," and "Decreased volume indicates moving away from the analog sound source." Users can determine the correspondence between the audio change rules and the relative position of the analog sound source based on the different tutorial prompts, so as to follow the neck movements subsequently.

[0201] In this embodiment, the tutorial prompt list can contain multiple tutorial prompts. Different tutorial prompts have a corresponding prompt order, and are arranged sequentially within the tutorial prompt list according to this order. When the electronic device broadcasts the prompts, it can do so sequentially based on the prompt order of the various pieces of information in the tutorial prompt list.

[0202] In one possible implementation, the electronic device can record the number of times the user initiates neck movements. If the user initiates a neck relaxation command for the first time, the device can execute operation S901 to teach the user the pre-activity process. After the teaching process is completed, operation S501 is executed to perform the neck movement.

[0203] In one possible implementation, the tutorial prompts for the user can be initiated not only when the user first uses the neck movement exercise, but also when the user's scores for multiple neck movement exercises are low. Specifically, the evaluation report contains a score for the neck movement exercise. If the user's scores in multiple consecutive evaluation reports are lower than a preset score threshold, it indicates that the user is unable to follow the simulated sound source according to the preset sound surround trajectory, and a tutorial prompt is required again. In this case, operation S901 is executed.

[0204] In one possible implementation, the electronic device can obtain the identity information of the user performing the neck movement, such as through biometric technology like facial recognition, or based on account information such as the logged-in user's account, to determine the number of times the user has performed the neck movement. If the number of movements is detected to be 0, it means that the user is performing the neck movement for the first time. At this time, the operation S901 will be executed to provide an activity tutorial prompt.

[0205] In S902, the user's actual movement information based on the Mth tutorial prompt information is obtained.

[0206] In this embodiment, when the electronic device plays the Mth tutorial prompt, it can prompt the user to perform activities according to the tutorial prompt. In this case, the electronic device can obtain the user's actual movement information based on the Mth tutorial prompt. The electronic device is equipped with a motion sensor, which can acquire the user's head movement and generate corresponding actual movement information based on the user's head movement within a preset effective time period. This actual movement information may include information such as head movement trajectory, movement direction, movement speed, and movement acceleration. The information items collected for different tutorial prompts may differ, specifically determined by the tutorial content; that is, the information items included in the actual movement information corresponding to different tutorial prompts may be different.

[0207] In this embodiment, each tutorial prompt can be associated with corresponding standard motion information. The electronic device can calculate the matching between the standard motion information and the actual motion information to determine whether the user should perform the specified action according to the tutorial prompt. If the two match, operation S903 is executed; otherwise, if the two do not match, operation S906 is executed.

[0208] In one possible implementation, the aforementioned standard motion information includes corresponding motion feature values. The electronic device can calculate the difference between the aforementioned motion feature values ​​and the actual motion values ​​in the aforementioned actual motion information. If the difference between the two is less than a preset deviation threshold, then the two are identified as matching. Conversely, if the difference between the motion feature values ​​and the actual motion values ​​is greater than or equal to the deviation threshold, then the actual motion information and the standard motion information are identified as not matching.

[0209] In S903, if the actual motion information matches the standard motion information associated with the Mth tutorial prompt information, a preset success prompt sound is output, and the value of M is increased.

[0210] In this embodiment, when the electronic device detects that the user's actual motion information matches the standard motion information associated with the longer prompt information, it indicates that the user has performed the corresponding action according to the tutorial prompt information. At this time, the corresponding success prompt sound can be output to notify the user that the tutorial has been completed and the next tutorial can be started. Therefore, the value of M is increased to announce the next tutorial prompt information in the tutorial prompt list.

[0211] In S904, if the value of M is less than or equal to the total number of tutorial prompts in the tutorial prompt list, then return to the preset tutorial prompt list and play the Mth tutorial prompt.

[0212] In S905, if the value of M is greater than the total number of tutorial prompts, a preset completion prompt sound is output.

[0213] In this embodiment, if the value of M is less than or equal to the total number of tutorial prompts after increasing the value of M, then the operation of playing the next tutorial prompt is returned until all tutorial prompts in the tutorial list have been played, that is, the value of M is greater than the total number of tutorial prompts. At this time, the corresponding completion prompt sound can be output to remind the user that the teaching process has been completed. At this time, the corresponding neck movement guidance process can be executed, that is, the operation of S501 is executed.

[0214] In S906, if the actual motion information does not match the standard motion information associated with the Mth tutorial prompt information, a preset failure prompt sound is output, and the operation of playing the Mth tutorial prompt information based on the preset tutorial prompt list is returned.

[0215] In this embodiment, if the electronic device detects that the user's actual movement information does not match the standard movement information associated with the tutorial prompts, it indicates that the user has not performed the corresponding movement according to the tutorial prompts. This may be because the user has questions about the tutorial content and does not understand the actions instructed in the tutorial. In this case, a failure prompt sound can be output to inform the user that the tutorial has failed, and the tutorial prompts can be replayed so that the user can hear the tutorial content clearly again and perform the corresponding actions.

[0216] For example, Figure 10 A schematic diagram illustrating a tutorial prompt provided in one embodiment of this application is shown. See also Figure 10 In example (a), the simulated sound source is located to the user's left, and the tutorial prompt is "Move your head closer to the simulated sound source." If the user moves their head to the left, they will move closer to the simulated sound source. At this point, the system can identify whether the user's actual movement information matches the standard movement information and output a success prompt sound, such as... Figure 10 As shown in (b), the next tutorial prompt can be played at this point; conversely, if the user controls their head to move to the right, they will move away from the simulated sound source. In this case, the system can identify the mismatch between the user's actual movement information and the standard movement information, and output a failure prompt sound, such as... Figure 10 As shown in (c), the tutorial prompt message "Move your head closer to the simulated sound source" is played again.

[0217] In this embodiment, before the user performs neck relaxation activities, corresponding tutorial prompts are played so that the user can follow the movement of the simulated sound source during subsequent neck activities, thereby greatly improving the success rate of subsequent neck activities and increasing efficiency.

[0218] In S502, as the simulated sound source moves along the sound surround trajectory, the user's head movement information is collected.

[0219] In this embodiment, during the playback of a simulated sound source, the simulated sound source moves along a preset sound surround track. The user controls their head to move accordingly in accordance with the movement of the simulated sound source. To determine whether the user's head movement matches the preset sound surround track, the electronic device can acquire the user's head movement information. This head movement information includes information such as the direction, speed, angle, and amplitude of the head movement.

[0220] In one possible implementation, the electronic device can establish a communication connection with a distributed camera module, acquire user activity video containing the user's head through the camera module, perform facial recognition on the user activity video, locate the user's face region (i.e., head region), thereby obtaining the coordinates of the head region in each frame of video image and determining the head activity information of the head region.

[0221] In one possible implementation, the electronic device is specifically a head-mounted device, which includes an earphone component and an inertial sensor. The earphone component includes speakers on both sides of the ears. The inertial sensor is used to collect head movement information. The head-mounted device can be a wireless headset, a virtual reality (VR) headset, a head massager, or similar device. The head-mounted device is equipped with earphone components, each of which includes a corresponding speaker to control the movement of a simulated sound source along a sound surround track. The head-mounted device also includes an inertial measurement unit (IMU) which can be used to acquire the angular acceleration and linear acceleration of the head-mounted device, thereby determining its motion information. Since the head-mounted device is worn on the user's head, its motion information can be considered as the user's head movement information. Since head movement depends on the neck, this allows for the monitoring of neck movement.

[0222] In this embodiment, the earphone component and IMU are encapsulated within a head-mounted device. The head-mounted device guides the user's neck movements, which improves the efficiency of guidance. The user can complete neck movements without holding the device, thereby greatly reducing the difficulty of neck movement operations.

[0223] Furthermore, as another embodiment of this application, Figure 11 This illustration shows a flowchart of the specific implementation of step S502 in an audio playback method according to another embodiment of this application. See also... Figure 11 As shown, with Figure 5 Compared with the previous embodiments, S502 in this application embodiment specifically includes the following steps, which are described in detail below:

[0224] In S5021, the simulated sound source is controlled to move to the Nth key position point on the sound surround trajectory; where N is a positive integer.

[0225] In this embodiment, when the simulated sound source moves on the sound surround track, it can be divided into multiple key position points. The number of key position points can be determined based on the audio adjustment precision or the length of time the simulated sound source stays at each position on the track. The more key position points there are, the faster the user's response speed will be when controlling their head to follow the sound track; conversely, the fewer key position points there are, the slower the user's response speed will be.

[0226] In one possible implementation, the electronic device can be configured with a preset interval distance, dividing the aforementioned sound surround track into multiple different key position points based on this interval distance, where the distance between different key position points is the preset interval distance. For example, Figure 12 A schematic diagram illustrating the division of a sound surround track according to an embodiment of this application is shown. See also Figure 12 As shown in (a), the sound surround track is a circle; however, in other implementations, it can be a curve of any shape. The electronic device can divide the sound surround track into multiple different key points according to a preset interval, with the distance between the different key points being the same.

[0227] In one possible implementation, the electronic device can identify key location points based on the dwell time of the simulated sound source at various locations. As the simulated sound source moves along the sound loop track, it can pause at one or more preset key location points. The electronic device can identify the dwell time corresponding to each location point on the track. If the dwell time at any location point exceeds a preset duration threshold, then that location point is identified as a key location point. For example, see [link to example]. Figure 12 As shown in (b), the simulated sound source stays at position point A for 1 second, but does not stay at position point B (i.e., it only passes through position point B; the corresponding stay time can be determined based on the moving speed of the simulated sound source. For example, the stay time for all position points that only pass through without staying can be set to 0.1 seconds). The corresponding time threshold is 0.5 seconds. Therefore, it can be determined that the stay time at position point A is greater than the time threshold, while the stay time at position point B is less than the time threshold. That is, position point A is a critical position point, while position point B is not a critical position point.

[0228] In this embodiment, the aforementioned sound surround trajectory can be a planar curve or a three-dimensional curve, specifically determined according to the user's neck movement requirements. Therefore, the aforementioned multiple key location points can be located in the same plane or in different planes.

[0229] In S5022, the user's head posture is collected when the simulated sound source moves to the Nth key position point.

[0230] In this embodiment, to facilitate the user's perception of the relative positional changes between their head and the simulated sound source during head movement, the electronic device acquires the user's head posture in real time during the simulated sound source's movement. This head posture includes two parameters: head position and head pose. Since the relative positional relationship with the simulated sound source depends not only on the head's location but also on the head posture, which affects the distance between the simulated sound source and the user's ears, the electronic device can determine the changes in the user's head posture as the simulated sound source moves to a key position. Based on these changes in head posture, the electronic device can determine the audio content to be played; that is, the audio content played by the speaker changes in real time according to the changes in head posture.

[0231] In S5023, the target audio of each of the speakers is determined based on the head posture and the Nth key position point; the target audio is used for the audio segment during the process of the simulated sound source moving from the Nth key position point to the N+1th key position point.

[0232] In this embodiment, changes in head posture alter the relative positions of the user's ears and the simulated sound source. To improve the simulation effect, the electronic device can determine the audio to be played by each speaker based on the real-time head posture, allowing the user to perceive changes in distance between themselves and the simulated sound source. Based on this, the electronic device can determine the target audio to be played based on the relative positional relationship between the head posture and key points, as well as the relative positional relationship between the head posture and subsequent key points (i.e., the N+1th key point). It should be noted that, due to the binaural effect and Doppler effect, the loudness and frequency of the sound received by the user change during head movement. Therefore, the electronic device can determine the frequency and loudness changes of the audio segment based on the head's movement speed and distance, matching the audio segment to the head movement state. This allows for the simulation of changes in the relative position between the user and the simulated sound source, improving the accuracy of guidance.

[0233] Furthermore, as another embodiment of this application, the above-described S5023 may specifically include the following five steps:

[0234] Step 1: Determine the first distance value between the user's head and the Nth key position point based on the user's initial pose when the simulated sound source moves to the Nth key position point and the Nth key position point.

[0235] Step 2: Determine the second distance value between the head pose and the Nth key position point.

[0236] Step 3: Determine a first audio segment based on the first distance value and the second distance value; the first audio segment is used to simulate the relative position change between the user's head and the simulated sound source.

[0237] Step 4: Determine the second audio segment based on the Nth key position point, the N+1th key position point, and the head pose; the second audio segment is used to simulate the simulated audio moving from the Nth key position point to the N+1th key position point.

[0238] Step 5: Generate the target audio based on the first audio segment and the second audio segment.

[0239] In this embodiment, the process of the user's head moving to follow the simulated sound source can be divided into two distinct stages. The first stage is when the user approaches the simulated sound source, and the second stage is when the simulated sound source moves to the next key position. Based on this, the electronic device can acquire the initial pose of the user's head before it moves to follow the simulated sound source. This initial pose can be acquired through the electronic device's IMU or determined by capturing a video image of the user's head. The electronic device can determine a first distance value between the user's initial head pose and the Nth key position point. This first distance value is used to determine the relative positional relationship between the two before the user's head approaches the simulated sound source.

[0240] In this embodiment, after the user's head attempts to approach the simulated sound source, the head's posture changes from the initial posture to a new head posture. At this point, the electronic device can again determine the second distance value between the head posture and the Nth key location point. This second distance value is used to determine the relative positional relationship between the user's head and the simulated sound source after the user's head approaches the simulated sound source.

[0241] In this embodiment, the electronic device can track the first distance value and the second distance value to determine the amount of positional change between the user's head and the simulated sound source, and then determine the loudness change. Based on the change time between the initial pose and the head pose, the speed of the user's head movement can be determined, thereby determining the frequency change of the first audio segment. Based on the above-mentioned loudness change and frequency change, the sound change of the simulated sound source received by the user can be determined, thereby determining the first audio segment corresponding to each speaker.

[0242] Besides the change in relative position between the user's head and the simulated sound source caused by head movement, the movement of the simulated sound source along its trajectory also causes this change. To determine the movement of the simulated sound source, it's necessary to identify the second distance between the Nth key position point and the head posture, and the third distance between the (N+1)th key position point and the head position. Based on the difference between these two distances, the loudness change of the second audio segment can be determined. Similarly, the simulated sound source moves along the sound surround trajectory at a preset speed. Therefore, the frequency change of the second audio segment can be determined based on this preset speed. Based on the loudness and frequency changes, the sound changes during the movement of the simulated sound source from the Nth key position point to the (N+1)th key position point can be determined, thus identifying the second audio segment corresponding to each speaker. Superimposing the first and second audio segments yields the target audio during the aforementioned process.

[0243] For example, Figure 13 A schematic diagram illustrating audio segment determination according to an embodiment of this application is shown. See also... Figure 13 As shown in (a), the user's head rotates from point A to point B, moving closer to the location C of the simulated sound source. At this point, the distance between them changes from d1 to d2. The electronic device can determine the first audio segment corresponding to each speaker based on this change in distance between d1 and d2. On the other hand, see... Figure 13 As shown in (b), when the simulated sound source moves from position point C to position point D, the distance between it and the user's head point B changes from d2 to d3. The electronic device can also determine the second audio segment corresponding to each speaker based on the change in distance between d2 and d3. Integrating the two audio segments yields the target audio segment.

[0244] In this embodiment, the audio change process is divided into two parts, and the audio segment corresponding to each part is determined. Then, the two audio segments are superimposed to obtain the target audio, which can improve the accuracy of target audio generation and thus improve the simulation effect.

[0245] In S5024, the value of N is increased. If the value of N after the increase is less than the total number of key position points on the sound surround trajectory, then the operation of collecting the user's head posture after the simulated sound source moves to the Nth key position point is returned.

[0246] In S5025, if the value of N after the increase is greater than or equal to the total number, then the head activity information is generated based on all the head poses.

[0247] In this embodiment, before the simulated sound source reaches its final critical position, the target audio for each speaker continues to be determined using the aforementioned method until the simulated audio has moved completely. Then, based on all head poses acquired during the movement of the simulated sound source, the user's head activity information can be generated. Specifically, the electronic device can associate each head pose with a corresponding acquisition time, determine the posture order of each head pose based on the acquisition time, and thus generate the aforementioned head activity information.

[0248] In this embodiment of the application, by obtaining the head posture corresponding to different key position points and determining the target audio for the next stage, the simulated sound source can change with the user's head posture, thereby improving the realism of the simulated sound source and thus improving the accuracy of neck movement guidance.

[0249] In S503, a scoring report is generated based on the head movement information and the sound surround trajectory.

[0250] In this embodiment, the electronic device can determine the user's head movement trajectory by acquiring the head movement information corresponding to the user's movement along the sound surround track during the simulated sound source movement. Based on the movement amplitude, movement duration, and similarity to the sound surround track corresponding to the movement trajectory, the electronic device can determine the activity effect of the user's neck movement, that is, obtain the corresponding evaluation score, and generate a corresponding rating report based on the evaluation score.

[0251] In one possible implementation, the electronic device can determine the amplitude of head movement information, determine the corresponding amplitude score based on the amplitude, then determine the total duration of movement of the current sound surround track (i.e., the total duration of the user's neck movement), determine the corresponding duration score, finally generate a head movement trajectory based on all head poses, calculate the similarity between the head movement trajectory and the sound surround trajectory, determine the matching score, and calculate the corresponding total activity score based on the amplitude score, duration score, and matching score, and obtain the corresponding scoring report.

[0252] In one possible implementation, if the electronic device is a head-mounted device, the aforementioned scoring report can be output via voice broadcast, allowing the user to listen to the corresponding audio to determine the content of the scoring report. Optionally, if the head-mounted device is connected to other electronic devices, such as a smartphone, the head-mounted device can send the corresponding scoring report to the smartphone, which will then display the report on its display module, allowing the user to view the report's content on the smartphone.

[0253] For example, Figure 14A schematic diagram showing a scoring report provided in one embodiment of this application is illustrated. See also: Figure 14 As shown in (a), the electronic device acquiring the user's head movement information is a head-mounted device. After generating a score report of the user's neck movement, the head-mounted device can send the score report to a smartphone via Bluetooth communication. The smartphone has a client program associated with the head-mounted device installed, which can be launched to display the corresponding score report, such as... Figure 14 As shown in (b), the rating report can display ratings in different dimensions, such as activity magnitude score, duration score, and consistency score.

[0254] Furthermore, as another embodiment of this application, Figure 15 This document shows a flowchart illustrating the specific implementation of step S503 in an audio playback method according to another embodiment of this application. (See attached flowchart.) Figure 15 As shown, with Figure 5 Compared with the aforementioned embodiments, S503 in this application embodiment specifically includes the following steps, which are described in detail below:

[0255] In S5031, based on multiple head poses within the head activity information, the user's head movement trajectory is generated during the movement of the simulated sound source along the sound surround trajectory.

[0256] In S5032, the trajectory similarity between the head movement trajectory and the sound surround trajectory is calculated.

[0257] In S5033, the user's activity range is determined based on the head movement trajectory.

[0258] In S5034, the scoring report is generated based on the trajectory similarity and the activity range.

[0259] In this embodiment, the electronic device collects multiple head postures as the user moves along with the simulated sound source. Based on the order in which the head postures are collected, these head postures are sequentially connected to generate the user's corresponding head movement trajectory. The similarity between this head movement trajectory and the sound movement trajectory of the simulated sound source is calculated to obtain the aforementioned trajectory similarity.

[0260] In one possible implementation, the trajectory similarity can be calculated as follows: Based on the acquisition time, establish the correlation between each coordinate point on the head movement trajectory and each coordinate point on the sound surround trajectory; then calculate the distance between each coordinate point on the head movement trajectory and the corresponding coordinate point on the sound surround trajectory; and determine the trajectory similarity between the two based on the sum or mean of the distance values ​​corresponding to all head movement trajectories. The smaller the sum or mean of the distance values, the greater the trajectory similarity; conversely, the larger the sum or mean of the distance values, the smaller the trajectory similarity.

[0261] In one possible implementation, the electronic device can scale the head movement trajectory to maximize the overlap between the scaled head movement trajectory and the sound surround trajectory, and use the value corresponding to the maximum overlap between the two trajectories as the trajectory similarity between the two.

[0262] In this embodiment, in addition to determining the similarity between the two trajectories, the electronic device also determines the range of head movement during the current neck movement, i.e., the amplitude of movement. This amplitude of movement can be determined based on the distance between the head movement trajectory and the human body axis, or it can be determined based on the head swing angle, depending on the actual situation.

[0263] In this embodiment, the electronic device can import trajectory similarity and activity range into a preset report template to generate a rating report corresponding to the neck movement. Optionally, the electronic device is equipped with a rating conversion algorithm. By importing the trajectory similarity and activity range into the aforementioned rating conversion algorithm, an evaluation score corresponding to the current neck movement can be calculated, and this evaluation score can be added to the aforementioned rating report, making it convenient for users to determine the effectiveness of the current neck movement.

[0264] In this embodiment of the application, the head movement trajectory of the current activity process is generated by the head movement information. The similarity between the head movement trajectory and the sound surround trajectory is determined by the head movement trajectory, and the user's activity range is determined. The user's neck movement is evaluated from multiple angles, which can improve the accuracy of the evaluation report.

[0265] Furthermore, as another embodiment of this application, after S503, it may also include:

[0266] In S5035, the actual range of motion of the user in multiple directions is determined based on the range of motion.

[0267] In S5036, if the actual activity amplitude corresponding to any activity direction is less than a preset amplitude threshold, then the any activity direction is identified as an enhanced activity direction.

[0268] In S5037, neck exercise prompts are generated based on all the described directions of enhanced activity.

[0269] In this embodiment, after generating a neck activity score report, the electronic device can provide relevant exercise suggestions based on the user's activity. For example, if the user's head turns significantly less to the left than to the right while following a simulated sound source, it may indicate an imbalance in the user's neck rotation, possibly due to insufficient exercise of the left side of the neck muscles. In this case, the electronic device can provide exercise reminders to the user. Therefore, after generating the neck activity score report, the electronic device can identify the actual activity amplitude corresponding to each preset activity direction within the activity range and compare the actual activity amplitude with a preset amplitude threshold. If the actual activity amplitude is greater than the amplitude threshold for the corresponding direction, it indicates that there is no abnormality in that activity direction, and no neck exercise prompt information needs to be generated for that activity direction. Conversely, if the actual activity amplitude is less than the preset amplitude threshold, it indicates that the user's neck movement in that direction is abnormally small and requires more exercise. Therefore, neck exercise prompt information for that activity direction can be generated.

[0270] For example, Figure 16 A schematic diagram of neck exercise prompts provided in an embodiment of this application is shown. See also Figure 16 As shown, after the electronic device outputs the scoring report, it can display neck exercise prompts via pop-up windows, thus providing users with targeted exercise reminders, such as "Left side range of motion is small, please strengthen the left side neck muscles," enabling users to perform targeted exercises.

[0271] In this embodiment of the application, when it is identified that a user's activity is restricted in a certain direction, a prompt can be given to the user, which improves the readability of the evaluation report and also makes it easier for the user to perform neck exercises.

[0272] As can be seen from the above, the audio playback method provided in this application embodiment can simulate a sound source by configuring corresponding speakers on both sides of the user's ears based on the binaural effect. By controlling the simulated sound source to move along a preset sound surround track, the method aims to prompt the user to perform neck movements. Furthermore, during the user's neck movements, i.e., as the simulated sound source moves along the sound surround track, the method collects the user's head movement information. Finally, based on the head movement information and the aforementioned sound surround track, a score is obtained for the user's neck movement, achieving screenless guidance. Compared with existing multimedia playback technologies, this application embodiment can simulate a corresponding sound source through the binaural effect and guide the user's movements by controlling the simulated sound source to move around. Users do not need to constantly stare at the screen; instead, they can move in a specified direction and along a specified trajectory based on the relative position of the simulated sound source and themselves. This reduces the difficulty of operation for the user, making neck movement more flexible and improving the user experience.

[0273] Example 2:

[0274] Corresponding to the audio playback method described in the above embodiments, Figure 17 This diagram illustrates the structure of an audio playback device provided in an embodiment of this application. For ease of explanation, only the parts relevant to the embodiments of this application are shown.

[0275] Reference Figure 17 The audio playback device includes:

[0276] The simulated sound source moving unit 171 is used to control the simulated sound source to move along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory; the simulated sound source is simulated by speakers located on both sides of the user's ears;

[0277] The head movement information acquisition unit 172 is used to acquire the user's head movement information as the simulated sound source moves along the sound surround trajectory;

[0278] The scoring report output unit 173 is used to generate a scoring report based on the head movement information and the sound surround trajectory.

[0279] Optionally, the head activity information acquisition unit 172 includes:

[0280] A key position point moving unit is used to control the simulated sound source to move to the Nth key position point on the sound surround trajectory; where N is a positive integer;

[0281] A head position and pose determination unit is used to collect the head position and pose of the user when the simulated sound source moves to the Nth key position point.

[0282] The target audio determination unit is used to determine the target audio of each of the speakers based on the head posture and the Nth key position point; the target audio is used for the audio segment of the simulated sound source during the process of moving from the Nth key position point to the N+1th key position point;

[0283] The loop unit is used to increase the value of N. If the value of N after the increase is less than the total number of key position points on the sound surround trajectory, then the operation of collecting the user's head posture after the simulated sound source moves to the Nth key position point is returned.

[0284] The head activity information encapsulation unit is used to generate the head activity information based on all the head poses if the value of N after the increase is greater than or equal to the total number.

[0285] Optionally, the target audio determination unit includes:

[0286] The first distance value determination unit is used to determine the first distance value between the user's head and the Nth key position point based on the user's initial pose when the simulated sound source moves to the Nth key position point and the Nth key position point;

[0287] The second distance value determination unit is used to determine the second distance value between the head pose and the Nth key position point;

[0288] The first audio segment determination unit is used to determine a first audio segment based on the first distance value and the second distance value; the first audio segment is used to simulate the relative position change between the user's head and the simulated sound source;

[0289] The second audio segment determination unit is used to determine a second audio segment based on the Nth key position point, the (N+1)th key position point, and the head pose; the second audio segment is used to simulate the simulated audio moving from the Nth key position point to the (N+1)th key position point;

[0290] The target audio encapsulation unit is used to generate the target audio based on the first audio segment and the second audio segment.

[0291] Optionally, the scoring report output unit 173 includes:

[0292] The head motion trajectory generation unit is used to generate the user's head motion trajectory during the process of the simulated sound source moving on the sound surround trajectory, based on multiple head poses in the head activity information.

[0293] A trajectory similarity calculation unit is used to calculate the trajectory similarity between the head movement trajectory and the sound surround trajectory;

[0294] An activity range determination unit is used to determine the user's activity range based on the head movement trajectory.

[0295] The rating report generation unit is used to generate the rating report based on the trajectory similarity and the activity range.

[0296] Optionally, the audio playback device further includes:

[0297] The actual activity range determination unit is used to determine the actual activity range of the user in multiple activity directions based on the activity range.

[0298] The enhanced activity direction determination unit is used to identify any activity direction as an enhanced activity direction if the actual activity amplitude corresponding to any activity direction is less than a preset amplitude threshold.

[0299] A neck exercise prompting unit is used to generate neck exercise prompting information based on all of the said directions of increased activity.

[0300] Optionally, the audio playback device further includes:

[0301] The tutorial prompt unit is used to play the Mth tutorial prompt message based on a preset tutorial prompt list; the initial value of M is 1.

[0302] The actual motion information acquisition unit is used to acquire the actual motion information of the user based on the Mth tutorial prompt information;

[0303] A success prompt tone output unit is used to output a preset success prompt tone and increase the value of M if the actual motion information matches the standard motion information associated with the Mth tutorial prompt information.

[0304] The execution unit is used to return to the preset tutorial prompt list and play the Mth tutorial prompt if the value of M is less than or equal to the total number of tutorial prompts in the tutorial prompt list.

[0305] The completion prompt unit is used to output a preset completion prompt sound if the value of M is greater than the total number of tutorial prompt messages.

[0306] Optionally, the audio playback device is applied to a head-mounted device; the head-mounted device includes an earphone component and an inertial sensor; the earphone component includes speakers on both sides of the ears; the inertial sensor is used to collect head movement information.

[0307] Therefore, the audio playback device provided in this application embodiment can also simulate a sound source by configuring corresponding speakers on both sides of the user's ears based on the binaural effect. By controlling the simulated sound source to move along a preset sound surround track, it achieves the purpose of prompting the user to perform neck movements. Furthermore, during the user's neck movements, i.e., as the simulated sound source moves along the sound surround track, the user's head movement information is collected. Finally, based on the head movement information and the aforementioned sound surround track, an evaluation score for the user's neck movement is obtained, achieving the purpose of screenless guided activity. Compared with existing multimedia playback technologies, this application embodiment can simulate a corresponding simulated sound source through the binaural effect and guide the user's activities by controlling the simulated sound source to move around. Users do not need to constantly stare at the screen; instead, they can move in a specified direction and along a specified trajectory based on the relative position of the simulated sound source and themselves. This reduces the difficulty of operation for the user, making neck movement more flexible and improving the user experience.

[0308] Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 18 As shown, the electronic device 18 of this embodiment includes: at least one processor 180 ( Figure 18 The diagram shows only one processor, a memory 181, and a computer program 182 stored in the memory 181 and executable on the at least one processor 180, which, when executed by the processor 180, implements the steps in any of the above-described audio playback method embodiments.

[0309] The electronic device 18 may be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor 180 and a memory 181. Those skilled in the art will understand that... Figure 18 This is merely an example of electronic device 18 and does not constitute a limitation on electronic device 18. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0310] The processor 180 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0311] In some embodiments, the memory 181 may be an internal storage unit of the electronic device 18, such as a hard disk or memory of the electronic device 18. In other embodiments, the memory 181 may be an external storage device of the electronic device 18, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 18. Furthermore, the memory 181 may include both internal and external storage units of the electronic device 18. The memory 181 is used to store operating systems, applications, bootloaders, data, and other programs, such as the program code of computer programs. The memory 181 can also be used to temporarily store data that has been output or will be output.

[0312] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0313] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0314] This application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0315] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0316] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0317] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0318] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0319] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0320] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0321] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0322] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for playing audio, characterized in that, include: The simulated sound source is controlled to move along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory; the simulated sound source is simulated by speakers located on both sides of the user's ears; As the simulated sound source moves along the sound surround trajectory, the user's head movement information is collected; A scoring report is generated based on the head movement information and the sound surround trajectory; The process of collecting the user's head movement information during the movement of the simulated sound source along the sound surround trajectory includes: Control the simulated sound source to move to the Nth key position point on the sound surround trajectory; where N is a positive integer. Collect the user's head posture when the simulated sound source moves to the Nth key position point; Based on the head pose and the Nth key position point, the target audio of each of the speakers is determined; the target audio is used for the audio segment during the process of the simulated sound source moving from the Nth key position point to the N+1th key position point; Increase the value of N. If the value of N after the increase is less than the total number of key position points on the sound surround trajectory, then return to the operation of collecting the user's head posture after the simulated sound source moves to the Nth key position point. If the value of N after the increase is greater than or equal to the total number, then the head movement information is generated based on all the head poses.

2. The playback method according to claim 1, characterized in that, The step of determining the target audio for each speaker based on the head posture and the Nth key position point includes: Based on the user's initial pose when the simulated sound source moves to the Nth key location point and the Nth key location point, determine the first distance value between the user's head and the Nth key location point; Determine the second distance value between the head pose and the Nth key location point; Based on the first distance value and the second distance value, a first audio segment is determined; the first audio segment is used to simulate the relative position change between the user's head and the simulated sound source; Based on the Nth key position point, the (N+1)th key position point, and the head pose, a second audio segment is determined; the second audio segment is used to simulate the simulated audio moving from the Nth key position point to the (N+1)th key position point; The target audio is generated based on the first audio segment and the second audio segment.

3. The playback method according to claim 1, characterized in that, The process of generating a scoring report based on the head movement information and the sound surround trajectory includes: Based on multiple head poses within the head activity information, the user's head movement trajectory is generated during the process of the simulated sound source moving on the sound surround trajectory. Calculate the trajectory similarity between the head movement trajectory and the sound surround trajectory; The user's activity range is determined based on the head movement trajectory; The scoring report is generated based on the trajectory similarity and the activity range.

4. The playback method according to claim 3, characterized in that, After generating the scoring report based on the head movement information and the sound surround trajectory, the method further includes: Based on the activity range, determine the actual range of the user's activity in multiple activity directions; If the actual activity amplitude corresponding to any activity direction is less than a preset amplitude threshold, then the any activity direction is identified as an enhanced activity direction. Neck exercise prompts are generated based on all the described directions of increased activity.

5. The playback method according to any one of claims 1-4, characterized in that, Before the simulated sound source is moved along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory, the method further includes: Based on a preset list of tutorial prompts, play the Mth tutorial prompt message; the initial value of M is 1; Obtain the user's actual movement information based on the Mth tutorial prompt; If the actual motion information matches the standard motion information associated with the Mth tutorial prompt, a preset success prompt sound is output, and the value of M is increased. If the value of M is less than or equal to the total number of tutorial prompts in the tutorial prompt list, then return to the preset tutorial prompt list and play the Mth tutorial prompt. If the value of M is greater than the total number of tutorial prompts, a preset completion prompt sound will be output.

6. The playback method according to any one of claims 1-4, characterized in that, The device is applied to a head-mounted device; the head-mounted device includes an earphone component and an inertial sensor; the earphone component includes speakers on both sides of the ears; the inertial sensor is used to collect head movement information.

7. An audio playback device, characterized in that, include: A simulated sound source movement unit is used to control the simulated sound source to move along a preset sound surround trajectory to prompt the user to move their neck according to the sound surround trajectory; the simulated sound source is simulated by speakers located on both sides of the user's ears; The head movement information acquisition unit is used to acquire the user's head movement information as the simulated sound source moves along the sound surround trajectory. A scoring report output unit is used to generate a scoring report based on the head movement information and the sound surround trajectory. The head activity information acquisition unit includes: A key position point moving unit is used to control the simulated sound source to move to the Nth key position point on the sound surround trajectory; where N is a positive integer; A head position and pose determination unit is used to collect the head position and pose of the user when the simulated sound source moves to the Nth key position point. The target audio determination unit is used to determine the target audio of each of the speakers based on the head posture and the Nth key position point; the target audio is used for the audio segment of the simulated sound source during the process of moving from the Nth key position point to the N+1th key position point; The loop unit is used to increase the value of N. If the value of N after the increase is less than the total number of key position points on the sound surround trajectory, then the operation of collecting the user's head posture after the simulated sound source moves to the Nth key position point is returned. The head activity information encapsulation unit is used to generate the head activity information based on all the head poses if the value of N after the increase is greater than or equal to the total number.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images