Audio playback method, device, mobile terminal and storage medium

Abstract

The embodiment of the present application discloses an audio playback method, device, mobile terminal and storage medium, the method is applied to a mobile terminal, and the method comprises: displaying the picture content of at least part of the picture partition of the spatial picture on the screen of the mobile terminal; the spatial picture is divided into at least two different picture partitions, each picture partition corresponds to at least one audio signal; if the picture content displayed on the screen changes, then play the audio signal corresponding to the picture partition displayed in the target area of ​​the screen after the picture content changes. The implementation of the embodiment of the present application enriches the interactive mode of audio playback, and makes the scene space sense created by both visual and auditory aspects more real and natural.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to an audio playback method, device, mobile terminal and storage medium. Background Art

[0002] As the pace of life and work accelerates, some users will choose applications that can provide stress relief functions to help relieve stress. This type of application mainly guides users to relax their minds by playing soothing and monotonous music, thereby relieving their stress. However, in practice, it is found that when users use this type of application, they can only mechanically select the music they are interested in to play, and the interactive mode of audio playback is relatively simple. Summary of the invention

[0003] The embodiments of the present application disclose an audio playback method, device, mobile terminal and storage medium, which can enrich the interactive mode of audio playback.

[0004] An embodiment of the present application discloses an audio playback method, which is applied to a mobile terminal, and the method includes: displaying the picture content of at least a part of the picture partition of the spatial picture on the screen of the mobile terminal; the spatial picture is divided into at least two different picture partitions, and each picture partition corresponds to at least one audio signal; if the picture content displayed on the screen changes, playing the audio signal corresponding to the picture partition displayed in the target area of ​​the screen after the picture content changes.

[0005] An embodiment of the present application discloses an audio playback device, which is applied to a mobile terminal, and the device includes: a display module, which is used to display the picture content of at least part of the picture partition of the spatial picture on the screen of the mobile terminal; the spatial picture is divided into at least two different picture partitions, and each picture partition corresponds to at least one audio signal; and a playback module, which is used to play the audio signal corresponding to the picture partition displayed in the target area of ​​the screen after the picture content displayed on the screen changes.

[0006] An embodiment of the present application discloses a mobile terminal, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements any one of the audio playback methods disclosed in the embodiment of the present application.

[0007] An embodiment of the present application discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, any one of the audio playback methods disclosed in the embodiment of the present application is implemented.

[0008] Compared with the related art, the embodiments of the present application have the following beneficial effects:

[0009] In an embodiment of the present application, the spatial screen can be divided into a plurality of different screen partitions, and the screen content of at least part of the screen partitions can be displayed on the screen of the mobile terminal. Therefore, when the screen content displayed on the screen changes, the user can see the screen content that has not been displayed before, and the screen content that has been displayed before may also become invisible, thereby visually creating a 360° scene space sense. At the same time, the audio signal played by the mobile terminal can change with the change of the screen content, thereby cooperating with the change of the visual scene in hearing, enriching the interactive mode of audio playback, and making the scene space sense created by both visual and auditory aspects more real and natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] Figure 1 is an example diagram of an application scenario of an audio playback method disclosed in an embodiment;

[0012] Figure 2 is a method flow chart of an audio music playing method disclosed in an embodiment;

[0013] Figure 3 is an example diagram of displaying a spatial picture on a screen disclosed in an embodiment;

[0014] Figure 4 is a method flow diagram of another audio playback method disclosed in an embodiment;

[0015] Figure 5A is another example diagram of displaying a spatial picture on a screen disclosed in an embodiment;

[0016] Figure 5B is another example diagram of displaying a spatial picture on a screen disclosed in an embodiment;

[0017] Figure 6 is a method flow diagram of another audio playback method disclosed in an embodiment;

[0018] Figure 7 is an example diagram of a display interface of different scene modes disclosed in an embodiment;

[0019] Figure 8 is another example diagram of displaying a spatial picture on a screen disclosed in an embodiment;

[0020] Fig. 9 is a schematic diagram of the structure of an audio playback device disclosed in an embodiment;

[0021] Fig.10 The present invention is a schematic diagram of the structure of a mobile terminal disclosed in an embodiment. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0024] The audio playback method, device, mobile terminal, and storage medium disclosed in the embodiments of the present application can enrich the interactive mode of audio playback.

[0025] See also Figure 1 , Figure 1 FIG. 1 is an example diagram of an application scenario of an audio playback method disclosed in an embodiment. Figure 1 As shown, the mobile terminal 10 may be loaded with an application for controlling audio playback. A display interface in the application may be used to display a spatial image. Among them:

[0026] The mobile terminal 10 may include electronic devices such as a smart phone, a smart tablet, a smart watch, etc., but is not limited thereto. The spatial picture may be any image, for example, a panoramic image including 360° environmental information in space.

[0027] like Figure 1 As shown, what is displayed on the screen 110 of the mobile terminal 10 may be a partial screen content 210 of the spatial screen; the spatial screen also includes another partial screen content 220 that is not displayed on the screen 110 . Figure 1 The dotted line may indicate the screen content that is not displayed on the screen 110 .

[0028] The screen content displayed on the screen 110 may change, so that the screen content 220 that was not originally displayed on the screen 110 is switched to be displayed on the screen 110; or, the screen content that was originally displayed on the screen 110 is switched to not be displayed on the screen 110.

[0029] Based on Figure 1 For an example of an application scenario of the audio playback method shown in Figure 2 , Figure 2 The present invention is a schematic diagram of a method flow of an audio music playing method disclosed in an embodiment, and the method can be applied to any of the above-mentioned mobile terminals. Figure 2 As shown, the audio playing method may include the following steps:

[0030] 210. Display the screen content of at least a portion of the screen partition of the spatial screen on a screen of the mobile terminal.

[0031] In the embodiment of the present application, the spatial screen is divided into at least two different screen partitions. The area attributes such as height and width of each screen partition may be the same or different, and are not specifically limited.

[0032] Among them, the dividing boundary of the screen partition may be invisible to the user, and each screen partition may be continuous. What is displayed on the screen may be part of the screen content in the spatial screen, that is, part of the screen partition. Therefore, when the screen content changes, the screen partition that was originally displayed may be moved to a place that is invisible to the user, or the screen partition that is not displayed on the screen may be moved to the screen for display, so that the change of the screen content can visually create a sense of space for the user to switch scenes.

[0033] Exemplarily, the spatial screen may be divided into N screen partitions, and what may be displayed on the screen may be M screen partitions of the N screen partitions. N and M may be positive integers, and M is less than N.

[0034] In an embodiment of the present application, each screen partition may correspond to at least one audio signal, and the same audio signal may correspond to one or more screen partitions. The correspondence between the screen partitions and the audio signals may be set according to actual business needs, without specific limitation. Each audio signal corresponding to each screen partition may be extracted from a complete audio source signal; or, each audio signal may also be derived from a completely different audio source signal, without specific limitation.

[0035] In the embodiment of the present application, the data format of the audio signal is not limited. Optionally, the audio signal may be Dolby Atmos. Dolby Atmos is a three-dimensional (3-Dimension, 3D) ambient sound that can create a sense of space for the user in terms of the sense of hearing of the sound.

[0036] In the embodiment of the present application, the content of the space picture displayed on the screen may change. The change of the content of the picture may be triggered by a touch operation input by the user, or may be changed automatically, and is not specifically limited.

[0037] In one embodiment, the change of the screen content may be triggered by a touch operation input by the user. Therefore, after executing step 210 and before executing step 220, the mobile terminal may further perform the following steps:

[0038] Detecting a first touch operation input. The first touch operation may be a movement operation input by a user on the screen, and may be any touch operation such as clicking, dragging, sliding, etc., without specific limitation. The first touch operation may also be an operation input by a user through voice, gesture, etc., without specific limitation.

[0039] In response to the first touch operation, the screen content displayed on the screen of the moving space screen is moved. The first touch operation may indicate one or more moving attributes of the moving space screen, such as a moving direction, a moving distance, and a moving speed.

[0040] Exemplarily, the first touch operation may indicate a moving direction of the spatial picture.

[0041] When the first touch operation is a click operation, the mobile terminal can determine the moving direction according to the click position of the click operation: if the click position is on the left side of the center line of the mobile terminal screen, the moving direction indicated by the click operation may be to the left; if the click position is on the right side of the center line of the mobile terminal screen, the moving direction indicated by the click operation may be to the right.

[0042] When the first touch operation is a sliding operation, the mobile terminal may determine the sliding direction of the sliding operation as the moving direction of the spatial screen.

[0043] Exemplarily, the first touch operation may indicate the moving direction and moving distance of the spatial screen. When the first touch operation is a drag operation, the mobile terminal may determine the moving direction of the drag operation as the moving direction of the spatial screen, and determine the moving distance of the drag operation on the screen as the moving distance of the spatial screen.

[0044] In one embodiment, the change of the screen content may also be automatic. Therefore, after executing step 210 and before executing step 220, the mobile terminal may further perform the following steps:

[0045] The screen content displayed on the screen of the spatial screen is moved according to a preset movement rule. The preset movement rule may include: the screen content displayed on the screen of the spatial screen is moved according to one or more movement attributes such as a preset movement direction, a preset movement distance, and a preset movement speed.

[0046] Exemplarily, the mobile terminal may move the screen content displayed on the spatial screen to the left at a speed of 10 pixels per second.

[0047] In the embodiment of the present application, the mobile terminal can automatically move the picture content displayed on the screen, or can move the picture content displayed on the screen in response to the first touch operation input by the user, so that the picture content displayed on the screen changes. When the picture content displayed on the screen changes, the mobile terminal executes the following step 220.

[0048] 220. If the picture content of the spatial picture displayed on the screen changes, then the audio signal corresponding to the picture partition displayed in the target area of ​​the screen after the picture content changes is played.

[0049] In step 220, a target area may be preset on the mobile terminal as a standard area for determining whether it is necessary to switch to play different audio signals, and the target area's setting position, height, width and other area attributes may be set according to actual business needs, without specific limitations. For example, the target area may be based on the center line of the screen as the central axis, the width and height of the target area may be set according to the width and height of the screen partition in the spatial screen, the width of the target area may be consistent with the width of the screen partition, and the height of the target area may be consistent with the height of the screen partition.

[0050] When the picture content displayed on the screen of the spatial picture changes, the mobile terminal can determine the picture partition overlapping with the target area based on the positional relationship between the display positions of each picture partition of the spatial picture after the change and the display position of the target area.

[0051] If only one screen partition overlaps with the target area, the screen partition currently displayed in the target area may be the overlapping screen partition.

[0052] If there are two screen partitions overlapping with the target area, the mobile terminal can determine the screen partition currently displayed in the target area according to the area of ​​the area where the screen partition overlaps with the target area, and the screen partition currently displayed in the target area is the screen partition with the larger overlapping area of ​​the two screen partitions; or, determine the screen partition currently displayed in the target area according to the distance between the contour center of gravity of the screen partition and the contour center of gravity of the target area, and the screen partition currently displayed in the target area is the screen partition that is closer to the contour center of gravity of the target area of ​​the two screen partitions. It should be noted that the area and the contour center of gravity are two possible implementation methods for determining the screen partition currently displayed in the target area, which are not specifically limited.

[0053] In step 220, the audio signal played by the mobile terminal may be determined based on the screen partition displayed on the target area after the screen content changes. If the screen partition displayed on the target area changes due to the change of the screen content, the audio signal played by the mobile terminal may also be switched accordingly; if the screen partition displayed on the target area remains unchanged after the screen content changes, the audio signal played by the mobile terminal also remains unchanged.

[0054] In addition, in the embodiments of the present application, the specific implementation method of playing the audio signal by the mobile terminal is not limited. Taking the operating system of the mobile terminal as an example, the mobile terminal can play the audio signal through the SoundPool class, the AudioTrack class, or the MediaPlayer class, which can be selected according to the data format of the audio signal, and the embodiments of the present application do not limit it.

[0055] For example, see Figure 3 , Figure 3 An example diagram of displaying a spatial image on a screen is disclosed in an embodiment. Figure 3 As shown, the screen partition 311 and the screen partition 312 displayed on the screen 110 both overlap with the target area 320. Figure 3 As shown, the overlapping area between screen partition 311 and target area 320 is larger than the overlapping area between screen partition 312 and target area 320, so the screen partition displayed on target area 320 is screen partition 311. The mobile terminal can play the audio signal corresponding to screen partition 311.

[0056] It can be seen that in the aforementioned embodiment, the spatial picture can be divided into a plurality of different picture partitions, and the screen of the mobile terminal can display the picture content of at least part of the picture partitions. Therefore, when the picture content displayed on the screen changes, the user can see the picture content that has not been displayed before, and the picture content that has been displayed before may also become invisible, visually creating a 360° scene space sense. At the same time, the audio signal played by the mobile terminal can change with the change of the picture content, and the auditory changes in coordination with the visual scene, enriching the interactive mode of audio playback, and making the scene space sense created by both visual and auditory aspects more real and natural. Furthermore, the audio signal can use Dolby Atmos, which can further enhance the sense of space in hearing.

[0057] In some embodiments, the spatial picture and the audio signals corresponding to the various picture partitions included in the spatial picture can be multimedia resources in an application loaded on the mobile terminal. The application can help users relax and is applicable in application scenarios such as meditation and assisted sleep. The audio playback method disclosed in the aforementioned embodiment is implemented to create a real sense of scene space for users through the coordination of vision and hearing, which is conducive to guiding users into the scene atmosphere created by the spatial picture and audio signals, helping users to relax more effectively, and improving user experience.

[0058] See also Figure 4 , Figure 4 The method flow chart of another audio playback method disclosed in an embodiment is a schematic diagram, and the method can be applied to any of the aforementioned mobile terminals. Figure 4 As shown, the audio playing method may include the following steps:

[0059] 410. Display the screen content of at least a portion of the screen partition of the spatial screen on the screen of the mobile terminal.

[0060] 420. In response to the input first touch operation, the screen content displayed on the screen of the moving space screen.

[0061] 430. After the picture content of the spatial picture displayed on the screen changes, determine whether the picture partition displayed on the target area of ​​the screen changes; if the picture partition displayed on the target area switches from the first picture partition to the second picture partition, execute step 440; if the picture partition displayed on the target area remains the first picture partition, execute step 450.

[0062] It should be noted that before the screen content displayed on the spatial screen changes, the screen partition displayed in the target area of ​​the screen may be the first screen partition. The first screen partition may be any one of the multiple screen partitions into which the spatial screen is divided, and is not specifically limited.

[0063] After the screen content displayed on the spatial screen changes, if the screen partition displayed on the target area switches from the first screen partition to the second screen partition, the mobile terminal may execute step 440 to switch to play a different audio signal. The second screen partition is any screen partition different from the first screen partition among the multiple screen partitions into which the spatial screen is divided.

[0064] For example, see Figure 5A , Figure 5A Another example of displaying a spatial image on a screen is disclosed in an embodiment. Figure 5AAs shown, before the picture content displayed on the screen 110 of the spatial picture changes, the picture partition displayed in the target area 520 may be the picture partition 511 , and the mobile terminal plays the first audio signal corresponding to the picture partition 511 .

[0065] The first touch operation 530a input by the user can be used to instruct the picture content displayed on the screen of the spatial picture to move to the left, so that the picture content displayed on the screen changes. It should be noted that when the mobile terminal moves the picture content displayed on the screen of the spatial picture in response to the first touch operation 530a, the mobile terminal can continue to play the first audio signal corresponding to the picture partition 511.

[0066] When the mobile terminal completes the movement operation on the screen content in response to the first touch operation 530a, the screen partition displayed in the target area 520 switches from the screen partition 511 to the screen partition 512. At this time, the mobile terminal can switch to play the second audio signal corresponding to the screen partition 512.

[0067] If the screen partition displayed on the target area remains unchanged after the screen content displayed on the screen changes, that is, remains the first screen partition, the mobile terminal may execute step 450 to continue playing the audio signal corresponding to the first screen partition.

[0068] For example, see Figure 5B , Figure 5B Another example diagram of displaying a spatial image on a screen disclosed in an embodiment. Figure 5B As shown, before the picture content displayed on the screen of the spatial picture changes, the picture partition displayed in the target area 520 may be the picture partition 511 , and the mobile terminal plays the first audio signal corresponding to the picture partition 511 .

[0069] The first touch operation 530b input by the user can be used to instruct the screen content displayed on the spatial screen to move to the left, so that the screen content displayed on the screen changes. The moving distance indicated by the first touch operation 530b is short, so when the mobile terminal completes the moving operation in response to the first touch operation 530b, the area of ​​the overlapping area between the screen partition 511 and the target area 520 is still greater than the area of ​​the overlapping area between the screen partition 512 and the target area 520. Therefore, after the screen content displayed on the screen changes, the screen partition displayed in the target area 520 still remains as the screen partition 511, and the mobile terminal can continue to play the first audio signal corresponding to the screen partition 511.

[0070] 440. Switch the played audio signal from the first audio signal corresponding to the first screen partition to the second audio signal corresponding to the second screen partition.

[0071] In step 440, the second audio signal may be different from the first audio signal, or may be the same as the first audio signal, which is specifically determined according to the corresponding relationship between the picture partitions and the audio signals.

[0072] In one embodiment, the mobile terminal can directly switch from the first audio signal to the second audio signal without transition, and the playback volume of the first audio signal and the second audio signal can always be the set playback volume of the mobile terminal operating system.

[0073] In another embodiment, the mobile terminal may first gradually reduce the playback volume of the first audio signal until the playback volume of the first audio signal is reduced to the first volume threshold. Furthermore, while reducing the playback volume of the first audio signal, the mobile terminal may switch to play the second audio signal and gradually increase the playback volume of the second audio signal until the playback volume of the second audio signal is increased to the second volume threshold.

[0074] The second volume threshold may be greater than the first volume threshold, and the specific values ​​of the second volume threshold and the first volume threshold may be set in accordance with actual business needs, and are not specifically limited. For example, the first volume threshold may be the minimum volume that can be heard by human ears, and the second volume threshold may be the current playback volume of the operating system of the mobile terminal.

[0075] In addition, when switching to play the second audio signal, the initial volume of the second audio signal can be any volume value less than the second volume threshold, which can be 0 or the first volume threshold, without specific limitation. The playback of the first audio signal and the playback of the second audio signal can be performed in two different threads of the mobile terminal, respectively. Exemplarily, the MediaPlayer class supports the simultaneous playback of multiple audio signals. The mobile terminal can start a MediaPlayer class as the first audio playback thread to play the first audio signal, and while reducing the playback volume of the first audio signal in the first audio playback thread, start another MediaPlayer class as the second audio playback thread to play the second audio signal.

[0076] That is to say, the mobile terminal can first reduce the playback volume of the first audio signal so that the first audio signal presents a gradually faded-out auditory effect. At the same time, switch to play the second audio signal, and gradually increase the playback volume of the second audio signal so that the second audio signal presents a gradually faded-in auditory effect. In addition, although the fade-in of the second audio signal and the fade-out of the first audio signal can be performed at the same time, the initial volume of the second audio signal can be set to a volume that is difficult for a human ear to hear, so that the initial stage of the fade-in of the second audio signal will not be heard by the user, and the final user will feel the auditory effect of the fade-out of the first audio signal and the fade-in of the second audio signal being seamlessly connected.

[0077] 450. Continue to play the first audio signal corresponding to the first screen partition.

[0078] In step 450, the mobile terminal may continue to play the first audio signal. If the first audio signal is about to be played, and the first screen partition corresponds to multiple audio signals, the mobile terminal may switch to play another third audio signal corresponding to the first screen partition when the first audio signal is played. If the first screen partition only corresponds to one first audio signal, the mobile terminal may recycle and play the first audio signal when the first audio signal is played.

[0079] In one embodiment, the mobile terminal may continue to play the first audio signal in the first audio playback thread, and when the first audio signal is finished playing, directly restart the first audio signal in the first audio playback thread to play the first audio signal in a loop.

[0080] In another embodiment, the mobile terminal may first continue to play the first audio signal in the first audio playback thread, and when the remaining playback time of the first audio signal played in the first audio playback thread is less than the duration threshold, the playback volume of the first audio signal is gradually reduced in the first audio thread until the playback volume of the first audio signal is reduced to the first volume threshold. Furthermore, when the remaining playback time of the first audio signal played in the first audio playback thread by the mobile terminal is less than the duration threshold, the first audio signal is played in the second audio playback thread, and the playback volume of the first audio signal in the second audio playback thread is gradually increased until the playback volume of the first audio signal is increased to the second volume threshold.

[0081] The second volume threshold may be greater than the first volume threshold. The setting of the second volume threshold and the first volume threshold may refer to the description of the aforementioned embodiment, which will not be described in detail in the following content.

[0082] In addition, the mobile terminal can start the second audio playback thread when the playback of the first audio signal in the first audio playback thread is about to end. Exemplarily, the MediaPlayer class supports simultaneous playback of multiple audio signals. The mobile terminal can start a MediaPlayer class as the first audio playback thread to play the first audio signal, and when the remaining playback time of the first audio signal in the first audio playback thread is less than the duration threshold, start another MediaPlayer class as the second audio playback thread to continue playing the first audio signal.

[0083] That is to say, in order to achieve a better loop playback effect, the mobile terminal can also reduce the playback volume in the currently playing first audio playback thread when the playback of the first audio signal is about to end (for example, when the first audio signal has only 4 seconds left to be played), so as to achieve the auditory effect of the audio signal played by the first audio playback thread fading out. At the same time, another second audio playback thread is started to replay the first audio signal, and the playback volume is increased in the second audio playback thread to achieve the auditory effect of the audio signal played by the second audio playback thread fading in. The fading in and out of the first audio signal in different audio playback threads can create a smooth loop playback effect for the user.

[0084] It can be seen that in the aforementioned embodiment, when the screen content seen by the user changes, the mobile terminal can switch to play different audio signals according to the changes in the screen partition currently seen by the user, so that the visual effect of the spatial screen seen by the user on the screen and the auditory effect of the audio signal heard are correlated with each other, creating a better audio-visual effect for the user, making it easier for the user to enter the scene atmosphere created by the application. In addition, when the mobile terminal switches to play different audio signals, or when playing the same audio signal in a loop, a smooth transition of the audio signal can be achieved through the fade-in and fade-out effects of the sound.

[0085] In one embodiment, see Figure 6 , Figure 6 The method flow chart of another audio playback method disclosed in an embodiment is a schematic diagram, and the method can be applied to any of the aforementioned mobile terminals. Figure 6 As shown, the audio playing method may include the following steps:

[0086] 610. Detect input spatial image triggering operation.

[0087] In the embodiment of the present application, the spatial screen and the audio signals respectively corresponding to the respective screen partitions included in the spatial screen may be multimedia resources in an application loaded on the mobile terminal.

[0088] Therefore, in step 610, the spatial screen triggering operation may be an operation of the user opening the application. Alternatively, the application may provide multiple different scene modes, each scene mode may correspond to a spatial screen, and different scene modes may correspond to different spatial screens. Accordingly, in step 610, the spatial screen triggering operation may also be an operation of the user triggering the entry into a certain scene mode after the application has been opened, which is not specifically limited.

[0089] For example, see Figure 7 , Figure 7 is an example diagram of a display interface of different scene modes disclosed in an embodiment. Figure 7As shown, after opening the application, mode icons 710 corresponding to multiple scene modes may be displayed on the screen of the mobile terminal. When the user clicks the scene mode icon 710, the mobile terminal may detect the spatial screen trigger operation.

[0090] It should be noted that the spatial screen trigger operation can be any touch operation input on the screen of the mobile terminal, such as clicking, long pressing, dragging, moving, etc., or it can be a user operation input through voice, gesture, etc., without specific limitation.

[0091] 620. In response to a spatial screen triggering operation, audio signals corresponding to respective screen partitions included in the spatial screen are loaded into a local storage space of the mobile terminal.

[0092] In step 620, the mobile terminal can determine the spatial screen to be displayed according to the spatial screen trigger operation, and download the audio signals corresponding to the various screen partitions included in the spatial screen from the network devices such as the service device, and load the downloaded audio signals to the local storage space of the mobile terminal. That is, the audio signal can be preloaded first. When the spatial screen includes many screen partitions and the data volume of the audio signal is large, preloading the audio signal is beneficial to improve the response speed of the mobile terminal when switching the audio signal according to the change of the spatial screen, reduce the loading waiting time of the audio signal, make the switching transition of the audio signal more natural and smooth, and improve the user experience.

[0093] In addition, the mobile terminal can completely load the audio signals corresponding to each screen partition into the current storage space of the mobile terminal. Alternatively, the mobile terminal can also only load the audio data corresponding to the first n seconds of each audio signal, where n is a positive integer. That is, the mobile terminal can preload part of the audio data of each audio signal, and the remaining unloaded audio data can be loaded after switching to play the audio signal.

[0094] 630. In response to the spatial picture triggering operation, the picture content of the initial picture partition preset by the spatial picture is displayed on the screen, and the initial audio signal corresponding to the initial picture partition is played.

[0095] In step 630, after detecting the spatial screen triggering operation, the mobile terminal may output the spatial screen corresponding to the spatial screen triggering operation on the screen. At this time, the display of the spatial screen is in an initial state, and the screen partition displayed in the target area of ​​the screen may be a preset initial screen partition among the multiple screen partitions included in the spatial screen. In addition, the mobile terminal plays the initial audio signal corresponding to the initial screen partition.

[0096] If the mobile terminal does not detect the first touch operation input on the screen, the mobile terminal can play the initial audio signal in a loop. The implementation of looping the initial audio signal can refer to the implementation of looping the first audio signal in the above embodiment, and the following content will not be repeated.

[0097] If the mobile terminal detects a first touch operation input on the screen, the mobile terminal may execute the following steps 640 to 650 to move the image content displayed on the screen according to the first touch operation and determine whether it is necessary to switch to play a different audio signal.

[0098] It should be noted that the aforementioned step 620 and step 630 may be executed simultaneously, or step 620 may be executed first and then step 630, without any specific limitation.

[0099] 640 . Detect a first touch operation input, and in response to the first touch operation, move the screen content displayed on the screen of the spatial screen according to an instruction of the first touch operation.

[0100] 650. After the picture content of the spatial picture displayed on the screen changes, determine whether the picture partition displayed on the target area of ​​the screen changes; if the picture partition displayed on the current area switches from the first picture partition to the second picture partition, execute step 660; if the picture partition displayed on the target area remains the first picture partition, execute step 670.

[0101] 660. Switch the played audio signal from the first audio signal corresponding to the first screen partition to the second audio signal corresponding to the second screen partition.

[0102] 670. Continue playing the first audio signal corresponding to the first screen partition.

[0103] The implementation of steps 640 to 670 may refer to the implementation of steps 430 to 450 in the aforementioned embodiment, and will not be described in detail below.

[0104] 680. Detect a second touch operation input, and determine a target object indicated by the second touch operation from the picture content displayed on the screen in response to the second touch operation.

[0105] In step 680, the screen content displayed on the screen may include one or more screen objects that can trigger a jump to a different information interface. The user can trigger the jump to the information interface corresponding to the screen object by a second touch operation output on the screen such as clicking, long pressing, dragging, etc.; alternatively, the user can also input a second touch operation through voice, gesture, etc. to trigger a jump to the corresponding information interface.

[0106] When the mobile terminal detects the second touch operation, it can determine the target object indicated by the second touch operation according to the touch position of the second touch operation on the screen and the display positions of various screen objects included in the spatial screen on the screen.

[0107] 690. Display an information interface corresponding to the target object on the screen.

[0108] The information interface corresponding to the target object can be used to display preset pictures, texts and other content to provide users with more information and increase the fun of interaction.

[0109] For example, see Figure 8 , Figure 8 Another example of displaying a spatial image on a screen is disclosed in an embodiment. Figure 8 As shown, the image content displayed on the screen of the space image includes three image objects 810 shaped like triangular stones, and each image object 810 may correspond to a different information interface. The image content displayed on the screen of the space image may also include soil, pool water, etc. These contents may be simple image contents and are not used to trigger the jump of the information interface.

[0110] When the target object indicated by the second touch operation 820 input by the user on the screen is a certain screen object 810 , the mobile terminal may output an information interface 830 corresponding to the screen object 810 .

[0111] It should be noted that there is no logically necessary order between the aforementioned steps 640-670 and steps 680-690. After executing step 630, if the mobile terminal detects a first touch operation, it can continue to execute steps 640-670; if it detects a second touch operation, it can continue to execute steps 680-690.

[0112] In addition, when the mobile terminal is executing steps 680-690, the mobile terminal can continue to play the audio signal corresponding to the screen partition currently displayed in the target area of ​​the screen, so that the user can use the audio signal being played as background music while reading the information interface.

[0113] It can be seen that in the aforementioned embodiment, the mobile terminal can preload the various audio signals corresponding to the spatial screen when detecting the spatial screen trigger operation, so as to reduce the waiting time when the subsequent audio signals are switched, and improve the user experience. In addition, the mobile terminal can display the spatial screen on the screen in the initial state, and play the initial audio signal corresponding to the initial screen partition displayed in the target area. Thereafter, if a first touch operation for moving the screen content is detected, different audio signals can be switched to play according to the changes in the screen content displayed on the screen, so as to create a better audio-visual effect for the user; if a second touch operation for triggering a jump to the information interface is detected, the corresponding information interface can be displayed on the screen, providing the user with more content and improving the fun of the interaction.

[0114] In the above embodiments, the mobile terminal can play the corresponding audio signal according to the screen content displayed on the screen. It should be noted that the image rendering when the spatial image is displayed and the data processing when the audio signal is played have certain requirements on the computing power of the mobile terminal. Therefore:

[0115] In some embodiments, if the computing capability of the mobile terminal is at the first level, the picture content change of the spatial picture is achieved through real-time rendering of the picture, and the audio signal corresponding to each picture partition is in a first data format, which is an uncompressed audio data format. For example, the audio signal can be a Dolby audio source signal in MP4 format.

[0116] In other embodiments, if the computing power of the mobile terminal is at the second level, the change of the picture content of the spatial screen is achieved by playing video animation, and the audio signal corresponding to each screen partition is in a second data format, which is a compressed audio data format. For example, the audio signal can be a sound source signal in OGG format.

[0117] Among them, the computing power corresponding to the aforementioned second level is lower than the computing power corresponding to the first level. That is to say, if the computing power of the mobile terminal is strong, for example, it is equipped with a better computing chip or processor, the mobile terminal can achieve the change of the picture content displayed on the screen by real-time rendering of different picture contents, and play uncompressed audio signals with good sound quality; if the computing power of the mobile terminal is weak, the mobile terminal can achieve the change of the picture content displayed on the screen by playing a pre-designed animation video, and play a compressed audio signal with a small data volume, so that the audio playback method disclosed in the embodiment of the present application can achieve the effect of smooth operation and rapid response when applied on various mobile terminals with different computing powers.

[0118] See also Fig. 9 , Fig. 9FIG. 1 is a schematic diagram of a structure of an audio playback device disclosed in an embodiment, and the audio playback device can be applied to a mobile terminal. Fig. 9 As shown, the audio playback device 900 may include: a display module 910 and a playback module 920 .

[0119] The display module 910 is used to display the picture content of at least part of the picture partition of the spatial picture on the screen of the mobile terminal; the spatial picture is divided into at least two different picture partitions, each picture partition corresponds to at least one audio signal;

[0120] The playing module 920 is used to play the audio signal corresponding to the screen partition displayed in the target area of ​​the screen after the screen content displayed on the screen changes.

[0121] In one embodiment, the audio playback device 900 may further include: an adjustment module.

[0122] The adjustment module can be used to move the picture content displayed on the screen of the space picture in response to the first touch operation before the picture content displayed on the screen changes; or move the picture content displayed on the screen of the space picture according to a preset movement rule.

[0123] In one embodiment, the playback module 920 can also be used to switch the played audio signal from the first audio signal to the second audio signal after the picture content displayed on the screen changes, if the picture partition displayed on the target area of ​​the screen switches from the first picture partition to the second picture partition; the first audio signal corresponds to the first picture partition, and the second audio signal corresponds to the second picture partition.

[0124] In one embodiment, the playback module 920 may also be used to gradually reduce the playback volume of the first audio signal until the playback volume of the first audio signal is reduced to a first volume threshold; and, when gradually reducing the playback volume of the first audio signal, switch to playing the second audio signal and gradually increase the playback volume of the second audio signal until the playback volume of the second audio signal is increased to a second volume threshold; the second volume threshold is greater than the first volume threshold.

[0125] In one embodiment, the playback module 920 may also be used to continue playing the first audio signal corresponding to the first screen partition if the screen partition displayed on the target area of ​​the screen remains the first screen partition after the screen content displayed on the screen changes; the screen partition displayed on the target area of ​​the screen before the screen content changes is the first screen partition.

[0126] In one embodiment, the playback module 920 may also be used to continue playing the first audio signal in the first audio playback thread; and, when the remaining playback time of the first audio signal played in the first audio playback thread is less than a duration threshold, gradually reduce the playback volume of the first audio signal in the first audio thread until the playback volume of the first audio signal played in the first audio playback thread is reduced to a first volume threshold; and, when the remaining playback time of the first audio signal played in the first audio playback thread is less than the duration threshold, play the first audio signal in the second audio playback thread, and gradually increase the playback volume of the first audio signal in the second audio playback thread until the playback volume of the first audio signal played in the second audio playback thread is increased to a second volume threshold; the second volume threshold is greater than the first volume threshold.

[0127] In one embodiment, the display module 910 may also be used to display at least part of the content of the spatial screen on the screen in response to an input spatial screen trigger operation; wherein the screen partition currently displayed in the target area of ​​the screen is a preset initial screen partition;

[0128] The playing module 920 may also be configured to play an initial audio signal corresponding to an initial screen partition in response to a spatial screen trigger operation.

[0129] In one embodiment, the audio playback device 900 may further include: a loading module.

[0130] The loading module can be used to load the audio signals corresponding to the various screen partitions included in the spatial screen into the local storage space of the mobile terminal in response to the input spatial screen trigger operation.

[0131] In one embodiment, if the computing capability of the mobile terminal is at the first level, the picture content change of the spatial picture is realized by real-time rendering of the picture, and the audio signal corresponding to each picture partition is in a first data format, which is an uncompressed audio data format;

[0132] If the computing power of the mobile terminal is at the second level, the change of the picture content of the spatial screen is realized by playing video animation, and the audio signal corresponding to each screen partition is in the second data format, which is a compressed audio data format; the computing power corresponding to the second level is lower than the computing power corresponding to the first level.

[0133] In one embodiment, the display module 910 can also be used to determine the target object indicated by the second touch operation from the picture content displayed on the screen in response to the second touch operation; and display an information interface corresponding to the target object on the screen.

[0134] In the aforementioned embodiment, the spatial screen can be divided into a plurality of different screen partitions, and the audio playback device can display the screen content of at least part of the screen partitions on the screen of the mobile terminal. Therefore, when the screen content displayed on the screen changes, the user can see the screen content that has not been displayed before, and the screen content that has been displayed before may also become invisible, visually creating a 360° scene space sense. At the same time, the audio signal played by the mobile terminal can change with the change of the screen content, and in terms of hearing, it can cooperate with the change of the visual scene, so that the scene space sense created from both audio and video aspects is more real and natural.

[0135] See also Fig.10 , Fig.10 FIG. 1 is a schematic diagram of a mobile terminal disclosed in an embodiment. Fig.10 As shown, the mobile terminal 1000 may include:

[0136] A memory 1010 storing executable program codes;

[0137] a processor 1020 coupled to the memory 1010;

[0138] The processor 1020 calls the executable program code (computer program) stored in the memory 1010 to execute any one of the audio playback methods disclosed in the embodiments of the present application.

[0139] It should be noted that Fig.10 The mobile terminal shown may also include components not shown, such as a power supply, input buttons, a camera, a speaker, a screen, an RF circuit, a Wi-Fi module, a Bluetooth module, and sensors, which are not described in detail in this embodiment.

[0140] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor implements any one of the audio playback methods disclosed in the embodiment of the present application.

[0141] An embodiment of the present application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to implement and execute any one of the audio playback methods disclosed in the embodiment of the present application.

[0142] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.

[0143] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0144] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0146] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above-mentioned methods of various embodiments of the present application.

[0147] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0148] The above is a detailed introduction to an audio playback method, device, mobile terminal and storage medium disclosed in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those of ordinary skill in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An audio playback method, characterized in that: Applied to a mobile terminal, the method comprises: Displaying the picture content of at least part of the picture partition of the spatial picture on the screen of the mobile terminal; the spatial picture is a panoramic image, the spatial picture is divided into at least two different picture partitions, and each picture partition corresponds to at least one audio signal; If the picture content displayed on the screen changes, then playing the audio signal corresponding to the picture partition displayed in the target area of ​​the screen after the picture content changes; If the computing power of the mobile terminal is at the first level, the change of the picture content of the spatial screen is achieved through real-time rendering of the screen, and the audio signal corresponding to each screen partition is in a first data format, which is an uncompressed audio data format; if the computing power of the mobile terminal is at the second level, the change of the picture content of the spatial screen is achieved through playing video animation, and the audio signal corresponding to each screen partition is in a second data format, which is a compressed audio data format, and the computing power corresponding to the second level is lower than the computing power corresponding to the first level.

2. The method according to claim 1, characterized in that Before the picture content displayed on the screen changes, the method further includes: In response to the input of the first touch operation, moving the picture content of the space picture displayed on the screen; or, The picture content of the space picture displayed on the screen is moved according to a preset movement rule.

3. The method according to claim 1 or 2, characterized in that: If the picture content displayed on the screen changes, playing the audio signal corresponding to the picture partition displayed in the target area of ​​the screen after the picture content changes, comprises: If the picture content displayed on the screen changes, the picture partition displayed on the target area of ​​the screen is switched from the first picture partition to the second picture partition, and the played audio signal is switched from the first audio signal to the second audio signal; the first audio signal corresponds to the first picture partition, and the second audio signal corresponds to the second picture partition.

4. The method according to claim 3, characterized in that The step of switching the played audio signal from the first audio signal to the second audio signal comprises: gradually reducing the playback volume of the first audio signal until the playback volume of the first audio signal is reduced to a first volume threshold; When the playback volume of the first audio signal is gradually reduced, the second audio signal is switched to be played, and the playback volume of the second audio signal is gradually increased until the playback volume of the second audio signal is increased to a second volume threshold; the second volume threshold is greater than the first volume threshold.

5. The method according to claim 1 or 2, characterized in that: If the picture content displayed on the screen changes, playing the audio signal corresponding to the picture partition displayed in the target area of ​​the screen after the picture content changes, comprises: If the picture partition displayed on the target area of ​​the screen remains the first picture partition after the picture content displayed on the screen changes, the first audio signal corresponding to the first picture partition continues to be played; the picture partition displayed on the target area of ​​the screen before the picture content changes is the first picture partition.

6. The method according to claim 5, characterized in that The continuing to play the first audio signal corresponding to the first screen partition includes: Continue playing the first audio signal in the first audio playback thread; And, the method further comprises: When the remaining playing time of the first audio signal played in the first audio playing thread is less than a time threshold, gradually reducing the playing volume of the first audio signal in the first audio playing thread until the playing volume of the first audio signal played in the first audio playing thread is reduced to a first volume threshold; When the remaining playback time of the first audio signal played in the first audio playback thread is less than the time threshold, the first audio signal is played in the second audio playback thread, and the playback volume of the first audio signal in the second audio playback thread is gradually increased until the playback volume of the first audio signal played in the second audio playback thread is increased to a second volume threshold; the second volume threshold is greater than the first volume threshold.

7. The method according to claim 1, characterized in that The displaying of the screen content of at least a part of the screen partition of the space screen on the screen of the mobile terminal includes: In response to an input space screen trigger operation, displaying the screen content of the initial screen partition preset by the space screen on the target area of ​​the mobile terminal screen; Furthermore, before the content of the picture displayed on the screen changes, the method further includes: In response to the spatial picture triggering operation, an initial audio signal corresponding to the initial picture partition is played.

8. The method according to claim 1 or 7, characterized in that: Before displaying the screen content of at least part of the screen partition of the spatial screen on the screen of the mobile terminal, the method further includes: In response to the spatial screen triggering operation, the audio signals corresponding to the respective screen partitions included in the spatial screen are loaded into the local storage space of the mobile terminal.

9. An audio playback device, characterized in that: Applied to a mobile terminal, the device comprises: A display module, configured to display the picture content of at least a part of the picture partition of the spatial picture on the screen of the mobile terminal; the spatial picture is a panoramic image, the spatial picture is divided into at least two different picture partitions, each picture partition corresponds to at least one audio signal; A playing module, used for playing an audio signal corresponding to a screen partition displayed in a target area of ​​the screen after the screen content displayed on the screen changes, after the screen content displayed on the screen changes; If the computing power of the mobile terminal is at the first level, the change of the picture content of the spatial screen is achieved through real-time rendering of the screen, and the audio signal corresponding to each screen partition is in a first data format, which is an uncompressed audio data format; if the computing power of the mobile terminal is at the second level, the change of the picture content of the spatial screen is achieved through playing video animation, and the audio signal corresponding to each screen partition is in a second data format, which is a compressed audio data format, and the computing power corresponding to the second level is lower than the computing power corresponding to the first level.

10. A mobile terminal, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images