Multi-sound-source output control method and device, equipment and storage medium

Abstract

The invention discloses a multi-sound-source output control method and device, equipment and a storage medium, and the method comprises the steps: responding to a mode selection operation for a multi-sound-source target output mode, and displaying the visual identifications of a plurality of sound sources at different sound source space positions of a graphic display interface according to the position information of the plurality of sound sources when the target output mode is used. Thus, different sound sources are distributed to different virtual space positions in a multi-sound-source output mode, multi-sound-source data streams are not interfered with each other and are independent and controllable, the sound source auditory sense separated playing effect is achieved, the complex sound source space position relation is visually displayed through a visual graphic display interface according to different output modes, and the sound source auditory sense separated playing effect is achieved. A user is helped to quickly know the spatial position relation of multiple sound sources, and application and popularization of the holographic audio technology are facilitated.

Description

Technical Field

[0001] The present application relates to holographic audio technology, and in particular to a multi-sound source output control method, device, equipment and storage medium. Background Art

[0002] In various smart devices currently, when the system plays multiple sound sources at the same time, some systems directly linearly superimpose the data of different sound sources and then play them, while some systems adjust the volume of each sound source to reduce the volume of some sound sources and highlight the volume of other sound sources. However, in these processing methods, each sound source is in the same position and has no difference in spatial dimension, so there is still interference problem between multiple sound sources. Summary of the invention

[0003] The embodiments of the present application are intended to provide a method, apparatus, device and storage medium for controlling multiple audio source outputs.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, a multi-source output control method is provided, comprising:

[0006] In response to a mode selection operation for a multi-source target output mode, obtaining position information of a plurality of sound sources when the target output mode is used;

[0007] According to the position information of the multiple sound sources, visual identifiers of the multiple sound sources are displayed at different sound source spatial positions on the graphic display interface.

[0008] In a second aspect, a multi-source output control device is provided, comprising:

[0009] The acquisition unit is further used to, in response to a mode selection operation for a multi-source target output mode, acquire position information of multiple sound sources when the target output mode is used;

[0010] The graphic display unit is further used to display the visual identifications of the multiple sound sources at different sound source spatial positions on the graphic display interface according to the position information of the multiple sound sources.

[0011] In a third aspect, a multi-source output control device is provided, comprising: a processor and a memory configured to store a computer program that can be run on the processor,

[0012] Wherein, the processor is configured to execute the steps of the aforementioned method when running the computer program.

[0013] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program implements the steps of the aforementioned method when executed by a processor.

[0014] The present application provides a multi-source output control method, device, equipment and storage medium, the method comprising: in response to a mode selection operation for a multi-source target output mode, displaying visual identifications of multiple sound sources at different sound source spatial positions on a graphic display interface according to the position information of multiple sound sources when using the target output mode. In this way, different sound sources are assigned to different virtual spatial positions using a multi-source output mode, so that the multi-source data streams do not interfere with each other and are independently controllable, thereby achieving a playback effect of audio source separation. For different output modes, the complex spatial position relationship of the sound source is intuitively displayed through a visual graphic display interface, helping users quickly understand the spatial position relationship of multiple sound sources, which is conducive to the application and promotion of holographic audio technology.

[0015] Furthermore, the graphical display interface is conducive to quickly and accurately adjusting the spatial position of the sound source, thereby achieving personalized customization for multi-source output. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the first process of the multi-source output control method in an embodiment of the present application;

[0017] Figure 2 A graphical display of the spatial position relationship of multiple sound sources in the embodiment of the present application Figure 1 ;

[0018] Figure 3 A graphical display of the spatial position relationship of multiple sound sources in the embodiment of the present application Figure 2 ;

[0019] Figure 4A A graphical display of the audio source playback status in the embodiment of the present application Figure 1 ;

[0020] Figure 4B A graphical display of the audio source playback status in the embodiment of the present application Figure 2 ;

[0021] Figure 5 This is a second flow chart of the multi-sound source output control method in an embodiment of the present application;

[0022] Figure 6 A schematic diagram of the spatial position relationship of the sound source in the intelligent mode in the embodiment of the present application;

[0023] Figure 7 A graphical display of the position change strategy in the embodiment of the present application Figure 1 ;

[0024] Figure 8 A graphical display of the position change strategy in the embodiment of the present application Figure 2 ;

[0025] Fig. 9 A graphical display of the position change strategy in the embodiment of the present application Figure 3 ;

[0026] Fig.10 Schematic diagram 4 of the graphical display of the position change strategy in the embodiment of the present application;

[0027] Fig.11 A graphical display of the position change strategy in the embodiment of the present application Figure 5 ;

[0028] Fig.12 This is a schematic diagram of adjusting the sound source position in the intelligent mode in an embodiment of the present application;

[0029] Fig.13 This is a third flow chart of the multi-sound source output control method in an embodiment of the present application;

[0030] Fig.14 The interface display diagram of the custom mode in the embodiment of the present application is shown as follows Figure 1 ;

[0031] Fig.15 The interface display diagram of the custom mode in the embodiment of the present application is shown as follows Figure 2 ;

[0032] Fig.16 This is a graphical display diagram of a sound source selected in the embodiment of the present application. Figure 1 ;

[0033] Fig.17 This is a graphical display diagram of a sound source selected in the embodiment of the present application. Figure 2 ;

[0034] Fig.18 This is a graphical display diagram of a sound source selected in the embodiment of the present application. Figure 3 ;

[0035] Fig.19 Schematic diagram of visual identification of each sound source in different display states in the embodiment of the present application;

[0036] Fig. 20 A visual representation of the volume adjustment item in the embodiment of the present application Figure 1 ;

[0037] Fig.21 A visual representation of the volume adjustment item in the embodiment of the present application Figure 2 ;

[0038] Fig. 22 A visual representation of the volume adjustment item in the embodiment of the present application Figure 3 ;

[0039] FIG. 23A to FIG. 23E A visual flowchart of the mode selection process in the embodiment of the present application;

[0040] Fig.24 This is a schematic diagram of the implementation process of an interface display method in an embodiment of the present application;

[0041] Fig.25 This is a schematic diagram of the implementation flow of another interface display method in an embodiment of the present application;

[0042] Fig.26 This is a schematic diagram of the composition structure of the multi-sound source output control device in an embodiment of the present application;

[0043] Fig. 27 This is a schematic diagram of the composition structure of a multi-source output control device in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0045] Figure 1 FIG. 1 is a schematic diagram of a first flow chart of a multi-source output control method in an embodiment of the present application. Figure 1 As shown, the method may specifically include:

[0046] Step 101: In response to a mode selection operation for a multi-source target output mode, obtaining position information of multiple sound sources when using the target output mode;

[0047] In the embodiment of the present application, the multi-source target output mode can be any output mode that realizes the holographic audio function. Holographic audio is an audio technology that can present music or sound through holographic imaging technology. This sound can come from different directions, distances and heights, giving the audience an immersive experience. When realizing the holographic audio function, the embodiment of the present application sets a corresponding virtual space position for the original audio data of each sound source, and then processes the position information and the original audio data through a spatial audio rendering algorithm, so that each sound source seems to be emitted from different positions, so that the sounds of multiple sound sources can be separated, solving the problem of interference between multiple audios playing simultaneously.

[0048] In the embodiment of the present application, the sound source may specifically include: incoming call ringtones, alarm clocks, game voice, video, audiobooks, music, notifications, and navigation. The graphic display interface may display the spatial position relationship of all predefined sound sources. When one or more sound sources enter the play state, the graphic display interface may also display the spatial position relationship of the visual identification of one or more sound sources entering the play state on the graphic display interface, and delete the visual identification of the sound source when exiting the play state.

[0049] It should be noted that the mode selection operation is used to select the target output mode and obtain the position information of multiple sound sources when using the target output mode. The user interface is rendered according to the position information of each sound source to obtain a graphical display interface. The visual identification of multiple sound sources is displayed at different sound source spatial positions in the graphical display interface, and the visual identification of different sound sources is different.

[0050] In some embodiments, the position information of the multiple sound sources may be default position information of each sound source.

[0051] In other embodiments, the location information of the multiple sound sources may be historical location information set last time.

[0052] Step 102: Displaying visual identifiers of multiple sound sources at different sound source spatial positions on a graphic display interface according to the position information of the multiple sound sources.

[0053] Exemplarily, the plurality of sound sources include at least a first sound source and a second sound source, and the graphical display interface includes a visual identifier of the first sound source displayed at a first position, and a visual identifier of the second sound source displayed at a second position, and the first position and the second position are different.

[0054] It should be noted that the graphic display interface shows the spatial position relationship of multiple sound sources when using the target output mode. The complex spatial position relationship of the sound sources is intuitively displayed through visual graphics, which is conducive to quickly and accurately adjusting the spatial position of the sound source, thereby realizing personalized customization for multi-sound source output.

[0055] The multiple sound sources include non-focus sound sources, and the visual identifiers of the non-focus sound sources in the graphical display interface are distributed around the center position of the sound source space. The sound source space can be the geometric center of the three-dimensional space. In practical applications, the center position can be the user position in the sound source space.

[0056] In some embodiments, the sound source space may be a spherical three-dimensional space, a hemispherical three-dimensional space, a cubic three-dimensional space, etc., and the graphical display interface displays the three-dimensional spatial position relationship of multiple sound sources, and the position information of the corresponding sound sources may include three-dimensional coordinates. In other embodiments, the sound source space may be a two-dimensional space, and the graphical display interface displays the two-dimensional spatial position relationship of multiple sound sources, and the corresponding position information may include two-dimensional coordinates.

[0057] Figure 2 A graphical display of the spatial position relationship of multiple sound sources in the embodiment of the present application Figure 1 .like Figure 2 As shown, a two-dimensional spatial position relationship of sound sources is shown, where 1-7 correspond to the positions of 7 sound sources, and each sound source is distributed around the center position.

[0058] In some embodiments, when the multiple sound sources also include a focus sound source, a visual identifier of the focus sound source is displayed at the focus sound source position of the graphical display interface; wherein the focus sound source position is the center position or has a certain distance from the center position.

[0059] The focus source position can be a position with a certain distance and azimuth relative to the center position. For example, the focus source position can be at a certain distance from the center position along the Y-axis direction. The focus source position can also be the center position, that is, the focus source position can coincide with the center position. The sound source with the focus source position is called the focus source. When multiple sound sources play sounds at the same time, there is only one focus source.

[0060] In some embodiments, a central visual identifier is displayed at the center of the graphical display interface, and the central visual identifier has directionality. Exemplarily, the central visual identifier may be an arrow, and the arrow points to the direction of the user's head. The central visual identifier may also be a user identifier. The central visual identifier with directionality enables the user to quickly and accurately perceive the position of each sound source relative to the user.

[0061] When multiple sound sources also include a focus sound source, and the focus sound source position is the center position, the visual logo of the focus sound source is displayed at the center position, and a directional logo can also be added to the visual logo of the focus sound source. When multiple sound sources also include a focus sound source, and the focus sound source position is not the center position, the visual logo of the focus sound source is displayed at the focus sound source position, and the center visual logo is displayed at the center position. When multiple sound sources also include a focus sound source, and the focus sound source position is not the center position, when the focus sound source position is very close to the center position, the visual logo of the focus sound source can be displayed only at the focus sound source position to ensure the graphic display effect.

[0062] In some embodiments, the method may further include: determining the distance and / or azimuth of the sound source relative to the center position based on the audio characteristics of the sound source; and determining the position information of each sound source based on the distance and / or azimuth of each sound source relative to the center position.

[0063] The smaller the distance between each sound source and the center position, the louder and clearer the sound will be. Conversely, the smaller the sound volume, the blurrier it will be. When the distance from the center position is the same, the playback effect of the sound source at different azimuths will also be different. In other words, the playback effect of each sound source is affected not only by the distance, but also by the azimuth. By adjusting the distance and azimuth of each sound source relative to the center position, the playback effect of each sound source can be changed.

[0064] In some embodiments, when the sound source space is a three-dimensional space, the sound source with a first audio characteristic is located in the first direction or the second direction of the center position, and the sound source with a second audio characteristic is located at a position having a certain angle relative to the first direction or the second direction of the center position.

[0065] In some embodiments, when the sound source space is a three-dimensional space, the sound source with a first audio characteristic is located in the first direction, the second direction or the third direction of the center position, and the sound source with a second audio characteristic is located at a position having a certain angle relative to the first direction, the second direction or the third direction of the center position.

[0066] Exemplarily, the first direction, the second direction and the third direction correspond to the horizontal direction, the vertical direction and the depth direction respectively, and the first audio feature can be a rich and diverse audio feature. Such one or more sound sources are set in the horizontal direction, the vertical direction and the depth direction of the focus sound source, which has a better sound effect. The second audio feature can be a single audio feature. Such one or more sound sources are set at a position with a certain angle relative to a specific direction of the focus sound source, which can reduce interference with other sound sources on the basis of ensuring its sound effect, so that the sound source distribution in the multi-sound source space is balanced.

[0067] like Figure 2 The two-dimensional space of sound sources shown is centered on the focus sound source, with music, video, call and other sound sources located in the horizontal and vertical directions, and alarm clocks, notifications, audiobooks, navigation and other sound sources located at a certain angle relative to the horizontal and vertical directions of the focus sound source.

[0068] Figure 3 A graphical display of the spatial position relationship of multiple sound sources in the embodiment of the present application Figure 2 .like Figure 3 As shown, the graphical display interface displays the three-dimensional spatial position relationship of multiple sound sources, and the three-dimensional space takes a hemispherical three-dimensional space as an example. According to the three-dimensional coordinates of each sound source, the visual identification of each sound source is displayed in the hemispherical three-dimensional space, and the user identification is displayed at the center of the hemispherical space. The visual identification of each sound source is distributed around the center, and the visual identification of different sound sources is different. Figure 3 The visual recognition and spatial location of sound sources such as incoming ringtones, alarms, game voices, videos, audio books, music, notifications and navigation are given as examples.

[0069] In some embodiments, the method further includes: in response to a play operation on the first sound source, displaying a visual identifier of the first sound source in a play state at a first position of the graphical display interface.

[0070] Exemplarily, in response to a play operation on the first sound source, the visual identifier of the first sound source is adjusted from the unplayed state to the played state in the graphic display interface; in response to a stop play operation on the first sound source, the visual identifier of the first sound source is adjusted from the played state to the unplayed state in the graphic display interface. That is, when the sound source is in the unplayed state, the visual identifier of the unplayed state is displayed at the default position of the graphic display interface, and when the sound source is in the playing state, the visual identifier of the playing state is displayed at the default position of the graphic display interface, and the visual identifier of the unplayed state is different from the visual identifier of the playing state.

[0071] Exemplarily, when the first sound source is in a playing state, the visual identifier of the first sound source is superimposed with an identifier representing the playing state, otherwise the identifier representing the playing state is removed. The identifier representing the playing state may be a dynamic identifier or a static identifier.

[0072] In some embodiments, the playing state includes a circular sound wave dynamic mark or a fan-shaped sound wave dynamic mark.

[0073] Figure 4A A graphical display of the audio source playback status in the embodiment of the present application Figure 1 .like Figure 4A As shown, when the music and navigation enter the playing state, a circular sound wave dynamic logo is superimposed on the music symbol and the navigation symbol.

[0074] Exemplarily, in response to a play operation on a first sound source, a visual identifier of the first sound source is displayed at a first position on the graphical display interface; in response to a stop play operation on the first sound source, the visual identifier of the first sound source is deleted on the graphical display interface. In other words, the visual identifier of the sound source is displayed at a default position on the graphical display interface only when the sound source enters the play state, otherwise it is not displayed.

[0075] Figure 4B A graphical display of the audio source playback status in the embodiment of the present application Figure 2 .like Figure 4B As shown in the figure, when music, navigation, alarm and notification enter the playing state, the music symbol, navigation symbol, alarm symbol and notification symbol will be displayed in the graphic display interface, and other sound sources that have not entered the playing state will not be displayed. For scenes with many sound sources, the display effect of the spatial position relationship of the sound sources can be guaranteed.

[0076] Different output modes correspond to different multi-source playback control methods. In an embodiment of the present application, the multi-source target output mode can be one of the following: intelligent mode and custom mode. In the intelligent mode, the sound source position can be changed based on a preset position transformation strategy, and the audio data can be rendered using the transformed sound source position information. In the custom mode, audio data is rendered according to the user-defined sound source position information. In actual applications, at least one intelligent mode may be included, and different intelligent modes change the sound source position based on different position transformation strategies.

[0077] By adopting the above technical solution, different sound sources are allocated to different virtual space positions using a multi-source output mode, so that the multi-source data streams do not interfere with each other and are independently controllable, thereby achieving the playback effect of separated auditory senses of the sound sources. For different output modes, the complex spatial position relationship of the sound sources is intuitively displayed through a visual graphic display interface, which is conducive to the application and promotion of holographic audio technology.

[0078] In order to better reflect the purpose of the present application, based on the above embodiments of the present application, the target output mode is further illustrated by taking the intelligent mode of changing the sound source position based on the preset position change strategy as an example, such as Figure 5 As shown, the method specifically includes:

[0079] Step 201: In response to a mode selection operation for a multi-source intelligent mode, displaying visual identifiers of multiple sound sources at different sound source spatial positions on a graphical display interface according to position information of multiple sound sources when the intelligent mode is used;

[0080] Figure 6 Schematic diagram of the spatial position relationship of the sound source in the intelligent mode in the embodiment of the present application. Figure 6 As shown, the graphical display interface shows the three-dimensional spatial position relationship of multiple sound sources, and the three-dimensional space takes a hemispherical three-dimensional space as an example. According to the three-dimensional coordinates of each sound source, the visual identification of each sound source is distributed around the center position of the hemisphere and displayed at different positions in the hemispherical three-dimensional space. The position of each sound source can be called the Home position, and the visual identification of different sound sources is different. In the smart mode, multiple sound sources take ringtones, music, alarm clocks, navigation and notifications as examples.

[0081] In smart mode, multiple audio sources include focus audio sources and non-focus audio sources. The position information of each audio source can be arranged in different combinations of audio source positions to perform an auditory test of the audio, thereby determining the position with the best audio effect. When using smart mode for multi-source output, each audio is not fixed at the default position, but changes accordingly following the position change strategy.

[0082] The focus sound source position can also be called (Center position). The focus sound source position can be a position with a certain distance and azimuth relative to the center position. For example, the focus sound source position can be at a certain distance from the center position along the Y-axis direction. The focus sound source position can also be the center position, that is, the focus sound source position can coincide with the center position. The sound source with the focus sound source position is called the focus sound source. When multiple sound sources play sounds at the same time, there is only one focus sound source. In actual applications, if the audio output device is a multi-channel output, the focus sound source position can also include multiple focus sound sources corresponding to different channels at different positions.

[0083] Step 202: In response to a play operation on a first sound source, according to the priority and position change strategy of the first sound source, target position information of the first sound source is determined from position information of the first sound source and position information of a preset focus sound source.

[0084] Step 203: Displaying a visual identifier of the first sound source on a graphic display interface according to the target position information of the first sound source.

[0085] It should be noted that the position change strategy specifically changes between the focus sound source position and the respective default positions according to the priority of each sound source.

[0086] Exemplarily, the position change strategy includes: when there is no focus sound source at the focus sound source position, or when there is a focus sound source at the focus sound source position and the priority of the first sound source is higher than the priority of the focus sound source, determining the target position information of the first sound source as the position information of the focus sound source; when there is a focus sound source at the focus sound source position and the priority of the first sound source is lower than the priority of the focus sound source, determining the target position information of the first sound source as the position information of the first sound source; when there is a focus sound source at the focus sound source position and the priority of the first sound source is equal to the priority of the focus sound source, if the creation time of the first sound source is later than the creation time of the focus sound source, determining the target position information of the first sound source as the position information of the focus sound source.

[0087] That is to say, by judging whether the priority of the sound source entering the playback state is the highest priority, it is determined whether the current sound source should occupy the focus sound source. If the focus sound source position is empty, that is, there is no focus sound source, the current sound source directly occupies the focus sound source. If the focus sound source position is not empty, that is, there is a focus sound source, then when the current sound source has a higher priority than the focus sound source, the focus sound source is first restored from the focus sound source position to the default position, and then the current sound source enters the focus sound source position.

[0088] The playback operation can be to click the "start audition" button on the graphical display interface to audition the playback effects of some or all sound sources under the current spatial position relationship. The playback operation can also be implemented based on preset playback instructions when using the smart mode.

[0089] In some embodiments, the sound source type includes a transient sound source and a non-transient sound source, and the priority of the non-transient sound source is higher than the priority of the transient sound source.

[0090] Among them, transient sound sources can be understood as sound sources that are triggered to play in a short period of time, and the playing time of such sound sources is relatively short. Exemplarily, transient sound sources include at least one of the following: notification, navigation, voice assistant, voice message. The same priority can be set for transient sound sources.

[0091] Non-transient sound sources can be understood as sound sources that are continuously played under certain conditions, and such sound sources have a longer playing time. Exemplarily, non-transient sound sources include at least one of the following: calls, incoming call ringtones, alarm clocks, games, voice calls, videos, audiobooks, and music.

[0092] For non-transient audio sources, different non-transient audio sources may be prioritized. For example, the priority order of non-transient audio is: call > incoming call ringtone > alarm > game > voice call > video > audiobook > music.

[0093] It should be noted that different priorities may be set for transient sound sources, and the same priority may be set for non-transient sound sources.

[0094] For example, the priorities of the five sound sources are as follows: incoming ringtone > alarm > music > notification and navigation. Notification and navigation are transient sound sources, and their priorities are the same and lower than non-transient sound sources. Figures 7 to 9 FIG. 1 is a graphical display diagram of the position change strategy in the embodiment of the present application, and takes the focus sound source position as the center position as an example for illustration. Figure 7 As shown in the figure, when music is currently playing, the focus source position is called the Center position. When the Center position is empty, the music takes over the Center position. At this time, a call ringtone comes in. Because the call ringtone has a higher priority than the music, the music returns to its Home position and the call ringtone enters the Center position. Figure 8 Then the alarm rings and the incoming call ring ends. The alarm takes the center position, the incoming call ring returns to its home position, and the playing animation of the incoming call ring disappears. Fig. 9 shown.

[0095] When the first sound source (first sound source) occupies the focus sound source position, and the priority of the second sound source (second sound source) is equal to the priority of the first sound source, the second sound source occupies the focus sound source position, and the first sound source returns to its default position. If the priority of the second sound source (second sound source) is lower than the priority of the first sound source, the second sound source is in its default position. Among them, the sound sources with the same priority can be the sound sources of the same sound source type.

[0096] Figure 10 to Figure 11 Schematic diagram of the graphical display of the position change strategy in the embodiment of the present application, such as Fig.10 As shown in FIG. 1 , the first music source is played first, and the first music source is in the Center position. Fig.11 As shown, the second music source is played next. Because both music scenes have the same priority, the second music source that is sent later enters the Center position, and the first music source that is sent earlier returns to its Home position.

[0097] In some embodiments, the method further includes: in response to the priority adjustment operation for the first sound source, obtaining the adjusted priority of the first sound source. It should be noted that, since the position information is determined according to the priority of each sound source in the intelligent mode, the adjustment of the playback effect of multiple sound sources can be achieved by adjusting the priority of each sound source, thereby ensuring the playback effect of the highest priority sound source.

[0098] In some embodiments, the method further includes: in response to a selection operation on the visual identifier of the first sound source, calling up a priority adjustment interface, and displaying a priority sorting list of the plurality of sound sources on the priority adjustment interface.

[0099] In some embodiments, the priority adjustment interface and the graphic display interface are in different display areas; or, the priority adjustment interface is displayed in a floating window on the graphic display interface.

[0100] In some embodiments, the method further includes: in response to the position adjustment operation for the visual identifier of the first sound source, the positions of the visual identifier of the first sound source and the visual identifier of the second sound source are swapped on the graphical display interface. That is, in the smart mode, the user can also adjust the default position of each sound source to achieve personalized customization.

[0101] It should be noted that according to the position adjustment operation, the positions of the visual identifier of the first sound source and the visual identifier of the second sound source in the graphical display interface are swapped, that is, the default positions of the two are swapped. When using the smart mode for multi-source output, the default positions of the two are the swapped default positions. Fig.12 As shown, the first sound source is music, the second sound source is an alarm clock, and the default positions of the music and the alarm clock are swapped.

[0102] In some embodiments, before responding to a position adjustment operation on the first sound source, the method also includes: in response to a selection operation on the first sound source, the visual identification of the first sound source is adjusted from a first display state to a second display state in the graphical display interface; wherein the visual identifications of the first display state and the second display state are different.

[0103] In some embodiments, the method may further include: in response to a selection operation on the visual identifier of the first sound source, the visual identifier of the second sound source is adjusted from the first display state to the third display state in the graphical display interface; wherein the third display state includes an identifier indicating that the second sound source is allowed to be swapped with the first sound source. Exemplarily, different display states may be distinguished by texture features, color features, addition and deletion features, dynamic and static features, etc.

[0104] That is to say, depending on whether the user manually selects the sound source, the sound source is divided into a selected state (corresponding to the first display state of the visual identification) and an unselected state (corresponding to the second display state of the visual identification). Different display states are used to distinguish whether the sound source is selected for position adjustment, thereby realizing a visual display of the selected sound source.

[0105] In some embodiments, based on the target position information of the first sound source, the visual identification of the first sound source is displayed on the graphical display interface, including: when the target position information of the first sound source is the position information of the focus sound source, the visual identification of the first sound source is displayed at the focus sound source position of the graphical display interface; or, when the target position information of the first sound source is the position information of the focus sound source, the visual identification of the first sound source is adjusted from the first position to the focus sound source position on the graphical display interface.

[0106] That is, when the first sound source is not in the playing state, it is not displayed; when the first sound source enters the playing state and occupies the focus sound source position, the visual logo of the first sound source is displayed at the focus sound source position; when the first sound source enters the playing state but does not occupies the focus sound source position, the visual logo of the first sound source is displayed at the default position. Alternatively, when the first sound source is not in the playing state, the visual logo is displayed at the default position of the graphic display interface; when the first sound source enters the playing state and occupies the focus sound source position, the visual logo of the first sound source is adjusted from the default position to the focus sound source position in the graphic display interface; when the first sound source enters the playing state but does not occupies the focus sound source position, the visual logo of the first sound source is still displayed at the default position.

[0107] In some embodiments, the method further includes: in response to a play operation on the first sound source, displaying a visual identifier of the first sound source in a play state at a first position of the graphical display interface.

[0108] In some embodiments, when the target position information of the first sound source is the position information of the focus sound source, the first sound source and the audio data are rendered according to the position information of the focus sound source to obtain rendering data; or, when the target position information of the first sound source is the position information of the first sound source, the first sound source and the audio data are rendered according to the position information of the first sound source to obtain rendering data; and the rendering data is output using an audio output device.

[0109] Exemplarily, the target position information and audio data of the first sound source are rendered based on the spatial audio rendering algorithm. Exemplarily, the core of the spatial audio rendering algorithm may be a head-related transfer function (HRTF), which is used to represent the effect of audio at different positions on the human ear, and the HRTF function parameters are determined according to the position information of each sound source, so that the audio data output at a specific position can be obtained with the help of the HRTF function.

[0110] In some embodiments, the method further includes: in response to a start-up operation for the holographic audio function, calling up a mode selection interface; and receiving a mode selection operation for a target output mode in the mode selection interface. Exemplarily, the mode selection interface may select the smart mode by default.

[0111] With the above technical solution, in the intelligent mode, the best position is intelligently matched according to the priority and position change strategy of each sound source, and the playback effect of the priority audio is improved on the basis of achieving the playback effect of the sound source hearing separation. The default position and priority of different sound sources can also be adjusted, and personalized customization can also be achieved for the intelligent mode of multiple sound sources.

[0112] In order to better reflect the purpose of the present application, based on the above embodiments of the present application, the target output mode is further illustrated by taking the custom mode as an example. Fig.13 As shown, the method specifically includes:

[0113] Step 301: In response to a mode selection operation for a multi-source custom mode, displaying visual identifiers of multiple sound sources at different sound source spatial positions on a graphical display interface according to position information of multiple sound sources when the custom mode is used;

[0114] In some embodiments, the position information of the multiple sound sources may be the default position information of each sound source. The position information of each sound source may be obtained by performing an auditory test of the audio by arranging different positions of the sound sources to determine the position with the best audio effect.

[0115] In other embodiments, the location information of the multiple sound sources may be historical location information set last time.

[0116] Exemplarily, when the graphic display interface of the custom mode is initially entered, the default position information of each sound source is displayed. After the user performs a custom position adjustment, the graphic display interface of the custom mode is entered again, and the historical position information of each sound source that was last set is displayed.

[0117] In some embodiments, the method further includes: in response to an initialization operation for a multi-sound source customization mode, displaying visual identifiers of multiple sound sources at different sound source spatial positions on the graphical display interface according to default position information of the multiple sound sources.

[0118] Step 302: In response to a position adjustment operation for the visual identifier of the first sound source, the visual identifier of the first sound source is adjusted from a first position to a second position on the graphic display interface; wherein the second position is different from the first position;

[0119] It should be noted that the position adjustment operation is used to adjust the position information of the first sound source. The first sound source is any sound source displayed on the graphic display interface, and the position adjustment method for the first sound source can be understood as the position adjustment method for each sound source.

[0120] The first position can be understood as being determined based on the position information of the first sound source before adjustment, and the second position can be understood as being determined based on the position information of the first sound source after adjustment. The second position can be any position other than the first position in the multi-sound source space displayed in the graphical interface. The second position can include the center position, and can also include the position of other sound sources, that is, swapping the positions of the first sound source and the other sound sources.

[0121] The position adjustment operation may be a continuous operation, so that the position change process of the visual identifier of the first sound source is displayed on the graphic display interface. In some embodiments, the position adjustment operation may include a sliding operation on the visual identifier of the first sound source on the graphic display interface. Fig.14 The interface display diagram of the custom mode in the embodiment of the present application is shown as follows Figure 1 ,like Fig.14 As shown, the display area includes a mode selection interface and a graphic display interface. The mode selection interface includes the selection of smart mode and custom mode. The graphic display interface includes a hemispherical three-dimensional space, which includes visual identifiers of multiple sound sources and a central visual identifier. The user clicks on the visual identifier of a sound source to select it, and adjusts the sound source position information by sliding the visual identifier. The graphic display interface also includes a "start audition" button, and the user can control the output effect of multiple sound sources under the current spatial position relationship by operating this button.

[0122] In some embodiments, the method may further include: displaying a coordinate adjustment item of the first sound source on a parameter adjustment interface; in response to a coordinate adjustment operation on the coordinate adjustment item of the first sound source, the visual identifier of the first sound source is adjusted from a first position to a second position on a graphic display interface.

[0123] In some embodiments, the parameter adjustment interface and the graphic display interface are different display areas. That is, the display area is divided into the graphic display interface and the parameter adjustment interface, and the position adjustment operation can be one of the following: a sliding operation on a visual identifier in the graphic display interface, a coordinate adjustment on the position information of the parameter adjustment interface, or a sliding operation on a visual identifier in the graphic display interface and a coordinate adjustment item of the parameter adjustment interface.

[0124] Exemplarily, the coordinate adjustment item of the parameter adjustment interface includes three slide bars for adjusting the coordinates in three directions, and the coordinate adjustment operation includes the dragging operation of any slide bar. The three slide bars are arranged in sequence in the horizontal or vertical direction on the parameter adjustment interface, and the slide bar may also include a slider, and the coordinate adjustment can be achieved by dragging the slider. Exemplarily, the coordinate adjustment item may also include an input box for the coordinates in three directions, and the coordinates in the output box may be modified to achieve position adjustment.

[0125] Fig.15 The interface display diagram of the custom mode in the embodiment of the present application is shown as follows Figure 2 ,like Fig.15 As shown, the display area includes a mode selection interface, a graphic display interface and a parameter adjustment interface. The mode selection interface includes the selection of smart mode and custom mode. The graphic display interface includes a hemispherical three-dimensional space, which includes visual identifiers of multiple sound sources and a central visual identifier. The user clicks on the visual identifier of a sound source to select it, and adjusts the sound source position information by sliding the visual identifier. The graphic display interface also includes a "start audition" button, and the user can control the output effect of multiple sound sources under the current spatial position relationship by operating this button. The parameter adjustment interface includes a sound source selection item and three coordinate sliding bars. The user clicks on a sound source selection item to call up a list of selection items and selects an audio from it. At this time, adjusting the three position coordinates in the position information is to adjust the position of the selected audio.

[0126] In some embodiments, the parameter adjustment interface is displayed in a floating window on the graphic display interface. That is, the parameter adjustment interface can be displayed in a floating window on the graphic display interface without affecting the graphic display interface.

[0127] In some embodiments, the method further includes: in response to a selection operation for the visual identifier of the first sound source, calling up a parameter adjustment interface. In response to a mode selection operation for a multi-source target output mode, only the graphic display interface may be displayed in the entire display area first, and when position adjustment or coordinate adjustment is required, the parameter adjustment interface is called up. The parameter adjustment interface and the graphic display interface are in different display areas; or, the parameter adjustment interface is displayed in a floating window on the graphic display interface.

[0128] like Fig.14 As shown, the entire display area only displays the graphic display interface. After performing the selection operation on the music symbol, the parameter adjustment interface is called up and the display result can be as follows Fig.15 As shown, the parameter adjustment interface and the graphic display interface are different display areas. The display results can also be Fig.16 As shown, the parameter adjustment interface is displayed as a floating window on the graphic display interface. After the adjustment is completed, close the parameter adjustment interface and restore to Fig.14 .

[0129] In some embodiments, before responding to the position adjustment operation for the visual identifier of the first sound source, the method further includes: in response to the selection operation for the visual identifier of the first sound source, the visual identifier of the first sound source is adjusted from a first display state to a second display state on the graphic display interface; wherein the visual identifiers of the first display state and the second display state are different. Exemplarily, the selection operation can be a click operation on the visual identifier of the first sound source on the graphic display interface.

[0130] That is to say, depending on whether the user manually selects the sound source, the sound source is divided into a selected state (corresponding to the first display state of the visual identification) and an unselected state (corresponding to the second display state of the visual identification). Different display states are used to distinguish whether the sound source is selected for position adjustment, thereby realizing a visual display of the selected sound source. Fig.17 Schematic diagram of a graphical display when a sound source is selected in an embodiment of the present application, such as Fig.17 As shown, the music symbol is selected, and only the surface texture of the music symbol changes, while the texture of other sound source symbols remains the same as when they were not selected.

[0131] In some embodiments, before responding to a position adjustment operation on the first sound source, the method also includes: in response to a selection operation on the visual identification of the first sound source, the visual identification of the second sound source is adjusted from the first display state to the third display state in the graphical display interface; wherein the third display state includes an identification indicating that the second sound source is allowed to swap positions with the first sound source.

[0132] In some embodiments, displaying the coordinate adjustment item of the first sound source in the parameter adjustment interface includes: in response to a selection operation on the visual identifier of the first sound source, the coordinate adjustment item of the preset sound source in the parameter adjustment interface is adjusted to the coordinate adjustment item of the first sound source.

[0133] In some embodiments, the parameter adjustment interface also includes a sound source selection item, and a coordinate adjustment item of the first sound source is displayed on the parameter adjustment interface, including: in response to a selection operation on the first sound source in the sound source selection item, the coordinate adjustment item of the preset sound source in the parameter adjustment interface is adjusted to the coordinate adjustment item of the first sound source.

[0134] In some embodiments, the method further includes: in response to a selection operation on a first audio source in the audio source selection items, a visual identifier of the first audio source is adjusted from a first display state to a second display state in the graphical display interface; wherein the visual identifiers of the first display state and the second display state are different.

[0135] That is, according to whether the user manually selects the sound source, the user is identified with which sound sources can be swapped, and the user is prompted to select the position adjustment range of the sound source. The second sound source may be all or part of the sound sources except the first sound source.

[0136] In other embodiments, before responding to the position adjustment operation for the first sound source, the method further includes: in response to the selection operation of the visual identifier of the first sound source in the sound source selection item of the parameter adjustment interface, the visual identifier of the first sound source is adjusted from the first display state to the second display state on the graphic display interface. In other embodiments, before responding to the position adjustment operation for the first sound source, the method further includes: in response to the selection operation of the visual identifier of the first sound source in the sound source selection item of the parameter adjustment interface, the visual identifier of the second sound source is adjusted from the first display state to the third display state on the graphic display interface; wherein the third display state includes an identifier indicating that the second sound source is allowed to swap positions with the first sound source.

[0137] In some embodiments, the position adjustment operation on the visual identifier of the first sound source includes a drag operation on the visual identifier of the first sound source.

[0138] In some embodiments, when the graphic display interface is dragged for the visual identifier of the first sound source, the graphic display perspective can also be switched synchronously. Fig.18 As shown, the music symbol is selected. When the music symbol is dragged to change its position, the center line of the hemisphere is used as the rotation axis to switch the graphic display perspective.

[0139] In some embodiments, the method further includes: in response to a play operation on the first sound source, displaying a visual identifier of the first sound source in a play state at a first position of the graphical display interface.

[0140] Exemplarily, in response to a play operation on the first sound source, the visual identifier of the first sound source is adjusted from the unplayed state to the played state in the graphic display interface; in response to a stop play operation on the first sound source, the visual identifier of the first sound source is adjusted from the played state to the unplayed state in the graphic display interface. That is, when the sound source is in the unplayed state, the visual identifier of the unplayed state is displayed at the default position of the graphic display interface, and when the sound source is in the playing state, the visual identifier of the playing state is displayed at the default position of the graphic display interface, and the visual identifier of the unplayed state is different from the visual identifier of the playing state.

[0141] When the first sound source is in the playing state, the visual mark of the first sound source is superimposed with the mark representing the playing state, otherwise the mark representing the playing state is removed. The mark representing the playing state can be a dynamic mark or a static mark.

[0142] In some embodiments, the playing state includes a circular sound wave dynamic mark or a fan-shaped sound wave dynamic mark.

[0143] Fig.19 Schematic diagram of visual identification of each sound source in different display states in the embodiment of the present application, such as Fig.19 As shown in the figure, it is divided into selected state and unselected state according to whether the user manually selects the audio; it is divided into playing state and unplayed state according to whether the current audio is playing. The playing state is a graphic superimposed with the playing sound wave effect logo, and the playing sound wave effect logo is superimposed on both the selected and unselected states.

[0144] In some embodiments, the parameter adjustment interface further includes at least one of the following: a volume adjustment item, a sound effect adjustment item, and a visual effect adjustment item of a sound source visual identifier. The adjustment item may include any one or more parameter adjustment methods such as a slider, an input box, and a selection item.

[0145] It should be noted that only the adjustment items of the selected sound source can be displayed, or the adjustment items of multiple sound sources can be displayed. The volume adjustment items, sound effect adjustment items, visual effect adjustment items of the sound source visual logo, and coordinate adjustment items can be displayed on the same parameter adjustment interface, or on different parameter adjustment interfaces according to interface design requirements.

[0146] like Fig. 20 As shown in the figure, the volume adjustment item and the three coordinate adjustment items are displayed in the same parameter adjustment interface. Fig.21 As shown in FIG. 1 , when adjusting the volume of the incoming call ringtone, the parameter adjustment interface of the volume adjustment item of the incoming call ringtone is displayed separately in a floating form. Fig. 22 As shown, when adjusting the volume, volume adjustment items of multiple sound sources may also be displayed. Specifically, volume adjustment items of all sound sources may be displayed, or volume adjustment items of some sound sources may be displayed, for example, only volume adjustment items in the playing state may be displayed.

[0147] In some embodiments, the method further includes: in response to a play operation on the first sound source, obtaining position information of the first sound source; rendering audio data of the first sound source according to the position information of the first sound source to obtain rendering data; and outputting the rendering data using an audio output device.

[0148] It should be noted that the play operation can be clicking the "start audition" button in the graphic display interface to audition the play effects of some or all sound sources in the current spatial position relationship. The play operation can also be the actual play operation when using the custom mode.

[0149] In some embodiments, the method further includes: in response to a start-up operation for a holographic audio function, calling up a mode selection interface; and receiving a mode selection operation for a target output mode in the mode selection interface.

[0150] By adopting the above technical solution, different sound sources are allocated to different virtual space positions using the multi-source output mode, so that the multi-source data streams do not interfere with each other and are independently controllable, thereby achieving the playback effect of separated auditory sense of the sound sources. For different custom modes, the graphical display interface is conducive to quickly and accurately adjusting the spatial position of the sound source, thereby realizing personalized customization for multi-source output.

[0151] FIG. 23A to FIG. 23E Schematic diagram of the visualization process of the mode selection process in the embodiment of the present application. Fig.23A As shown in the figure, the user turns on the holographic audio function and enters the graphic display interface of the smart mode by default. In the graphic display interface of the smart mode, the position change strategy of the smart mode can be graphically displayed, and the Home position and / or priority of each sound source can be adjusted. After the adjustment is completed, exit the current interface, the smart mode takes effect, and multi-audio playback is performed according to the preset position change strategy to realize the holographic audio function.

[0152] like Fig. 23B As shown, the user selects the custom mode and switches to the graphic display interface of the custom mode. In the custom mode, the graphic display interface and the parameter adjustment interface are displayed. The incoming call ringtone is selected by default. The user can adjust the position by sliding the incoming call ringtone in the graphic display, or adjust the position coordinates in the left, right, front, back, and up and down directions in the parameter adjustment interface by dragging the slide bar. The distance between each audio and the Center position affects the sound volume, which is louder when closer and smaller when farther away.

[0153] like Fig.23C As shown, the switching can be achieved by clicking on the music symbol, or by selecting items in the parameter adjustment interface.

[0154] like Fig.23D As shown, after the music enters the selected state, the user can adjust the position by sliding the music in the displayed graphic, or adjust the position coordinates in the left and right, front and back, and up and down directions in the parameter adjustment interface by dragging the slide bar.

[0155] like Fig.23E As shown, after the adjustment is completed, exit the current interface, the custom mode takes effect, and multiple audios are played according to the user-defined position information to realize the holographic audio function.

[0156] Based on the above embodiments, the embodiments of the present application also provide an interface display method for a multi-source output mode. Fig.24 Schematic diagram of the implementation process of an interface display method in an embodiment of the present application, such as Fig.24 As shown, the method may include:

[0157] Step 401: Start;

[0158] Step 402: Graphics engine initialization;

[0159] Step 403: Load the sound source model;

[0160] Step 404: Obtain input information of the sound source model;

[0161] Step 405a: monitoring the sound source selection operation, step 405b: monitoring the space position adjustment operation, step 405c: monitoring the sound source playback operation;

[0162] Step 406a: Determine whether the selection has changed, if not, execute step 410, if yes, execute step 407a; Step 406b: Determine whether the position has changed, if not, execute step 410, if yes, execute step 407b; Step 406c: Determine whether to play, if not, execute step 410, if yes, execute step 407c;

[0163] Step 407a: update the visual identification, step 407b: update the position, step 407c: update the play identification;

[0164] Step 408: redraw the interface;

[0165] Step 409: Update the UI Buffer;

[0166] Step 410: Display interface cache;

[0167] Step 411: End.

[0168] It should be noted that after loading each sound source model, the graphics engine will render different display effects according to different user input operations. For example, the sound source model rendering can use the traditional physically based rendering (Physically Base Rendering, PBR) algorithm to present the authenticity of the model color, and different display effects can be achieved through customized materials.

[0169] In some embodiments, rendering may be performed only when the user input changes, the interface may be redrawn and the interface cache may be updated. Otherwise, the interface cache does not need to be updated and the last rendering result may be used directly for presentation. For example, the interface cache may be updated when any of the position, selection and playback changes, and the interface cache may not be updated when none of them changes.

[0170] The interface display implementation method of the embodiment of the present application mainly involves the interaction between the interface layer (referred to as "UI layer") and the holographic audio service layer (referred to as "service layer"). Fig.25 FIG. 1 is a schematic diagram of another implementation flow of an interface display method in an embodiment of the present application. Fig.25 As shown, the specific process is as follows:

[0171] S1: UI layer binds holographic audio service;

[0172] S2: The UI layer reads information from the service layer, including function switches (controlling whether the holographic audio function is effective), mode selection (smart mode and custom mode), scene position (spatial position of each audio source), playback status (whether each audio source is playing), etc.

[0173] S3. The UI layer displays the read information and passes the scene position and playback status to the graphics engine for rendering and display;

[0174] S4. The user adjusts the settings of each audio source and transmits the user input to the service layer to make the user input effective.

[0175] The beneficial effects of the technical solution provided by the embodiments of the present application specifically include:

[0176] 1. Use holographic audio technology for multi-source output, which can avoid the mutual interference problem when multiple audio sources are played at the same time;

[0177] 2. When multiple sound sources are played simultaneously, they appear in different spatial locations, creating an immersive three-dimensional auditory experience;

[0178] 3. Use graphic visualization to convert the adjustment of sound source playback effect into processing of visualization space, which is intuitive and clear.

[0179] To implement the method of the embodiment of the present application, based on the same inventive concept, the embodiment of the present application also provides a multi-source output control device, such as Fig.26 As shown, the multi-source output control device 260 includes:

[0180] The acquisition unit 2601 is used to obtain the position information of multiple sound sources when the target output mode is used in response to the mode selection operation for the multi-sound source target output mode;

[0181] The graphic display unit 2602 is further used to display visual identifiers of multiple sound sources at different sound source spatial positions on the graphic display interface according to the position information of the multiple sound sources.

[0182] In some embodiments, the plurality of sound sources include non-focus sound sources, and visual identifiers of the non-focus sound sources in the graphical display interface are distributed around a central position of the sound source space.

[0183] In some embodiments, when the multiple sound sources also include a focus sound source, a visual identifier of the focus sound source is displayed at the focus sound source position of the graphical display interface; wherein the focus sound source position is the center position or has a certain distance from the center position.

[0184] In some embodiments, the central visual mark is displayed at the center of the graphic display interface, and the central visual mark has directionality.

[0185] In some embodiments, the distance and / or azimuth of the sound source relative to the center position is determined based on the audio characteristics of the sound source; and the position information of each sound source is determined based on the distance and / or azimuth of each sound source relative to the center position.

[0186] In some embodiments, the sound source with the first audio characteristic is located in the first direction, the second direction or the third direction of the center position, and the sound source with the second audio characteristic is located at a position having a certain angle relative to the first direction, the second direction or the third direction of the center position.

[0187] In some embodiments, the sound source space of the graphic display interface is a hemispherical three-dimensional space, and the center position is the center position of the hemispherical three-dimensional space.

[0188] In some embodiments, the graphic display unit 2602 is further configured to display a visual identifier of the first sound source in a playing state at a first position of the graphic display interface in response to a play operation on the first sound source.

[0189] In some embodiments, the playing state includes a circular sound wave dynamic mark or a fan-shaped sound wave dynamic mark.

[0190] In some embodiments, when the target output mode is an intelligent mode for changing the position of the sound source based on a preset position change strategy, the graphic display unit is also used to respond to a playback operation for the first sound source, determine the target position information of the first sound source from the position information of the first sound source and the position information of the preset focus sound source according to the priority and position change strategy of the first sound source; and display the visual identification of the first sound source on the graphic display interface according to the target position information of the first sound source.

[0191] In some embodiments, the position change strategy includes: when the target position information of the first sound source is the position information of the focus sound source, the visual identification of the first sound source is displayed at the focus sound source position of the graphical display interface; or, when the target position information of the first sound source is the position information of the focus sound source, the visual identification of the first sound source is adjusted from the first position to the focus sound source position in the graphical display interface.

[0192] In some embodiments, the position change strategy specifically includes: when there is no focus sound source at the focus sound source position, or when there is a focus sound source at the focus sound source position and the priority of the first sound source is higher than the priority of the focus sound source, determining the target position information of the first sound source as the position information of the focus sound source; when there is a focus sound source at the focus sound source position and the priority of the first sound source is lower than the priority of the focus sound source, determining the target position information of the first sound source as the position information of the first sound source; when there is a focus sound source at the focus sound source position and the priority of the first sound source is equal to the priority of the focus sound source, if the creation time of the first sound source is later than the creation time of the focus sound source, determining the target position information of the first sound source as the position information of the focus sound source.

[0193] In some embodiments, the multi-sound source output control device 260 also includes an audio processing unit, which is used to render the first sound source and audio data according to the position information of the focus sound source to obtain rendering data when the target position information of the first sound source is the position information of the focus sound source; or, when the target position information of the first sound source is the position information of the first sound source, render the first sound source and audio data according to the position information of the first sound source to obtain rendering data; and output the rendering data using an audio output device.

[0194] In some embodiments, the sound source type of the sound source includes a transient sound source and a non-transient sound source, and the priority of the non-transient sound source is higher than the priority of the transient sound source.

[0195] In some embodiments, the transient sound source includes at least one of the following: notification, navigation, voice assistant, voice message; the non-transient sound source includes at least one of the following: call, incoming ringtone, alarm clock, game, voice call, video, audiobook, music.

[0196] In some embodiments, the acquisition unit 2601 is further configured to acquire the adjusted priority of the first sound source in response to a priority adjustment operation on the first sound source.

[0197] In some embodiments, the graphic display unit 2602 is further configured to, in response to a selection operation on the visual identifier of the first sound source, call out a priority adjustment interface, and display a priority sorting list of the plurality of sound sources on the priority adjustment interface.

[0198] In some embodiments, the graphic display unit 2602 is further configured to, in response to a position adjustment operation on the visual identifier of the first sound source, swap the positions of the visual identifier of the first sound source and the visual identifier of the second sound source on the graphic display interface. The position adjustment operation is specifically a position adjustment operation on the visual identifier of the first sound source and the visual identifier of the second sound source.

[0199] In some embodiments, before responding to a position adjustment operation for the visual identifier of the first sound source, the graphic display unit 2602 is also used to respond to a selection operation for the visual identifier of the first sound source, and the visual identifier of the first sound source is adjusted from a first display state to a second display state in the graphic display interface; wherein the visual identifiers of the first display state and the second display state are different.

[0200] In some embodiments, the graphic display unit 2602 is also used to, in response to a selection operation on the visual identifier of the first sound source, adjust the visual identifier of the second sound source from the first display state to the third display state in the graphic display interface; wherein the third display state includes an identifier indicating that the second sound source is allowed to swap positions with the first sound source.

[0201] In some embodiments, the position adjustment operation includes a drag operation on the visual indicator of the first sound source.

[0202] In some embodiments, when the target output mode is the sound source position customization mode, the graphic display unit 2602 is also used to display the coordinate adjustment item of the first sound source in the parameter adjustment interface; in response to the coordinate adjustment operation of the coordinate adjustment item of the first sound source, the visual identification of the first sound source is adjusted from the first position to the second position in the graphic display interface.

[0203] In some embodiments, the parameter adjustment interface and the graphic display interface are different display areas; or, the parameter adjustment interface is displayed in a floating window above the graphic display interface.

[0204] In some embodiments, the graphic display unit 2602 is further configured to call out a parameter adjustment interface in response to a selection operation on the visual identifier of the first sound source.

[0205] In some embodiments, the parameter adjustment interface also includes a sound source selection item, and the graphic display unit 2602 is also used to respond to the selection operation of the first sound source in the sound source selection item, and the coordinate adjustment item of the preset sound source in the parameter adjustment interface is adjusted to the coordinate adjustment item of the first sound source.

[0206] In some embodiments, the graphic display unit 2602 is also used to respond to the selection operation of the first sound source in the sound source selection items, and the visual identification of the first sound source is adjusted from the first display state to the second display state in the graphic display interface; wherein the visual identifications of the first display state and the second display state are different.

[0207] In some embodiments, the coordinate adjustment item of the parameter adjustment interface includes three slide bars for adjusting coordinates in three directions, and the coordinate adjustment operation includes a drag operation of any slide bar.

[0208] In some embodiments, the parameter adjustment interface further includes at least one of the following: a volume adjustment item, a sound effect adjustment item, and a visual effect adjustment item of a sound source visual identifier.

[0209] In some embodiments, the multi-sound source output control device 260 also includes an audio processing unit for obtaining position information of the first sound source in response to a playback operation on the first sound source; rendering the audio data of the first sound source according to the position information of the first sound source to obtain rendering data; and outputting the rendering data using an audio output device.

[0210] In some embodiments, the graphic display unit 2602 is further used to call up a mode selection interface in response to a start-up operation for a holographic audio function; and receive a mode selection operation for a target output mode in the mode selection interface.

[0211] In practical applications, the above-mentioned device can be any electronic device that realizes the holographic audio function, or it can be a chip applied to an electronic device. In the present application, the device can realize the functions of multiple units through software, hardware, or a combination of software and hardware, so that the device can execute the multi-source output control method provided in any of the above embodiments. And the technical effects of each technical scheme of the device can refer to the technical effects of the corresponding technical scheme in the multi-source output control method, and this application will not go into details one by one.

[0212] The electronic devices implementing the holographic audio function described in this application may include mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), vehicle-mounted devices, wearable devices, etc.

[0213] Based on the hardware implementation of each unit in the above-mentioned multi-source output control device, the embodiment of the present application also provides a multi-source output control device, such as Fig. 27 As shown, the multi-source output control device 270 includes: a processor 2701 and a memory 2702 configured to store a computer program that can be run on the processor;

[0214] The processor 2701 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0215] Of course, in practical applications, Fig. 27As shown, each component in the multi-source output control device 270 is coupled together through a bus system 2703. It can be understood that the bus system 2703 is used to realize the connection communication between these components. In addition to the data bus, the bus system 2703 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as bus system 2703 in the figure.

[0216] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present application do not specifically limit this.

[0217] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0218] Optionally, the multi-source output control device may be a chip, and the chip may further include an input interface, wherein the processor may control the input interface to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0219] Optionally, the chip may further include an output interface, wherein the processor may control the output interface to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0220] Optionally, the chip can be applied to the electronic device in the embodiments of the present application, and can implement the corresponding processes of the various methods in the embodiments of the present application. For the sake of brevity, they will not be described here.

[0221] In an exemplary embodiment, the embodiment of the present application further provides a computer-readable storage medium, such as a memory including a computer program, and the computer program can be executed by a processor of a multi-source output control device to complete the steps of the aforementioned method.

[0222] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0223] Optionally, the computer program product can be applied to the multi-sound source output control device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the multi-sound source output control device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0224] The embodiment of the present application also provides a computer program.

[0225] Optionally, the computer program can be applied to the multi-sound source output control device in the embodiments of the present application. When the computer program is run on a computer, the computer executes the corresponding processes implemented by the multi-sound source output control device in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0226] It should be understood that in the embodiments of the present application, related data such as user information is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0227] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments, but are not intended to limit the present application. The singular forms of "a", "said" and "the" used in the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items. The expressions "having", "may have", "include" and "include", or "may include" and "may include" in this application can be used to indicate the presence of corresponding features (e.g., elements such as numerical values, functions, operations or components), but the presence of additional features is not excluded.

[0228] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other and are not necessarily used to describe a specific order or sequence. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0229] The technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0230] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and equipment can be implemented in other ways. The embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0231] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0232] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0233] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A multi-source output control method, characterized in that: The method comprises: In response to a mode selection operation for a multi-source target output mode, obtaining position information of a plurality of sound sources when the target output mode is used; According to the position information of the multiple sound sources, visual identifiers of the multiple sound sources are displayed at different sound source spatial positions on the graphic display interface.

2. The method according to claim 1, characterized in that The multiple sound sources include non-focus sound sources, and the visual identifiers of the non-focus sound sources on the graphic display interface are distributed around the center position of the sound source space.

3. The method according to claim 2, characterized in that When the multiple sound sources also include a focus sound source, a visual identifier of the focus sound source is displayed at the focus sound source position on the graphic display interface; wherein the focus sound source position is the center position or has a certain distance from the center position.

4. The method according to claim 2 or 3, characterized in that: A central visual marker is displayed at the central position of the graphic display interface, and the central visual marker has directionality.

5. The method according to claim 2, characterized in that: The method further comprises: Determining the distance and / or azimuth of the sound source relative to the center position according to the audio characteristics of the sound source; The position information of each sound source is determined according to the distance and / or azimuth of each sound source relative to the central position.

6. The method according to claim 5, characterized in that The sound source having the first audio characteristic is located in the first direction, the second direction or the third direction of the center position, The sound source having the second audio characteristic is located at a position having a certain angle with respect to the first direction, the second direction or the third direction of the center position.

7. The method according to claim 2, characterized in that The sound source space of the graphic display interface is a hemispherical three-dimensional space, and the center position is the center position of the hemispherical three-dimensional space.

8. The method according to claim 1, characterized in that The method further comprises: In response to a play operation on a first sound source, a visual identifier of the first sound source in a play state is displayed at a first position on the graphic display interface.

9. The method according to claim 8, characterized in that The playing state includes a circular sound wave dynamic mark or a fan-shaped sound wave dynamic mark.

10. The method according to claim 1, characterized in that When the target output mode is an intelligent mode for changing the sound source position based on a preset position change strategy, the method further includes: In response to a play operation on a first sound source, determining target position information of the first sound source from position information of the first sound source and position information of a preset focus sound source according to the priority of the first sound source and the position change strategy; A visual identifier of the first sound source is displayed on the graphic display interface according to the target position information of the first sound source.

11. The method according to claim 10, characterized in that The step of displaying a visual identifier of the first sound source on the graphic display interface according to the target position information of the first sound source includes: In a case where the target position information of the first sound source is the position information of the focus sound source, displaying a visual identifier of the first sound source at the focus sound source position of the graphic display interface; Alternatively, when the target position information of the first sound source is the position information of the focus sound source, the visual identifier of the first sound source is adjusted from the first position to the focus sound source position on the graphic display interface.

12. The method according to claim 10, characterized in that The position change strategy includes: In a case where there is no focus sound source at the focus sound source position, or in a case where there is a focus sound source at the focus sound source position and the priority of the first sound source is higher than the priority of the focus sound source, determining the target position information of the first sound source as the position information of the focus sound source; When there is a focus sound source at the focus sound source position and the priority of the first sound source is lower than the priority of the focus sound source, determining the target position information of the first sound source to be the position information of the first sound source; When there is a focus sound source at the focus sound source position and the priority of the first sound source is equal to the priority of the focus sound source, if the creation time of the first sound source is later than the creation time of the focus sound source, the target position information of the first sound source is determined to be the position information of the focus sound source.

13. The method according to claim 10, characterized in that The method further comprises: When the target position information of the first sound source is the position information of the focus sound source, rendering the first sound source and the audio data according to the position information of the focus sound source to obtain rendering data; Alternatively, when the target position information of the first sound source is the position information of the first sound source, the first sound source and the audio data are rendered according to the position information of the first sound source to obtain rendering data; The rendered data is outputted using an audio output device.

14. The method according to claim 10, characterized in that The sound source type of the sound source includes a transient sound source and a non-transient sound source, and the priority of the non-transient sound source is higher than the priority of the transient sound source.

15. The method according to claim 14, characterized in that The transient sound source includes at least one of the following: notification, navigation, voice assistant, and voice message; The non-transient sound source includes at least one of the following: a call, an incoming call ringtone, an alarm clock, a game, a voice call, a video, an audio book, and music.

16. The method according to claim 10, characterized in that The method further comprises: In response to the priority adjustment operation for the first sound source, the adjusted priority of the first sound source is obtained.

17. The method according to claim 16, characterized in that The method further comprises: In response to a selection operation on the visual identifier of the first sound source, a priority adjustment interface is called up, and a priority ranking list of the plurality of sound sources is displayed on the priority adjustment interface.

18. The method according to claim 10, characterized in that The method further comprises: In response to the position adjustment operation for the visual identifier of the first sound source, the positions of the visual identifier of the first sound source and the visual identifier of the second sound source on the graphic display interface are swapped.

19. The method according to claim 1, characterized in that When the target output mode is a sound source position customization mode, the method further includes: In response to a position adjustment operation for the visual identifier of the first sound source, the visual identifier of the first sound source is adjusted from the first position to the second position on the graphic display interface; wherein the second position is different from the first position.

20. The method according to claim 19, characterized in that Before the position adjustment operation in response to the visual identification of the first sound source, the method further includes: In response to a selection operation on the visual identifier of the first sound source, the visual identifier of the first sound source is adjusted from a first display state to a second display state on the graphic display interface; wherein the visual identifiers of the first display state and the second display state are different.

21. The method according to claim 20, characterized in that The method further comprises: In response to a selection operation on the visual identifier of the first sound source, the visual identifier of the second sound source is adjusted from the first display state to a third display state on the graphical display interface; wherein the third display state includes an identifier indicating that the second sound source is allowed to be swapped with the first sound source.

22. The method according to claim 19, characterized in that The position adjustment operation includes a drag operation on the visual indicator of the first sound source.

23. The method according to claim 1 or 19, characterized in that When the target output mode is a sound source position customization mode, the method further includes: Display the coordinate adjustment items of the first sound source on the parameter adjustment interface; In response to a coordinate adjustment operation on the coordinate adjustment item of the first sound source, the visual identifier of the first sound source is adjusted from the first position to the second position on the graphic display interface.

24. The method according to claim 23, characterized in that The parameter adjustment interface and the graphic display interface are different display areas; Alternatively, the parameter adjustment interface is displayed in a floating window on top of the graphic display interface.

25. The method according to claim 23, characterized in that The method further comprises: In response to a selection operation on the visual identifier of the first sound source, the parameter adjustment interface is called out.

26. The method according to claim 23, characterized in that The displaying of the coordinate adjustment items of the first sound source on the parameter adjustment interface includes: In response to a selection operation on the visual identifier of the first sound source, the coordinate adjustment item of the preset sound source in the parameter adjustment interface is adjusted to the coordinate adjustment item of the first sound source.

27. The method according to claim 23, characterized in that The parameter adjustment interface further includes a sound source selection item, and the coordinate adjustment item of the first sound source displayed on the parameter adjustment interface includes: In response to a selection operation on the first sound source in the sound source selection items, the coordinate adjustment items of the preset sound source in the parameter adjustment interface are adjusted to the coordinate adjustment items of the first sound source.

28. The method according to claim 27, characterized in that The method further comprises: In response to a selection operation on the first sound source in the sound source selection items, a visual identifier of the first sound source is adjusted from a first display state to a second display state in the graphical display interface; wherein the visual identifiers of the first display state and the second display state are different.

29. The method according to claim 23, characterized in that The coordinate adjustment item of the parameter adjustment interface includes three slide bars for adjusting coordinates in three directions, and the coordinate adjustment operation includes a dragging operation of any slide bar.

30. The method according to claim 23, characterized in that The parameter adjustment interface also includes at least one of the following: a volume adjustment item, a sound effect adjustment item, and a visual effect adjustment item of a sound source visual identifier.

31. The method according to claim 1, characterized in that When the target output mode is a sound source position customization mode, the method further includes: In response to a play operation on a first sound source, obtaining position information of the first sound source; Rendering the audio data of the first sound source according to the position information of the first sound source to obtain rendering data; The rendered data is outputted using an audio output device.

32. The method according to claim 1, characterized in that The method further comprises: In response to a start-up operation for the holographic audio function, calling out a mode selection interface; A mode selection operation for the target output mode in the mode selection interface is received.

33. A multi-source output control device, characterized in that: The device comprises: an acquisition unit, configured to, in response to a mode selection operation for a multi-sound source target output mode, acquire position information of a plurality of sound sources when the target output mode is used; The graphic display unit is used to display the visual identifications of the multiple sound sources at different sound source spatial positions on the graphic display interface according to the position information of the multiple sound sources.

34. A multi-source output control device, characterized in that: The device comprises: a processor and a memory configured to store a computer program capable of being executed on the processor, Wherein, the processor is configured to execute the steps of the method described in any one of claims 1 to 32 when running the computer program.

35. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 32 are implemented.

Images