Split screen processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a split screen processing method and device, a storage medium and electronic equipment, and relates to the technical field of electronic equipment, and the method comprises the steps: for each split screen window in a plurality of split screen windows, obtaining a split screen audio signal corresponding to the split screen window, the split screen audio signal being an audio signal corresponding to an application in the split screen window; the split-screen audio signals corresponding to the split-screen windows are transmitted to physical audio output interfaces corresponding to the split-screen windows, and the physical audio output interfaces corresponding to the multiple split-screen windows are different or the same. The split-screen use experience of the user can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic devices, in particular to a split-screen processing method and device, a storage medium and an electronic device. BACKGROUND

[0002] The existing devices with split-screen display capability, such as televisions or mobile phones, can only output the audio signals of all split-screen windows through one physical audio output interface when displaying multiple pictures in split screen, so that the user cannot simultaneously and independently obtain the audio content corresponding to the application pictures in each split-screen window, which limits the user's experience and functional expansion of multitasking, collaborative work and multi-scene application, and results in poor user experience of split screen. SUMMARY

[0003] The embodiments of the present application provide a split-screen processing scheme, which can output the split-screen audio signals corresponding to different split-screen windows to the corresponding physical audio output interfaces when displaying multiple pictures in split screen, thereby improving the user's experience of split screen.

[0004] The embodiments of the present application provide the following technical solutions: According to an embodiment of the present application, a split-screen processing method comprises: obtaining a split-screen audio signal corresponding to each split-screen window in a plurality of split-screen windows, the split-screen audio signal being an audio signal corresponding to an application in the split-screen window; and delivering the split-screen audio signal corresponding to the split-screen window to a physical audio output interface corresponding to the split-screen window, wherein the physical audio output interfaces corresponding to the plurality of split-screen windows are different or the same.

[0005] In some embodiments of the present application, before the split-screen audio signal corresponding to the split-screen window is delivered to the physical audio output interface corresponding to the split-screen window, the method further comprises: determining the physical audio output interface corresponding to the split-screen window according to a user-defined interface configuration, wherein the user-defined interface configuration includes one or more physical audio output interfaces corresponding to the split-screen window or an application in the split-screen window specified by the user.

[0006] In some embodiments of the present application, before the physical audio output interface corresponding to the split-screen window is determined according to the user interface configuration, the method further comprises: displaying an interface self-definition interface when an interface setting condition is met; and saving the user-defined interface configuration set by the user through the interface self-definition interface.

[0007] In some embodiments of the present application, before the step of delivering the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method further comprises: determining the physical audio output interface corresponding to the split-screen window according to interface automatic allocation information, wherein the interface automatic allocation information is obtained according to split-screen data decision analysis, and the split-screen data comprises one or more of audio data, user historical interface allocation data, application data and device interface configuration data.

[0008] In some embodiments of the present application, before the step of determining the physical audio output interface corresponding to the split-screen window according to interface automatic allocation information, the method further comprises: determining a window priority corresponding to each split-screen window according to the split-screen data; and performing interface allocation according to the window priority corresponding to each split-screen window to obtain the interface automatic allocation information, wherein the interface automatic allocation information comprises one or more physical audio output interfaces corresponding to each split-screen window or an application in the split-screen window.

[0009] In some embodiments of the present application, before the step of determining the physical audio output interface corresponding to the split-screen window according to interface automatic allocation information, the method further comprises: performing analysis and processing on the split-screen data by using a preset interface allocation model to obtain the interface automatic allocation information output by the preset interface allocation model, wherein the interface automatic allocation information comprises one or more physical audio output interfaces corresponding to each split-screen window or an application in the split-screen window.

[0010] In some embodiments of the present application, the step of delivering the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window comprises: when it is detected that the physical audio output interface corresponding to the split-screen window meets a switching condition, performing an interface switching operation to determine a new physical audio output interface corresponding to the split-screen window; and delivering the split-screen audio signal corresponding to the split-screen window to the new physical audio output interface.

[0011] In some embodiments of the present application, the step of delivering the split-screen audio signal corresponding to the split-screen window to the new physical audio output interface comprises: transferring the split-screen audio signal corresponding to the split-screen window from a main audio buffer to a backup audio buffer; and performing signal transmission from the backup audio buffer to the new physical audio output interface.

[0012] In some embodiments of the present application, before the step of delivering the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method further comprises: determining an interface allocation mode selected by a user, the interface allocation mode comprising a first mode or a second mode; wherein in the first mode, the physical audio output interface corresponding to the split-screen window is determined based on a user-defined interface configuration; and in the second mode, the physical audio output interface corresponding to the split-screen window is determined based on interface automatic allocation information.

[0013] In some embodiments of the present application, the step of obtaining, for each of the plurality of split-screen windows, a split-screen audio signal corresponding to the split-screen window comprises: intercepting, from audio data, an audio signal corresponding to an application in the split-screen window before the audio data is written into a unified buffer, to obtain the split-screen audio signal corresponding to the split-screen window.

[0014] In some embodiments of the present application, before the step of delivering the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method further comprises: respectively configuring, for each of the plurality of split-screen windows, a main audio buffer corresponding to the split-screen window, the main audio buffer corresponding to the split-screen window being used to buffer the split-screen audio signal corresponding to the split-screen window.

[0015] In some embodiments of the present application, the split-screen audio signal corresponding to each of the plurality of split-screen windows is independently decoded by a concurrent decoding engine.

[0016] According to an embodiment of the present application, a split-screen processing apparatus comprises: an audio capturing module configured to: obtain, for each of a plurality of split-screen windows, a split-screen audio signal corresponding to the split-screen window, the split-screen audio signal being an audio signal corresponding to an application in the split-screen window; and an audio routing module configured to: deliver the split-screen audio signal corresponding to the split-screen window to a physical audio output interface corresponding to the split-screen window, wherein the physical audio output interfaces corresponding to the plurality of split-screen windows are different or the same.

[0017] According to another embodiment of the present application, a storage medium has a computer program stored thereon, and when the computer program is executed by a processor of an electronic device, the electronic device performs the method described in the embodiments of the present application.

[0018] According to another embodiment of the present application, an electronic device can comprise: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the method described in the embodiments of the present application.

[0019] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0020] In this embodiment of the application, for each of the multiple split-screen windows, the split-screen audio signal corresponding to the split-screen window is obtained, and the split-screen audio signal refers to the audio signal corresponding to the application in the split-screen window; the split-screen audio signal corresponding to the split-screen window is sent to the physical audio output interface corresponding to the split-screen window, wherein the physical audio output interfaces corresponding to the multiple split-screen windows are different or the same.

[0021] In this embodiment of the application, when displaying multiple screens in split-screen mode, the physical audio output interfaces corresponding to the multiple split-screen windows can be different or the same. The split-screen audio signals corresponding to each split-screen window can be obtained separately and sent to the physical audio output interface corresponding to each split-screen window. In this way, users can simultaneously and independently obtain the audio content corresponding to the application screen in each split-screen window. This can further flexibly meet users' experience and functional expansion for multi-tasking, collaborative work, and multi-scenario applications, and improve the user's split-screen experience. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart of a split-screen processing method according to an embodiment of this application is shown.

[0024] Figure 2 A schematic diagram of a split-screen system architecture according to an embodiment of this application is shown.

[0025] Figure 3 A flowchart illustrating the interface determination process according to one embodiment of this application is shown.

[0026] Figure 4 An interface definition flowchart according to an embodiment of this application is shown.

[0027] Figure 5 An interface switching flowchart according to an embodiment of this application is shown.

[0028] Figure 6 A block diagram of a split-screen processing apparatus according to an embodiment of this application is shown.

[0029] Figure 7 A block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0030] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure. Furthermore, the embodiments provided below are some embodiments for implementing the present disclosure, and not all embodiments for implementing the present disclosure. Unless otherwise specified, the technical solutions described in the embodiments of the present disclosure can be implemented in any combination. It should be noted that, in the embodiments of this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element. For example, the split-screen processing method provided in this embodiment includes a series of steps, but the split-screen processing method provided in this embodiment is not limited to the steps described. Similarly, the split-screen processing device provided in this embodiment includes a series of units, but the device provided in this embodiment is not limited to the units explicitly described, but may also include units that need to be set up for obtaining relevant information or processing based on information. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. It is understood that in the specific implementation of this application, relevant data is involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0031] Existing devices with split-screen display capabilities, such as TVs or mobile phones, typically only output the audio signals of all split-screen windows to a single physical audio output interface when displaying multiple screens in split-screen mode. This prevents users from simultaneously and independently obtaining the audio content corresponding to the application screen in each split-screen window, limiting the user experience and functional expansion for multitasking, collaborative work, and multi-scenario applications, resulting in a poor user split-screen experience.

[0032] To address these issues, this application provides a split-screen processing solution that allows the audio signals corresponding to different split-screen windows to be output to their respective physical audio output interfaces during multi-screen split-screen display, thereby improving the user's split-screen experience.

[0033] The following is a detailed description of the relevant embodiments of the split-screen processing solution provided in this application.

[0034] Figure 1 A flowchart illustrating a split-screen processing method according to an embodiment of this application is shown. The entity executing this split-screen processing method can be an electronic device with allocation processing capabilities, such as a television, computer, mobile phone, tablet, in-vehicle device, smartwatch, and home appliance, etc.

[0035] like Figure 1 As shown, the split-screen processing method may include steps S110 to S120.

[0036] Step S110: For each split-screen window in the multiple split-screen windows, obtain the split-screen audio signal corresponding to the split-screen window. The split-screen audio signal refers to the audio signal corresponding to the application in the split-screen window. Step S120: The split-screen audio signal corresponding to the split-screen window is sent to the physical audio output interface corresponding to the split-screen window. The physical audio output interfaces corresponding to multiple split-screen windows may be different or the same.

[0037] In multi-screen split-screen display, the screen can include multiple split-screen windows, each of which can display the screen of at least one application. Users can drag and drop App A, App B, and App C into the window grids of split-screen windows A, B, and C respectively to achieve multi-screen split-screen display of three windows. A split-screen mapping table (such as a hash table) can be used to record the application corresponding to each split-screen window. For example, the split-screen mapping table can record the mapping relationship: {Split-screen window A: Application A process ID, Split-screen window B: Application B process ID ...}.

[0038] The operating system can integrate a multi-application window manager (such as the Android-based Multi-Window / Multi-Instance management module). When a user triggers multi-screen split-screen display, the multi-application window manager dynamically instantiates multiple application containers. Each split-screen window can correspond to at least one application container, and the application containers corresponding to each split-screen window can be bound to the application processes of the respective applications, thereby realizing multi-screen split-screen display.

[0039] Furthermore, in this application, the physical audio output interfaces corresponding to multiple split-screen windows can be different or the same. For example, split-screen window A, split-screen window B, and split-screen window C can all correspond to a single physical audio output interface A, meaning that the physical audio output interfaces corresponding to multiple split-screen windows are the same. Further, split-screen window A can correspond to a single physical audio output interface A, split-screen window B and split-screen window C can correspond to a single physical audio output interface B, and split-screen window D can correspond to a single physical audio output interface C. In other words, the physical audio output interfaces corresponding to multiple split-screen windows can be different, rather than being limited to the same physical audio output interface.

[0040] Physical audio output interfaces refer to the physical audio output interfaces themselves, such as speaker jacks, headphone jacks, digital optical output, Bluetooth, and so on. Different physical audio output interfaces correspond to different audio output devices, such as speakers, headphones, loudspeakers, and Bluetooth devices. For example, ... Figure 2 As shown, the screen 200 may include a split-screen window 210 and a split-screen window 220. The split-screen window 210 and the split-screen window 220 correspond to a physical audio output interface 230 and a physical audio output interface 240, respectively. The physical audio output interface 230 corresponds to an audio output device 250 (such as a speaker), and the physical audio output interface 240 corresponds to an audio output device 260 (such as headphones).

[0041] In conventional methods, the audio signals of all split-screen windows are usually output through a single physical audio output interface. For example, in conventional methods, the audio signals corresponding to split-screen window A, split-screen window B, and split-screen window C are all sent to physical audio output interface A (such as a speaker interface) through a single interface. Consequently, the mixed audio of all split-screen windows can only be played through the audio output device A corresponding to physical audio output interface A (such as listening to the mixed audio through a speaker).

[0042] In this application, the audio signals corresponding to each split-screen window are captured independently. The split-screen audio signal refers to the audio signal corresponding to the application within the split-screen window. Since the physical audio output interfaces corresponding to multiple split-screen windows can be different, the split-screen audio signals corresponding to each split-screen window can be sent to their respective physical audio output interfaces. Specifically, the operating system's underlying audio module (such as AudioFlinger, Audio HAL, or PulseAudio) can provide an independent audio output channel for each application within a split-screen window. This audio output channel is used to send the split-screen audio signals to the physical audio output interface. Furthermore, an audio capture interface can be set. When an application within a split-screen window calls this audio capture interface, the call information can be forcibly appended with the split-screen window's identifier, allowing for accurate delivery of the split-screen audio stream to the corresponding physical audio output interface based on this identifier.

[0043] For example, based on the solution of this application, the split-screen audio signal corresponding to split-screen window A can be sent to physical audio output interface A (such as a speaker interface), and then the audio output device A (such as a speaker) corresponding to physical audio output interface A can play the audio corresponding to split-screen window A alone; while the split-screen audio signals corresponding to split-screen windows B and C can be sent to physical audio output interface B (such as a headphone jack), and then the audio output device B (such as headphones) corresponding to physical audio output interface B can play the mixed audio of split-screen windows B and C; the split-screen audio signal corresponding to split-screen window D can be sent to physical audio output interface C (such as a digital optical fiber), and then the audio output device C (such as a speaker) corresponding to physical audio output interface C can play the audio corresponding to split-screen window D alone.

[0044] In summary, using the method described in this embodiment, when displaying multiple split-screen windows, the physical audio output interfaces corresponding to the multiple split-screen windows can be different or the same. The split-screen audio signals corresponding to each split-screen window can be obtained separately and sent to the physical audio output interface corresponding to each split-screen window. Thus, users can simultaneously and independently obtain the audio content corresponding to the application screen in each split-screen window. This can further flexibly meet users' experience and functional expansion for multi-tasking, collaborative work, and multi-scenario applications, and improve the user's split-screen experience.

[0045] The following description Figure 1 Further optional specific embodiments for each step performed during the split-screen processing in the example.

[0046] In one embodiment, obtaining the split-screen audio signal corresponding to each of the multiple split-screen windows may specifically include: intercepting the audio signal corresponding to the application in the split-screen window from the audio data before writing the audio data into a unified buffer, thereby obtaining the split-screen audio signal corresponding to the split-screen window.

[0047] In operating systems, the audio signals (i.e., audio data) of all split-screen windows are typically written to a unified buffer through the "write audio buffer" operation, and then sent from the unified buffer to a unified physical audio output interface.

[0048] In this application, by means of hooking or intercepting, the audio write-to-buffer operation can be intercepted before the audio signals (i.e., audio data) of all split-screen windows are written to a unified buffer. The audio signals corresponding to the application (APP) in each split-screen window can be extracted and separated from the audio data. The audio signals corresponding to the application in the split-screen window are the split-screen audio signals corresponding to the split-screen window.

[0049] Optionally, in some other embodiments, obtaining the split-screen audio signal corresponding to each of the multiple split-screen windows may include: after the audio data is written to a unified buffer, obtaining the audio data from the unified buffer and separating the audio signal corresponding to the application in each split-screen window to obtain the split-screen audio signal corresponding to each split-screen window.

[0050] See Figure 3 In one embodiment, before sending the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method may further include: step S310, determining the physical audio output interface corresponding to the split-screen window according to the user-defined interface configuration, wherein the user-defined interface configuration includes one or more physical audio output interfaces corresponding to the split-screen window or application specified by the user.

[0051] User-defined interface configuration refers to a user-defined interface allocation strategy. This configuration can include one or more physical audio output interfaces specified by the user for each split-screen window or application. The physical audio output interface corresponding to each split-screen window is determined based on this configuration. Specifically, the physical audio output interface corresponding to a split-screen window can be either "one or more physical audio output interfaces corresponding to the split-screen window itself" or "one or more physical audio output interfaces corresponding to the application within the split-screen window".

[0052] When a split-screen window corresponds to multiple different physical audio output interfaces, in some methods, the split-screen audio signal corresponding to the split-screen window can be simultaneously sent to all of these different physical audio output interfaces; or, in some methods, the split-screen audio signal corresponding to the split-screen window can be sent to only one physical audio output interface according to a preset priority or user selection; or, in some methods, the split-screen audio signals corresponding to multiple different split-screen windows can be simultaneously sent to the same physical audio output interface for mixing and output.

[0053] The physical audio output interface corresponding to the split-screen window is determined based on the user-defined interface configuration. Then, the split-screen audio signal corresponding to each split-screen window can be sent to the corresponding physical audio output interface according to the user's actual needs.

[0054] For further details, please refer to [link / reference]. Figure 4 In one embodiment, before determining the physical audio output interface corresponding to the split-screen window based on the user interface configuration, the method may further include: step S410, displaying the interface customization interface when the interface setting conditions are met; step S420, saving the user-defined interface configuration set by the user through the interface customization interface.

[0055] When the interface settings conditions are met, the interface customization interface is displayed. Users can assign interfaces as needed in this interface, allocating one or more physical audio output interfaces to each split-screen window or to each split-screen window. Users can also set preset priorities and other information for multiple different physical audio output interfaces.

[0056] Furthermore, the user-defined interface configuration can include one or more physical audio output interfaces corresponding to the split-screen window or application specified by the user. By saving the user-defined interface configuration set by the user through the interface customization interface, the physical audio output interface corresponding to the split-screen window can be determined based on the user-defined interface configuration.

[0057] In some methods, the interface setting conditions can be determined when the user performs the operation of opening the custom interface of the interface; in some methods, the interface setting conditions can be determined when the physical audio output interface corresponding to the split-screen window meets the switching conditions; in some methods, the interface setting conditions can be determined when two split-screen windows are assigned the same physical audio output interface; and in some methods, the interface setting conditions can be determined when the user reports an audio abnormality message (such as a message reporting that audio cannot be heard or that the audio mixing playback effect is poor).

[0058] Furthermore, when displaying the interface customization interface, some methods can directly display the interface customization interface when the interface setting conditions are met; other methods can ask the user whether to make settings or display the interface customization interface through voice broadcast, pop-up window or in-interface display when the interface setting conditions are met, and then display the interface customization interface only if the user agrees.

[0059] Furthermore, the interface customization interface can include controls for assigning one or more physical audio output interfaces to each split-screen window or application. Additionally, the interface customization interface can display interface-related information for the physical audio output interfaces on the device, including interface occupancy status and availability.

[0060] See Figure 3 In one embodiment, before transmitting the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method may further include: step S320, determining the physical audio output interface corresponding to the split-screen window based on the interface automatic allocation information, wherein the interface automatic allocation information is obtained based on split-screen data decision analysis, and the split-screen data includes one or more of audio data, user historical interface allocation data, application data, and device interface configuration data.

[0061] In this embodiment, the interface automatic allocation information is obtained by the device based on intelligent decision analysis of split-screen data. The user-defined interface configuration may include one or more physical audio output interfaces corresponding to each split-screen window or application in the split-screen window.

[0062] The split-screen data includes one or more of the following: audio data, user history interface allocation data, application data, and device interface configuration data. Audio data may include text content converted from the split-screen audio signal corresponding to each split-screen window, or it may include audio characteristic parameters such as the sampling rate, number of channels, average volume, and maximum peak value of the split-screen audio signal corresponding to each split-screen window. User history interface allocation data may be a record of the physical audio output interfaces allocated by the user to various applications on the device in the past. Application data may be the application data of the applications in each split-screen window, including but not limited to application name, application usage data, etc. Device interface configuration data refers to the relevant data of the physical audio output interfaces configured in the device, which may include the number of physical audio output interfaces configured in the device, interface identifier, whether the interface is normal, and whether the interface is available.

[0063] In this embodiment, the automatic interface allocation information is an interface allocation strategy for intelligent decision-making. The automatic interface allocation information may include one or more physical audio output interfaces corresponding to each split-screen window or application in each split-screen window. Based on this automatic interface allocation information, the physical audio output interface corresponding to the split-screen window can be determined. That is, the physical audio output interface corresponding to the split-screen window can specifically be "one or more physical audio output interfaces corresponding to the split-screen window itself" or "one or more physical audio output interfaces corresponding to the application in the split-screen window".

[0064] When a split-screen window corresponds to multiple different physical audio output interfaces, in some methods, the split-screen audio signal corresponding to the split-screen window can be simultaneously sent to all of these different physical audio output interfaces; or, in some methods, the split-screen audio signal corresponding to the split-screen window can be sent to only one physical audio output interface according to a preset priority or user selection; or, in some methods, the split-screen audio signals corresponding to multiple different split-screen windows can be simultaneously sent to the same physical audio output interface for mixing and output.

[0065] Based on the interface automatic allocation information, the physical audio output interface corresponding to the split-screen window is determined. Then, the split-screen audio signal corresponding to each split-screen window can be intelligently sent to the corresponding physical audio output interface.

[0066] Furthermore, in one embodiment, before determining the physical audio output interface corresponding to the split-screen window based on the interface automatic allocation information, the method further includes: determining the window priority corresponding to each split-screen window based on the split-screen data; allocating interfaces based on the window priority corresponding to each split-screen window to obtain interface automatic allocation information, wherein the interface automatic allocation information includes one or more physical audio output interfaces corresponding to each split-screen window or application in the split-screen window.

[0067] The window priority of each split-screen window is determined based on the split-screen data, and the interface is allocated according to the window priority of each split-screen window. One or more physical audio output interfaces can be allocated to the split-screen window with higher window priority, which further ensures the user's split-screen experience.

[0068] Specifically, by using preset decision data or rules, the window priority of each split-screen window can be determined based on the split-screen data. For example, in some methods, the higher the audio experience importance of an application in a split-screen window, the higher the window priority of that application. For instance, split-screen windows of applications with high audio experience importance, such as emergency voice applications or speaker applications, have higher window priorities. In other methods, split-screen windows of applications used more frequently by users have higher window priorities.

[0069] Furthermore, in one embodiment, before determining the physical audio output interface corresponding to the split-screen window based on the interface automatic allocation information, the method may further include: analyzing and processing the split-screen data using a preset interface allocation model to obtain the interface automatic allocation information output by the preset interface allocation model, wherein the interface automatic allocation information includes one or more physical audio output interfaces corresponding to each split-screen window or application in the split-screen window.

[0070] In this embodiment, the preset interface allocation model is a pre-trained machine learning model, such as a Large Language Model (LLM) or other neural network models trained before being used as the preset interface allocation model. Using the preset interface allocation model allows for global analysis and processing of split-screen data, resulting in more reasonable automatic preset interface allocation information. Based on this embodiment, the physical audio output interface corresponding to the split-screen window is determined, which can further improve the audio experience in split-screen scenarios under automatic interface allocation mode.

[0071] In particular, when split-screen data includes multiple types such as audio data, user history interface allocation data, application data, and device interface configuration data, a preset interface allocation model is used to perform global analysis and processing of the split-screen data and output preset automatic interface allocation information, which can further improve the audio experience in split-screen scenarios.

[0072] In this embodiment, the automatic interface allocation information is an interface allocation strategy based on intelligent analysis. This information may include one or more physical audio output interfaces corresponding to each split-screen window or application within each split-screen window. Based on this information, the physical audio output interface corresponding to each split-screen window can be determined. Specifically, the physical audio output interface corresponding to a split-screen window can be either "one or more physical audio output interfaces corresponding to the split-screen window itself" or "one or more physical audio output interfaces corresponding to the application within the split-screen window."

[0073] When a split-screen window corresponds to multiple different physical audio output interfaces, in some methods, the split-screen audio signal corresponding to the split-screen window can be simultaneously sent to all of these different physical audio output interfaces; or, in some methods, the split-screen audio signal corresponding to the split-screen window can be sent to only one physical audio output interface according to a preset priority or user selection; or, in some methods, the split-screen audio signals corresponding to multiple different split-screen windows can be simultaneously sent to the same physical audio output interface for mixing and output.

[0074] In some methods, a preset interface allocation model can be used to analyze and process the split-screen data before splitting the screen, obtaining the automatic interface allocation information output by the preset interface allocation model. In other methods, when a plug-in / plug-out event (i.e., the audio output device corresponding to the physical audio output interface is connected to or disconnected from the interface) or a disconnection event (i.e., the physical audio output interface cannot be found) is detected at the physical audio output interface, the preset interface allocation model can be triggered to analyze and process the split-screen data, obtaining the automatic interface allocation information output by the preset interface allocation model.

[0075] See Figure 5 In one embodiment, transmitting the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window may include: step S510, when it is detected that the physical audio output interface corresponding to the split-screen window meets the switching conditions, performing an interface switching operation to determine the new physical audio output interface corresponding to the split-screen window; step S520, transmitting the split-screen audio signal corresponding to the split-screen window to the new physical audio output interface.

[0076] During the process of determining the physical audio output interface corresponding to the split-screen window and sending the split-screen audio signal to that interface, if the physical audio output interface of the split-screen window meets the switching conditions, an interface switching operation is performed. This interface switching operation determines the new physical audio output interface for each split-screen window. By sending the split-screen audio signal to the new physical audio output interface, hot-switching of the physical audio output interface for each split-screen window can be achieved.

[0077] In some methods, the switching condition can be determined when the physical audio output interface corresponding to the split-screen window is detected to be unavailable. The unavailability of the physical audio output interface may include, but is not limited to, situations where the audio output device corresponding to the physical audio output interface is disconnected from the interface, or the physical audio output interface cannot be found due to abnormalities. In other methods, the switching condition can be determined when a user-sent instruction to switch interfaces (such as a voice command or a control-triggered instruction) is received.

[0078] In some methods, the interface switching operation can be performed by automatically selecting a new available physical audio output interface according to a preset priority; or, in other methods, the interface switching operation can be performed by displaying an interface customization interface so that the user can select a new physical audio output interface.

[0079] Furthermore, in one embodiment, before sending the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, it may further include: configuring a main audio buffer corresponding to each split-screen window for each split-screen window in the multiple split-screen windows, wherein the main audio buffer corresponding to each split-screen window is used to cache the split-screen audio signal corresponding to each split-screen window respectively.

[0080] For each of the multiple split-screen windows, an independent main audio buffer is configured for each window. After capturing the split-screen audio signal for each window independently, the split-screen audio signal for each window can be cached in its corresponding main audio buffer, which can prevent audio interleaving or data loss between multiple split-screen windows. Furthermore, the split-screen audio signal is sent to the physical audio output interface through this main audio buffer. This allows for the use of a producer-consumer model or similar approach, dynamically preloading the split-screen audio signal for each window into its corresponding main audio buffer, further prioritizing the prevention of audio stuttering. Furthermore, in one embodiment, transmitting the split-screen audio signal corresponding to the split-screen window to the new physical audio output interface may include: transferring the split-screen audio signal corresponding to the split-screen window from the main audio buffer to the backup audio buffer; and transmitting the signal from the backup audio buffer to the new physical audio output interface.

[0081] In this embodiment, a corresponding backup audio buffer is configured for each split-screen window. During hot switching of the physical audio output interface, the split-screen audio signal corresponding to the split-screen window is transferred from the main audio buffer to the backup audio buffer, and the signal is transmitted from the backup audio buffer to the new physical audio output interface. Furthermore, each audio stream implements "double buffering" technology, achieving seamless transfer of audio signals through the backup audio buffer during hot switching, reducing frame loss.

[0082] Furthermore, in one embodiment, for each split-screen window, a synchronization compensation algorithm (such as a timestamp alignment algorithm) can be used to align the audio with the application screen, ensuring that the application screen and audio in the split-screen window match naturally.

[0083] Furthermore, in one embodiment, the split-screen audio signals corresponding to each of the multiple split-screen windows are independently decoded by a concurrent decoding engine (such as a decoding engine based on FFmpeg, LibAV, etc.). When the split-screen audio signals are compressed data (such as AAC, MP3, etc.), the concurrent decoding engine can concurrently and independently decode the split-screen audio signals corresponding to multiple split-screen windows, which can further ensure audio processing efficiency.

[0084] In some examples, the split-screen audio signal can be decoded by a concurrent decoding engine before being written to the main audio buffer; in other examples, the split-screen audio signal can be decoded by a concurrent decoding engine when it is sent from the main audio buffer to the physical audio output interface.

[0085] Furthermore, in one embodiment, before transmitting the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method may further include: determining the interface allocation mode selected by the user, the interface allocation mode including a first mode or a second mode; wherein, in the first mode, the physical audio output interface corresponding to the split-screen window is determined based on the user-defined interface configuration; in the second mode, the physical audio output interface corresponding to the split-screen window is determined based on the automatic interface allocation information.

[0086] The system provides an interface allocation mode selection function, allowing users to choose either a first mode or a second mode based on their needs. In the first mode, the physical audio output interface corresponding to the split-screen window is determined based on the aforementioned user-defined interface configuration. In the second mode, the physical audio output interface corresponding to the split-screen window is determined based on the aforementioned automatic interface allocation information. This flexible mode selection further enhances the user experience.

[0087] To facilitate better implementation of the split-screen processing method provided in this application embodiment, this application embodiment also provides a split-screen processing device based on the above-described split-screen processing method. The meanings of the terms used are the same as in the above-described split-screen processing method, and specific implementation details can be found in the descriptions in the method embodiments. Figure 6 A block diagram of a split-screen processing apparatus according to an embodiment of this application is shown.

[0088] like Figure 6 As shown, the split-screen processing device 600 may include: an audio capture module 610, which can be used to: acquire the split-screen audio signal corresponding to each of the multiple split-screen windows, wherein the split-screen audio signal refers to the audio signal corresponding to the application in the split-screen window; and an audio routing module 620, which can be used to: send the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, wherein the physical audio output interfaces corresponding to the multiple split-screen windows are different or the same.

[0089] In some embodiments of this application, before transmitting the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the device further includes a first interface determination module that can be used to: determine the physical audio output interface corresponding to the split-screen window according to a user-defined interface configuration, wherein the user-defined interface configuration includes one or more physical audio output interfaces corresponding to the split-screen window specified by the user or the application in the split-screen window.

[0090] In some embodiments of this application, before determining the physical audio output interface corresponding to the split-screen window based on the user interface configuration, the device further includes a first allocation module that can be used to: display the interface customization interface when the interface setting conditions are met; and save the user-defined interface configuration set by the user through the interface customization interface.

[0091] In some embodiments of this application, before the split-screen audio signal corresponding to the split-screen window is transmitted to the physical audio output interface corresponding to the split-screen window, the device further includes a second interface determination module which can be used to: determine the physical audio output interface corresponding to the split-screen window according to interface automatic allocation information, wherein the interface automatic allocation information is obtained based on split-screen data decision analysis, and the split-screen data includes one or more of audio data, user historical interface allocation data, application data, and device interface configuration data.

[0092] In some embodiments of this application, before determining the physical audio output interface corresponding to the split-screen window based on the interface automatic allocation information, the device further includes a second allocation module that can be used to: determine the window priority corresponding to each split-screen window based on the split-screen data; allocate interfaces based on the window priority corresponding to each split-screen window to obtain the interface automatic allocation information, wherein the interface automatic allocation information includes one or more physical audio output interfaces corresponding to each split-screen window or application in the split-screen window.

[0093] In some embodiments of this application, before determining the physical audio output interface corresponding to the split-screen window based on the interface automatic allocation information, the device further includes a second allocation module that can be used to: analyze and process the split-screen data using a preset interface allocation model to obtain the interface automatic allocation information output by the preset interface allocation model, wherein the interface automatic allocation information includes one or more physical audio output interfaces corresponding to each split-screen window or application in the split-screen window.

[0094] In some embodiments of this application, when the split-screen audio signal corresponding to the split-screen window is sent to the physical audio output interface corresponding to the split-screen window, the audio routing module can be used to: when it is detected that the physical audio output interface corresponding to the split-screen window meets the switching conditions, perform an interface switching operation to determine the new physical audio output interface corresponding to the split-screen window; and send the split-screen audio signal corresponding to the split-screen window to the new physical audio output interface.

[0095] In some embodiments of this application, when the split-screen audio signal corresponding to the split-screen window is transmitted to the new physical audio output interface, the audio routing module can be used to: transfer the split-screen audio signal corresponding to the split-screen window from the main audio buffer to the backup audio buffer; and transmit the signal from the backup audio buffer to the new physical audio output interface.

[0096] In some embodiments of this application, before transmitting the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the device further includes a mode switching module for: determining the interface allocation mode selected by the user, the interface allocation mode including a first mode or a second mode; wherein, in the first mode, the physical audio output interface corresponding to the split-screen window is determined based on the user-defined interface configuration; in the second mode, the physical audio output interface corresponding to the split-screen window is determined based on the automatic interface allocation information.

[0097] In some embodiments of this application, when obtaining the split-screen audio signal corresponding to each of the multiple split-screen windows, the frequency capture module can be used to: intercept the audio signal corresponding to the application in the split-screen window from the audio data before the audio data is written to the unified buffer, and obtain the split-screen audio signal corresponding to the split-screen window.

[0098] In some embodiments of this application, before the split-screen audio signal corresponding to the split-screen window is transmitted to the physical audio output interface corresponding to the split-screen window, the buffer configuration module is used to: configure a main audio buffer corresponding to each of the multiple split-screen windows, and the main audio buffer corresponding to each of the split-screen windows is used to cache the split-screen audio signal corresponding to each of the split-screen windows respectively.

[0099] In some embodiments of this application, the split-screen audio signals corresponding to each of the plurality of split-screen windows are independently decoded by a concurrent decoding engine.

[0100] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0101] Furthermore, embodiments of this application also provide an electronic device, such as... Figure 7 As shown, Figure 7 A block diagram of an electronic device according to an embodiment of this application is shown, specifically: The electronic device may include components such as a processor 701 with one or more processing cores, a memory 702 with one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will understand that... Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 701 is the control center of the electronic device, connecting various parts of the computer device via various interfaces and lines. It executes various functions and processes data by running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and application programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 701.

[0102] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0103] The electronic device also includes a power supply 703 that supplies power to the various components. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0104] The electronic device may also include an input unit 704, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0105] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the electronic device can load the executable files corresponding to the processes of one or more computer programs into the memory 702 according to the following instructions, and the processor 701 runs the computer programs stored in the memory 702, thereby realizing the various functions in the foregoing embodiments of this application.

[0106] The mainboard of the electronic device can be configured with multi-channel audio codecs (such as TI PCM3168A, Realtek ALC series, etc.) or FPGA audio matrix modules. The physical output of each split-screen audio signal can be independently converted and amplified by a DA converter. The hardware chip of the electronic device can support physical isolation, independent gain adjustment, and mute of multiple split-screen audio signals to multiple physical audio output interfaces. The peripheral circuitry of the electronic device can have dedicated buffers / isolation amplification for each split-screen audio signal output to ensure no crosstalk or interference between interfaces.

[0107] For example, processor 701 can perform the following: for each of the multiple split-screen windows, obtain the split-screen audio signal corresponding to the split-screen window, wherein the split-screen audio signal refers to the audio signal corresponding to the application in the split-screen window; and send the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, wherein the physical audio output interfaces corresponding to the multiple split-screen windows are different or the same.

[0108] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0109] Therefore, embodiments of this application also provide a storage medium storing a computer program that can be loaded by a processor to execute the steps in any of the methods provided in embodiments of this application.

[0110] The storage medium can be a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0111] Since the computer program stored in the storage medium can execute the steps of any of the methods provided in the embodiments of this application, the beneficial effects that the methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0112] According to another embodiment of this application, a computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in the various optional implementations described in the embodiments of this application.

[0113] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0114] It should be understood that this application is not limited to the embodiments described above and shown in the accompanying drawings, but various modifications and changes can be made without departing from its scope.

Claims

1. A split-screen processing method, characterized by, The method comprises: For each of the plurality of split-screen windows, obtaining a split-screen audio signal corresponding to the split-screen window, the split-screen audio signal being an audio signal corresponding to an application in the split-screen window; delivering the split-screen audio signal corresponding to the split-screen window to a physical audio output interface corresponding to the split-screen window, wherein the physical audio output interfaces corresponding to the plurality of split-screen windows are different or the same.

2. The method of claim 1, wherein, Before the delivering the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method further comprises: determining the physical audio output interface corresponding to the split-screen window according to a user-defined interface configuration, wherein the user-defined interface configuration includes one or more physical audio output interfaces corresponding to the split-screen window or an application in the split-screen window specified by a user.

3. The method of claim 2, wherein, Before the determining the physical audio output interface corresponding to the split-screen window according to the user-defined interface configuration, the method further comprises: displaying an interface customization interface when an interface setting condition is met; saving the user-defined interface configuration set by the user through the interface customization interface.

4. The method of claim 1, wherein, Before the delivering the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window, the method further comprises: determining the physical audio output interface corresponding to the split-screen window according to interface automatic allocation information, wherein the interface automatic allocation information is obtained according to split-screen data decision analysis, and the split-screen data includes one or more of audio data, user historical interface allocation data, application data, and device interface configuration data.

5. The method of claim 4, wherein, Before the determining the physical audio output interface corresponding to the split-screen window according to the interface automatic allocation information, the method further comprises: determining a window priority corresponding to each of the split-screen windows according to the split-screen data; performing interface allocation according to the window priority corresponding to each of the split-screen windows to obtain the interface automatic allocation information, wherein the interface automatic allocation information includes one or more physical audio output interfaces corresponding to each of the split-screen windows or an application in the split-screen window.

6. The method of claim 4, wherein, Before the determining the physical audio output interface corresponding to the split-screen window according to the interface automatic allocation information, the method further comprises: analyzing and processing the split-screen data by using a preset interface allocation model to obtain the interface automatic allocation information output by the preset interface allocation model, wherein the interface automatic allocation information includes one or more physical audio output interfaces corresponding to each of the split-screen windows or an application in the split-screen window.

7. The method of claim 1, wherein, The delivering the split-screen audio signal corresponding to the split-screen window to the physical audio output interface corresponding to the split-screen window comprises: when it is detected that the physical audio output interface corresponding to the split-screen window meets a switching condition, performing an interface switching operation to determine a new physical audio output interface corresponding to the split-screen window; delivering the split-screen audio signal corresponding to the split-screen window to the new physical audio output interface.

8. The method of claim 7, wherein, The delivering the split-screen audio signal corresponding to the split-screen window to the new physical audio output interface comprises: transfer the split-screen audio signal corresponding to the split-screen window from a main audio buffer to a backup audio buffer; and signal transmission from the backup audio buffer to the new physical audio output interface.

9. The method according to any one of claims 1 to 8, characterized in that, Before the split-screen audio signal corresponding to the split-screen window is transported to the physical audio output interface corresponding to the split-screen window, the method further comprises: determining a user-selected interface allocation mode, the interface allocation mode including a first mode or a second mode; wherein the first mode determines the physical audio output interface corresponding to the split-screen window based on a user-defined interface configuration; and the second mode determines the physical audio output interface corresponding to the split-screen window based on interface automatic allocation information.

10. The method according to any one of claims 1 to 8, characterized in that, The method of obtaining the split-screen audio signal corresponding to each of the split-screen windows in the plurality of split-screen windows comprises: intercepting the audio signal corresponding to the application in the split-screen window from the audio data before the audio data is written to a unified buffer, to obtain the split-screen audio signal corresponding to the split-screen window.

11. The method according to any one of claims 1 to 8, characterized in that, Before the split-screen audio signal corresponding to the split-screen window is transported to the physical audio output interface corresponding to the split-screen window, the method further comprises: configuring a main audio buffer corresponding to each of the split-screen windows in the plurality of split-screen windows, respectively, the main audio buffer corresponding to each of the split-screen windows being used to buffer the split-screen audio signal corresponding to each of the split-screen windows, respectively.

12. The method according to any one of claims 1 to 8, characterized in that, The split-screen audio signal corresponding to each of the split-screen windows in the plurality of split-screen windows is independently decoded by a concurrent decoding engine, respectively.

13. A split-screen processing apparatus, characterized by comprising: The method comprises: an audio capture module configured to obtain a split-screen audio signal corresponding to each of a plurality of split-screen windows, the split-screen audio signal corresponding to an audio signal corresponding to an application in the split-screen window; an audio routing module configured to transport the split-screen audio signal corresponding to the split-screen window to a physical audio output interface corresponding to the split-screen window, wherein the physical audio output interfaces corresponding to the plurality of split-screen windows are different or the same.

14. A storage medium, characterized by A computer program is stored thereon, and when the computer program is executed by a processor of an electronic device, the electronic device executes the method of any one of claims 1 to 12.

15. An electronic device, comprising: The method comprises: a memory storing a computer program; a processor reading the computer program stored in the memory to execute the method of any one of claims 1 to 12.

Images