Audio processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an audio processing method and device, equipment, a storage medium and a program product, and the method comprises the steps: responding to a starting operation of a target application on a target screen, and obtaining the identity information of the target application and the screen information of the target screen; searching whether identity information of the target application exists in a preset database or not; and if the identity information of the target application exists in the preset database and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, starting the target application. And if yes, switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing. According to the embodiment of the invention, the audio frequency can automatically follow the application to switch the screens when the plurality of screens are switched.

Description

Technical Field

[0001] The present application relates to the field of computer application technologies, and in particular, to an audio processing method, an audio processing device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the progress of multi-screen display technology, electronic devices have started to adopt multi-screen different display technologies, that is, one device supports connecting multiple screens, and this technology enables each screen to display different contents. Some products not only support multi-screen different display but also have the function of different sounds, allowing each screen to have an independent audio playback channel.

[0003] In the existing technical solutions for multi-screen audio playback, when a user starts an application on the main screen and plays audio, if the application is switched to another screen, although the interface of the application can be successfully switched, the audio output of the application still stays in the audio player on the main screen and cannot be synchronously switched to the audio player on another screen, resulting in the audio not automatically following the application to switch screens when the application switches between the main screen and other screens. Summary of the Invention

[0004] Embodiments of the present application provide an audio processing method, device, equipment, storage medium, and program product, which can realize that the audio automatically follows the application to switch screens when the application switches between multiple screens.

[0005] On the one hand, an embodiment of the present application provides an audio processing method, which includes:

[0006] In response to a start operation of a target application on a target screen, obtain the identity information of the target application and the screen information of the target screen;

[0007] Search in a preset database to find out whether there is the identity information of the target application; the preset database includes the identity information of at least one application and the screen information for starting each application;

[0008] If the identity information of the target application exists in the preset database and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, then switch the audio of the target application from the audio player on the screen corresponding to the screen information for starting the target application included in the preset database to the audio player on the target screen for playback.

[0009] In one of the embodiments, in response to a start operation of a target application on a target screen, obtaining the identity information of the target application and the screen information of the target screen includes:

[0010] Receive string information through an information transmission interface, and the string information is sent by calling an information transmission method;

[0011] Parse the string information to obtain the identity information of the target application and the screen information of the target screen.

[0012] In one embodiment, the string information is obtained by assembling the identity information of the target application and the screen information of the target screen using an information assembly method for calls, and the identity information of the target application and the screen information of the target screen are obtained by the activity start module in response to the start operation of the target application on the target screen.

[0013] In one embodiment, before switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing, the method further includes:

[0014] If the audio player of the target screen is playing the audio of another application, obtain the audio playback permission of the target screen;

[0015] Adjust the audio playback of other applications according to the audio playback permission, and switch the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing; adjusting the audio playback of other applications includes reducing the volume level of the audio playback of other applications and pausing the audio playback of other applications.

[0016] In one embodiment, the method further includes:

[0017] In response to the start operation of the target application on the target screen, if the remaining hardware resources are less than the hardware resources required to start the target application, release some of the hardware resources allocated to the applications in the playing state to obtain the updated remaining hardware resources;

[0018] Allocate the updated remaining hardware resources to the target application to start the target application.

[0019] In one embodiment, after switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing, it further includes:

[0020] Record the audio switching information of the target application in the log file; the log file is used for fault detection of the audio switching operation of the target application; the audio switching information includes the time information of the audio switching operation of the target application, the identity information of the target application, the screen information of the target screen, and the screen information for starting the target application included in the preset database.

[0021] On the other hand, an audio processing device is provided in an embodiment of the present application, which is characterized by including:

[0022] An acquisition unit, in response to a start operation of a target application on a target screen, acquires the identity information of the target application and the screen information of the target screen;

[0023] A processing unit is configured to search in a preset database for the identity information of the target application; the preset database includes the identity information of at least one application and the screen information for starting each application;

[0024] If the identity information of the target application exists in the preset database and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, the processing unit switches the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing.

[0025] On the other hand, an embodiment of the present application provides a computer device, which includes a processor, an input device, an output device, and a memory. The processor, the input device, the output device, and the memory are interconnected. Among them, the memory is used to store a computer program, and the computer program includes program instructions. The processor is configured to call the program instructions to execute the method according to any one of the above.

[0026] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is characterized in that the computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to any one of the above.

[0027] On the other hand, an embodiment of the present application provides a computer program product, which is characterized in that the computer program product includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, the method according to any one of the above is implemented.

[0028] In the embodiment of the present application, when the target application is started on the target screen, the identity information of the target application and the screen information of the target screen are acquired. Then, the acquired identity information and screen information are compared with the information stored in the database. If the identity information of the target application is detected in the database, but the screen information corresponding to the target application in the database is different from the currently acquired screen information, the audio of the target application is switched to the audio player of the target screen for playing, realizing that when the interface of the target application is switched between different screens, the audio of the target application automatically follows the screen switch of the target application. Description of the Drawings

[0029] To more clearly illustrate the technical methods in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0030] Figure 1 It is a schematic diagram of the data flow of an audio processing method provided by an embodiment of the present application;

[0031] Figure 2 It is a schematic flowchart of a process based on an audio processing method provided by an embodiment of the present application;

[0032] Figure 3 It is a schematic diagram of multi-audio focus strategy management provided by an embodiment of the present application;

[0033] Figure 4 It is a schematic diagram of the structure of an audio processing device provided by an embodiment of the present application;

[0034] Figure 5 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0035] The following will clearly and completely describe the technical methods in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0036] The embodiments of the present application provide an audio processing method, specifically providing a method for the audio to follow the application for screen switching. After the screen display of the application interface is switched, the audio of the application follows the interface of the application and is played on the audio player corresponding to the screen. First, the technical terms and related concepts involved in the audio processing method are introduced, where:

[0037] 1. Audio

[0038] Audio is the sound transmitted through a certain medium, which can be naturally produced or man-made. Its physical characteristics include pitch, loudness, and timbre, which are determined by the frequency, amplitude, and waveform of sound waves respectively. Audio is divided into analog audio and digital audio. Analog audio directly simulates the characteristics of sound waves, while digital audio converts analog signals into discrete binary signals through sampling, quantization, and encoding, which is convenient for computer processing and storage. In audio files, bit rate and sampling rate are important parameters to describe digital audio signals, which affect file size and sound quality. Audio signal processing technologies such as noise reduction, equalization, and dynamic range compression are used to improve sound quality or adapt to different environments. Audio technology is widely used in fields such as music production, film and video production, broadcasting, communication, game sound effects, and speech recognition.

[0039] 2. Audio Stream

[0040] An audio stream is a continuous data sequence that exists throughout the process from the generation to the playback of audio. First, an audio input device such as a microphone collects and digitizes the analog signal of sound waves. Then, it is encoded and compressed to reduce the data volume for storage and transmission. At the receiving end, the original digital audio data is restored through decoding, and then an analog signal is generated with the help of a digital-to-analog converter. Finally, it is played by audio output devices such as speakers and headphones to achieve the effective transmission of sound.

[0041] 3. Multi-Screen Different Display

[0042] Multi-screen different display technology is a technology that connects multiple screens to one device, and each screen displays different visual content. The core advantage of this technology lies in providing more efficient multitasking capabilities and a richer and more immersive user experience, especially suitable for scenarios that require monitoring multiple audio and video streams simultaneously or performing complex operations. At the hardware level, the motherboard of the device must have the ability to interact with multiple screens simultaneously, which usually involves advanced graphics processing units and multiple video output interfaces. At the software level, the operating system needs to be able to recognize and manage multiple screens, as well as their display content and user interactions. To achieve multi-screen different display, specific software configurations are required, including installing specific firmware and software, and managing the display content of multiple screens at the operating system level. Users can set the display content of multiple screens according to their needs, such as videos, audios, pictures, documents, etc., to meet different usage requirements.

[0043] Since the audio cannot automatically follow the switch when the current application switches between the main screen and other screens, the synchronization experience of audio is poor when the application switches between multiple screens.

[0044] To better solve the problem that the audio of an application does not switch with the application interface when the application interface switches screens, the embodiments of the present application propose an audio processing method, aiming to improve the synchronization of audio switching when the application switches between multiple screens. This method realizes a scheme in which the application audio follows the application to switch screens. After the application interface switches screens to display the application interface, the application audio can also be switched to the audio player of the corresponding screen for playback, effectively improving the synchronization of audio switching when the application switches between multiple screens.

[0045] The audio processing method proposed by the embodiments of the present application can detect the startup state of the application and switch the application audio to play audio with the corresponding screen. The general process of this method may include: after the target application starts, obtain the identity information and screen information of the target application, and obtain the identity information and screen information of all the started applications in the device, compare the identity information and screen information of the target application and the started applications, and adjust the audio playback position of the target application according to the comparison result to realize the switching of the application audio following the application when the application switches screens, which can improve the audio management efficiency and user experience in a multi-screen environment.

[0046] An application is computer software designed to achieve a specific purpose. An application usually has a graphical user interface or a command-line interface, allowing users to interact with the application through graphical elements or text commands. In a web application, the application can be divided into a front end for direct user interaction and a back end for processing data storage and business logic. The operating systems of many computer devices include an application store, where users can browse, download, and install application programs. Security and privacy protection are important considerations in designing applications. Developers need to ensure that the application can protect user data from being accessed without authorization and comply with relevant privacy regulations. Applications usually need to be updated regularly to fix vulnerabilities, improve functions, and enhance performance. These updates can be automatic or installed manually by the user.

[0047] The screen is the core component for visual interaction in modern electronic devices. Its basic function is to display images, videos, and text information, enabling users to receive visual information and interact with it. There are various types of screens, including liquid crystal screens, light-emitting diode screens, organic light-emitting diodes, quantum dot screens, and plasma screens, as well as electronic paper that mimics the paper reading experience. The resolution of the screen determines the number of pixel points it can display, and the size of the screen refers to the length of the diagonal of the screen. The color performance of the screen involves parameters such as color gamut, brightness, and contrast, which jointly affect the vividness and realism of the image. Many screens integrate touch technologies such as capacitive screens and resistive screens, allowing users to directly touch and operate. The refresh rate of the screen, that is, the number of times the image is refreshed per second, affects the clarity of dynamic images. Screen protection technologies such as scratch-resistant glass, anti-glare coatings, and anti-fingerprint coatings are designed to protect the screen from damage. With the development of technology, screen technology is also constantly advancing, and new screen technologies such as curved screens, flexible screens, and foldable screens have emerged. Screens are widely used in mobile phones, tablets, personal computers, TVs, monitors, automotive display systems, medical devices, industrial control, and many other fields, and their technological development directly affects the user experience.

[0048] An audio processing method is proposed in an embodiment of this application. By recording and comparing the identity information and screen information of applications, it realizes that the audio of an application follows the application during screen switching, optimizes the audio output management on multi-screen display devices, and improves the auditory experience of users in a multi-screen environment. The audio processing method provided in the embodiment of this application can be applied to computer devices and multiple screens, and the computer device includes a processor.

[0049] The screen is responsible for displaying application icons, capturing users' click operations on application icons, interacting with the processor, and ensuring continuous playback of audio and video content. The computer device is connected to multiple screens. When the user clicks on the icon of an application on any screen, the processor starts the application on the current screen and displays the interface of the currently started application, and at the same time starts playing the audio content.

[0050] When an application has been launched on Screen A and is playing the application audio, if the user clicks on the same application on Screen B, it will trigger the switching process of the application screen and audio. The processor interacts with the screen to obtain the identity number (Identity, ID) information of the screen that is currently displaying the application, that is, the screen information of the screen, which is used for the information verification of the application. After the interface of the application is switched to Screen B, the audio of the application is also switched to the audio player on Screen B for playback. During the audio switching process, if any errors or exceptions occur, the processor will take relevant recovery measures and provide corresponding user prompts on the screen to ensure the stability of audio and video playback. The performance and responsiveness of the screen directly affect the user experience and the convenience of operation. In a multi-screen environment, different screens need to work together to enable the application audio to follow the application as the screen is switched, ensuring the synchronization and consistency of audio and video content when switching between multiple screens.

[0051] In the computer field, an ID is a code or identifier used to uniquely identify and distinguish each component, entity, or data item in a system. At the hardware level, IDs are used to identify specific hardware devices, such as the Media Access Control (MAC) address of a network card and the Internet Protocol (IP) address. IDs ensure the uniqueness of devices in the network, enabling data to be accurately sent and received. In addition, each hardware component, such as the Central Processing Unit (CPU), hard disk, memory, etc., usually has its own ID. The ID of the hardware provides a unique identification and recognition for each device. By reading the ID, the type, manufacturer, and other key attributes of the device can be determined, thus facilitating device management, configuration, and compatibility checks. At the software level, IDs are used to identify software products, license keys, version numbers, etc. Software registration numbers or license IDs ensure the legal use and copyright protection of software. In software development, IDs are also used to identify variables, functions, classes, and modules in a program to facilitate code organization and management.

[0052] At the hardware level, the functions of the hardware ID include: authorizing and licensing verification of software products. By binding software or digital content to a specific hardware ID, developers can ensure that their products can only run on authorized devices, thus preventing unauthorized copying and use; used to identify and verify devices to ensure the validity of software licenses, prevent illegal copying and distribution, and can also be used to track lost or stolen devices in the Internet of Things environment; used to identify devices in the network to ensure that data can be accurately sent and received; used to help the operating system identify devices and find appropriate drivers to ensure the normal operation of the devices; by monitoring changes in specific hardware IDs, potential security threats or intrusion behaviors can be detected.

[0053] The processor in the computer device is responsible for collecting the identity information of the started application and the ID information of the screen displaying the application. After the application switches screens, the playback position of the application audio is switched by comparing the identity information of the application and the ID information of the screen where it is located.

[0054] When the user starts a certain application and plays audio and video content on a certain screen, the processor obtains the user identification (UserIdentification, UID) information of the application and the ID information of screen A through the information acquisition interface of the ActivityStarter module. The UID information is the identity information of the application. Subsequently, the processor assembles the UID information of the application and the ID information of the screen displaying the application into the string handle_hdmi = display:uid through the startActivityInner method. Each string contains the UID information of an application and the ID information of the screen displaying the application, and through the AudioManager::setParameters method, the string containing the UID information of the application and the ID information of the screen is transmitted to the audio hardware abstraction layer Audio Hal in a way of inter-process communication. The string is received by the audio hal adev_set_parameters interface of Audio Hal. According to the information of the application and the screen received by Audio Hal, the processor compares the UID information of different applications and the ID information of the screen displaying the application to realize the switching of the playback position of the application audio.

[0055] ActivityStarter is a component in the operating system of the computer device. In the audio processing method proposed in this embodiment, ActivityStarter is mainly responsible for information acquisition. The functions of ActivityStarter include creating and starting data as a controller, as well as transmitting relevant parameters, preparing information, managing the task stack, designing interfaces, and so on.

[0056] In the computer field, UID, that is, user identification, is a unique identifier used to identify the user identity. It is widely used in operating systems, database management systems, and other occasions where different users need to be distinguished. In the audio processing method proposed in this embodiment, UID is mainly used to identify different applications. UID has consistency, and the UID of each application usually does not change.

[0057] startActivityInner is a method in the operating system of a computer device. In the audio processing method proposed in this embodiment, startActivityInner is mainly responsible for the assembly operation of information, assembling the UID information of the application and the ID information of the screen into a string. The functions of startActivityInner include status initialization, task station management, cross-process communication, transaction scheduling, and so on.

[0058] handle_hdmi = display:uid is a string format, where handle_hdmi is the parameter header for identifying this string; display represents display_id, that is, the ID information of the screen. uid represents the UID information of the application.

[0059] The AudioManager::setParameters method is a function for setting audio parameters, for passing audio parameters, and for dynamically modifying the behavior and configuration of audio hardware.

[0060] Audio Hal provides an interface for the application to communicate with the audio hardware and is an important component in the operating system. Its functions include: managing audio streams, determining the input and output devices for audio stream transmission, and processing functions such as audio focus, audio sessions, and audio policies; handling the format conversion, encoding, and decoding of audio data, and providing audio effect processing such as equalizers and sound field effects; managing audio latency and synchronization to ensure the timing consistency of audio stream input and output; providing a set of standard audio application programming interfaces (Application Programming Interface, API) to implement functions such as audio playback, recording, and processing.

[0061] audio hal adev_set_parameters is an interface in Audio Hal and is used to receive the audio information of the device in the audio processing method proposed in this embodiment.

[0062] Figure 1 It is a schematic diagram of the data flow of an audio processing method provided by an embodiment of this application, as Figure 1As shown, the data flow of the audio processing method is divided into three layers. The ActivityStarter module is the first layer. After the processor obtains the UID information of the application and the ID information of the screen through the information acquisition interface of ActivityStarter, it uses the AudioManager::setParameters method to send the handle_hdmi=display:uid string containing the UID information of the application and the ID information of the screen to the Audio Service. In Figure 1 the handle_hdmi=display:uid string, the UID information of the application it contains is represented by uid, and the ID information of the screen it contains is represented by display_id. The Audio Service is the second layer of the data flow, responsible for inter-process communication of information. The processor sends the received information to the Audio Hal through the Audio Service. The Audio Hal is the third layer of the data flow. After receiving the string, the Audio Hal obtains the UID information of the application and the ID information of the screen by parsing the content of the string, and compares the parsed information with the information of all applications currently playing audio. If it is found that the parsed UID information of the application is the same as the UID information of a certain audio application, and the parsed ID information of the screen is different from the ID information of the screen of the audio application with the same UID information of the application, then the audio playback position of the application is switched.

[0063] The Audio Service is a module in the operating system of a computer device used to manage audio resources, mainly responsible for allocating, controlling, and processing audio resources.

[0064] The functions of the processor also include checking whether the audio player on the screen is playing the audio of other applications before the audio of the application is sent to the audio player on the screen for playback. If it is detected that the audio player is in use, the processor will obtain the audio playback permission of the screen according to the preset audio policy and play the audio of the application.

[0065] The functions of the processor also include that when the user performs the startup operation of the application, if it is detected that the hardware resources required to start the application are insufficient, the processor will increase the priority of the application's use of hardware resources, and ensure the smooth startup of the target application by reducing the hardware resources used by other applications that are playing audio.

[0066] The functions of the processor also include recording the audio switching information of the target application in the log file after the playback position of the audio of the application is switched. The audio switching information includes the time information of the audio switching operation of the application, the UID information, the ID information of the screen where the audio is played, and the ID information of the screen that starts the target application in the preset database, etc. The recorded information is used for subsequent possible troubleshooting and other related work.

[0067] The embodiments of the present application do not specifically limit the execution subject of the audio processing method to ensure the flexibility and wide applicability of the method. The following will use a specific embodiment to illustrate the execution process of the audio processing method and explain the audio processing process in detail.

[0068] The user clicks on a certain application on Screen A, and the application starts for the first time. The processor is responsible for obtaining the UID information of the application and the ID information of Screen A through the information acquisition interface of the ActivityStarter module. Then, a string containing the UID information of the application and the ID information of the screen displaying the application is assembled through the startActivityInner method, and the string is passed to the audio hardware abstraction layer Audio Hal using the AudioManager::setParameters method. After receiving the string containing the UID information of the application and the ID information of the screen through the audio hal adev_set_parameters interface, the processor parses the string, identifies the UID information of the application and the ID information of the screen displaying the application, and accurately sends the audio to the audio player of the screen displaying the application for playback.

[0069] After the application is launched on screen A, when the user clicks on the same application on screen B, the application is launched for the second time. The processor triggers the ActivityStarter module again, and retrieves the current UID information of the application and the ID information of the screen through the information retrieval interface of the ActivityStarter module. The processor then sends a string containing the re-retrieved UID information of the application and the ID information of the screen to the Audio Hal through the AudioManager::setParameters method. The processor parses this information through the audiohal adev_set_parameters interface, traverses all the currently playing audio, and compares the parsed UID information of the application and the ID information of the screen with the UID information of the application to which the currently playing audio belongs and the ID information of the screen of the application with the same UID information. When the parsed UID information of the application is the same as the UID information of an application to which one of the currently playing audio belongs, and the parsed ID information of the screen is different from the ID information of the screen of the application with the same UID information of the audio, the audio of the application corresponding to the parsed UID information of the application is sent to the screen where the parsed ID information of the screen is located, that is, the audio of the application is sent to the audio player on screen B for playback.

[0070] Through the close cooperation between the screen and the processor of the computer device, the audio processing method provided by the embodiments of the present application can ensure that the audio automatically follows the application to switch screens when the application switches between multiple screens.

[0071] It should also be noted that the collection and processing of relevant data in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations. Obtaining personal information requires the informed consent of the personal subject (or having a legal basis for information acquisition), and subsequent data use and processing behaviors should be carried out within the scope authorized by laws, regulations and the personal information subject.

[0072] Based on the above-described audio processing method, the embodiments of the present application further propose a more detailed audio processing method. The following will introduce the audio processing method proposed by the embodiments of the present application in detail with reference to the accompanying drawings.

[0073] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of an audio processing method provided by the embodiments of the present application. Figure 2 The audio processing method shown mainly records and compares the UID information of the application and the ID information of the screen, and switches the playback position of the application audio according to the comparison result. The video processing method includes but is not limited to steps S201 - S203:

[0074] S201: In response to the startup operation of the target application on the target screen, obtain the identity information of the target application and the screen information of the target screen.

[0075] In an optional instance, in response to the startup operation of the target application on the target screen, obtain the identity information of the target application and the screen information of the target screen, including receiving string information through an information transmission interface, where the string information is sent by invoking an information transmission method; parse the string information to obtain the identity information of the target application and the screen information of the target screen.

[0076] The information transmission interface is the audio hal adev_set_parameters interface for receiving string information. The string information is the handle_hdmi=display:uid string, which contains the UID information of the application and the ID information of the screen. handle_hdmi is the parameter header, display represents display_id, i.e., the ID information of the screen, and uid represents the UID information of the application. The information transmission method is the AudioManager::setParameters method, which uses inter-process communication to transmit the handle_hdmi=display:uid string to the Audio Hal. Parse the handle_hdmi=display:uid string stored in the Audio Hal to obtain the UID information of the application and the ID information of the screen.

[0077] In an optional instance, the string information is obtained by invoking an information assembly method to assemble the identity information of the target application and the screen information of the target screen, where the identity information of the target application and the screen information of the target screen are obtained by the activity startup module in response to the startup operation of the target application on the target screen.

[0078] The information assembly method is the startActivityInner method, which assembles the UID information of the application and the ID information of the screen into a handle_hdmi=display:uid string for information transmission. The activity startup module is the ActivityStarter module, which obtains the UID information of the application and the ID information of the screen through the information acquisition interface of the ActivityStarter module after the application starts, and performs subsequent data processing operations.

[0079] S202: Check whether the identity information of the target application exists in a preset database; the preset database includes the identity information of at least one application and the screen information for starting each application.

[0080] The preset database stores the UID information of each started application and the ID information of the screen in the computer device. Whenever an application is started, the relevant information of the application is obtained and compared with the information of the started applications already stored in the preset database.

[0081] In the computer device, optionally, the preset database may include an operating system built-in database, a relational database management system, a non-relational database (Not Only SQL, NoSQL) database, an embedded database, a dedicated audio database, a device management system database, an overclocking database, and so on.

[0082] S203: If the identity information of the target application exists in the preset database and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, then the audio of the target application is switched from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playback.

[0083] Based on the data in the preset database, the processor detects whether the UID information of the target application exists. If the UID information of the target application exists, the processor continues to detect whether the ID information of the screen bound to the UID information matches the ID information of the current target screen. If the ID information of the screen bound to the UID information in the preset database is inconsistent with the ID information of the current target screen, it indicates that the audio of the current application is not on the same screen as the application and needs to be switched from one screen to another. For example, the application is initially started on screen A, but the user subsequently starts the same application on screen B. The processor will detect the change in screen information and needs to switch the audio playback position. After determining that the audio playback position needs to be switched, the processor performs the switching of the audio playback position through the relevant audio management service. The processor switches the audio from the audio player of the screen recorded in the preset database to the audio player of the target screen and records the ID information of the target screen in the preset database.

[0084] Optionally, before audio switching, the processor first needs to store the information of the application display interface and the audio information of the application. The information of the application display interface includes the content of the application display interface, etc., and the audio information includes information such as the content of the audio, the playback progress, and the volume level. These information are stored in the memory or a preset database. When the user launches the same application on the target screen, the processor will switch the audio playback position, switching the audio from the audio player on the screen recorded in the preset database to the audio player on the target screen. After the audio playback position switching is completed, the processor restores the content of the application display interface and the playback of the application audio on the target screen according to the stored information of the application display interface and the audio information of the application, realizing the continuity of the switching of the content of the application display interface and the application audio playback between different screens.

[0085] In an optional example, if the audio player on the target screen is playing the audio of other applications, obtain the audio playback permission of the target screen.

[0086] Adjust the audio playback of other applications according to the audio playback permission, and switch the audio of the target application from the audio player on the screen corresponding to the screen information including the launch of the target application in the preset database to the audio player on the target screen for playback; adjusting the audio playback of other applications includes reducing the volume level of the audio playback of other applications and pausing the audio playback of other applications.

[0087] When the target application switches to the target screen, if there are other applications on the target screen playing audio, the target application needs to request the audio playback permission, that is, the audio focus, on the target screen from the processor. The processor decides whether to grant the target application the audio focus according to certain rules and strategies. For example, if there is no other application playing audio currently, the processor usually grants the requesting application the audio focus immediately; if there is already another application playing audio, the processor will judge whether to transfer the focus to it according to factors such as the audio playback priority of the newly requested application. When the processor transfers the audio focus from other applications to the target application, it notifies the application to pause the audio playback or reduce the volume level of the audio playback according to the high or low audio playback priority of the application currently having the audio focus. When the application that loses the audio focus meets certain conditions again, such as the playback of the high-priority application that had the focus before ends or the user manually switches back to this application, etc., the processor restores the audio focus of this application so that it can continue to play audio.

[0088] Optionally, in the audio processing method proposed in this embodiment, a multi-audio focus management scheme is included, such that in a multi-screen heterogeneous display scenario, by setting multiple audio foci, the audio players of each screen can be independently managed, realizing the function of playing audio on multiple screens simultaneously. The multi-audio focus management scheme includes: the processor processes the requirements of multi-audio foci through a multi-audio focus policy management module, which can identify and distinguish the audio requests of different screens and allocate independent audio foci to each screen; modify the existing audio output device management logic to enable it to support the independent management and control of multiple audio output devices; when an application requests an audio focus on a certain screen, the multi-audio focus policy management module can decide whether to grant the audio focus according to the current focus state and the priority of the request; when applications on two screens simultaneously request audio foci, the audio foci of the applications can be allocated according to the preset audio policy and a reasonable conflict resolution mechanism.

[0089] Figure 3 FIG. 4 is a schematic diagram of a multi-audio focus policy management provided by an embodiment of the present application. Screen A and Screen B respectively have independent audio foci. After an application is opened, the UID information of the application is recorded using the Window Manager Service (WMS). The WMS is mainly responsible for managing all window-related affairs. The UID information is saved in the content provider of the application. The content provider is a mechanism for sharing data between different application components; then the application applies to play audio, and the processor allocates audio foci respectively according to the audio application lists of different screens, and performs operations such as callback of focus loss events or re-acquisition of focus according to the multi-screen focus policy. The audio foci of the two screens are independent of each other, and different applications can play audio simultaneously on the two screens.

[0090] Taking a computer device with two screens as an example, the implementation of the multi-audio focus management scheme is as follows:

[0091] When multiple applications on one of the two screens request audio foci, the multi-audio focus policy management module sets an audio focus for this screen separately, and allocates the audio foci to the applications according to the audio playback priorities of the multiple applications.

[0092] When different applications are playing audio on two screens respectively, and the audio playback priorities of the two applications are the same, the multi-audio focus strategy management module grants audio focus to the two applications respectively, enabling the audio players on the two screens to play audio simultaneously. When playing a video on Screen A of a dual-screen device and playing music on Screen B at the same time, with the same audio playback priorities for the video application and the music application, the multi-audio focus strategy management module can identify the audio requests from the two screens and grant audio focus respectively, achieving simultaneous audio playback on the two screens; when the audio playback priority of the application on Screen A is higher, the multi-audio focus strategy management module temporarily reduces the volume level of the audio of the application on Screen B or pauses the audio playback of that application according to the set priority and conflict resolution mechanism to ensure that the high-priority audio request can obtain focus. For example: When starting a navigation application on Screen A and playing music on the other screen. When the navigation application requests audio focus, the multi-audio focus strategy management module can temporarily reduce the volume level of the music or pause the playback according to the audio playback priority of the application, provide navigation prompts, and then resume the music playback. At the same time, the current audio focus status of the application is displayed on the screen that has obtained the audio playback focus to avoid audio playback confusion.

[0093] The independent audio focus management for different screens implemented by the multi-audio focus management solution solves problems such as single audio focus management, low utilization rate of audio players, and poor user experience in the multi-screen different display scenario. By means of multi-audio focus strategy management, independent audio output device management, and audio focus conflict resolution mechanism, etc., the audio focus of each screen is independently controlled, improving the usage rate of audio players on different screens and enhancing the user experience and work efficiency in the multi-screen different display scenario. A reasonable conflict resolution mechanism is designed to ensure the efficient operation of the multi-audio focus strategy, playing different audio contents simultaneously on multiple screens according to the preset audio strategy without interference from each other, increasing the flexibility and convenience of user operations.

[0094] In an optional instance, in response to the start operation of a target application on a target screen, if the remaining hardware resources are less than the hardware resources required to start the target application, then part of the hardware resources allocated to the application in the playing state is released to obtain the updated remaining hardware resources.

[0095] The updated remaining hardware resources are allocated to the target application to start the target application.

[0096] When the user triggers the start operation of the target application on the target screen, the processor will first perform a hardware resource detection of the computer device to evaluate the current usage of the hardware resources. If the processor finds that the hardware resources required to start the target application are insufficient, such as excessive memory occupancy or heavy CPU load, etc., in order to ensure that the target application can be started, the processor will increase the priority of the target application's hardware resource application and adjust the hardware resources used by the applications with a lower hardware resource application priority and in the playing state. After adjusting the hardware resources and ensuring that there are sufficient hardware resources, the processor will start the target application. To minimize the impact on the user, the processor's adjustment measures for the hardware resources occupied by other applications that are playing audio are temporary. After the target application starts and runs stably, the processor will monitor the usage of the hardware resources and restore the usage of the hardware resources of other applications before at an appropriate time. During the entire process of adjusting the hardware resources, the processor records relevant information in the log file, including the specific adjustment methods of the hardware resources of each application, the start time and status of the target application, etc., for subsequent problem diagnosis and performance optimization.

[0097] An application in the playing state refers to an application that is outputting audio content through relevant components such as an audio player. For example, a music player application is playing a song, a video player application is playing a video and the audio part is outputting normally, an audiobook application is reading the content, etc. These applications are all occupying hardware resources in real time to maintain the continuous playback of the audio, and at this time they belong to the applications in the playing state.

[0098] Optionally, adjusting the hardware resources used by the started applications includes CPU resources, memory resources, and audio device resources. The methods for adjusting the CPU resources used by the started applications include reducing the complexity of the audio processing algorithm. For example, switching the high-quality audio decoding algorithm to a lower-quality decoding algorithm; adjusting the sampling rate and bit depth of the audio playback. By reducing the sampling rate and bit depth of the audio, the application's usage of CPU resources can be reduced.

[0099] The methods for adjusting the memory resources used by the started applications include reducing the audio cache size. By reducing the cache area opened by the application in the memory, the application's usage of memory resources can be reduced; releasing non-critical memory resources related to audio playback. During the audio playback process, the audio application may occupy some additional memory to store non-critical elements such as audio cover pictures and lyric data. By releasing the memory resources of these non-critical elements, the application's usage of memory resources can be reduced.

[0100] The ways to adjust the audio device resources used by a started application include reducing the output quality parameters of the audio output device, such as reducing the number of its output channels or the dynamic range of the output audio, to reduce the audio output quality parameters of the audio output device and thus reduce the application's use of audio device resources; adjusting the buffering strategy of the audio output device, such as reducing the buffer depth or increasing the buffer update interval, to reduce the application's use of audio device resources.

[0101] In an optional example, record the audio switching information of the target application in a log file; the log file is used for fault detection of the audio switching operation of the target application; the audio switching information includes the time information of the audio switching operation of the target application, the identity information of the target application, the screen information of the target screen, and the screen information for starting the target application included in the preset database.

[0102] After the processor executes the audio switching task, it will store the relevant information in the log file for easy fault detection. The processor will record the specific time point of the audio switching operation of the target application, the UID information of the target application, the ID information of the target screen, and the ID information of the screen for starting the target application included in the preset database. Optionally, in addition to the above information, the content recorded by the processor also includes the state of the application audio before and after switching, including quality-related parameters such as audio coding format, bit rate, and sampling rate; the user's interaction behavior during the audio switching process, such as click, pause, play, etc. operations; possible delays, packet losses, or synchronization problems during the switching process. If any error or abnormality occurs during the audio switching process, the processor will take relevant countermeasures based on the information recorded in the log.

[0103] Optionally, during the audio switching process, if a delay is detected, the processor will ensure the continuity and real-time performance of audio playback through relevant mechanisms, including: analyzing historical audio data through a prediction algorithm to predict the trend of audio signal changes and adjusting the processing parameters of the audio signal in advance based on these prediction results; providing hardware acceleration through the audio codec to reduce the encoding and decoding time; optimizing cache management to reduce the cache time of audio data at the playback end; controlling the cache to reduce the waiting time of data in the cache; using an audio synchronization tool, such as audio editing software, to manually adjust the audio delay.

[0104] Optionally, when the target application is performing an audio switching operation, changes in the state of external audio devices such as headphones or Bluetooth speakers may interfere with or affect audio playback, and the processor needs to establish a timely detection mechanism for changes in the state of external devices.

[0105] For the plugging and unplugging of wired headphones, it is determined whether the headphones are inserted or removed by listening for specific hardware interrupt signals or periodically polling the connection status of the headphone interface. For example, in a computer operating system, when the headphone plug is inserted into or removed from the audio interface, a corresponding hardware interrupt is triggered, and the audio driver records this hardware interrupt to determine the device connection status of the headphones. For wireless devices such as Bluetooth speakers, the connection status of the device is monitored through the Bluetooth communication protocol stack. If the Bluetooth protocol stack fails to receive a response signal from the Bluetooth speaker within a certain period of time, or receives a disconnection request sent by the Bluetooth speaker, it is determined that the Bluetooth speaker is disconnected.

[0106] After detecting a change in the status of an external device, the processor adjusts the audio output path according to the device connection situation. If the audio was originally output through headphones and the headphones are unplugged, and if the Bluetooth speaker is in a connected and available state at this time, the processor will automatically switch the audio output route to the Bluetooth speaker; if there is no available Bluetooth speaker or other external audio device, the audio will be switched to the audio player of the screen connected to the computer for output. While adjusting the audio output path, it is also necessary to adapt the audio playback parameters. For example, different headphones and Bluetooth speakers have different audio characteristics, such as impedance, frequency response range, etc. When switching to a new device, the processor adjusts the gain of the audio signal according to the parameters of the device to ensure accurate audio output. For some devices with special functions, such as speakers that support surround sound effects, the processor will correspondingly adjust the audio channel configuration to adapt to the audio playback function of the device.

[0107] Figure 4 It is a schematic structural diagram of an audio processing device provided by an embodiment of the present application.

[0108] In an implementation manner of the audio processing device according to an embodiment of the present application, the audio processing device includes the following structure.

[0109] The acquisition unit 401, in response to the start operation of the target application on the target screen, the acquisition unit acquires the identity information of the target application and the screen information of the target screen;

[0110] The processing unit 402 is used to find out whether the identity information of the target application exists in the preset database; the preset database includes the identity information of at least one application and the screen information for starting each application;

[0111] If the identity information of the target application exists in the preset database, and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, then the processing unit 402 switches the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playback.

[0112] In one embodiment, in response to a startup operation of a target application on a target screen, the processing unit 402 obtains the identity information of the target application and the screen information of the target screen, including:

[0113] Receiving string information through an information transmission interface, where the string information is sent by invoking an information transmission method;

[0114] Parsing the string information to obtain the identity information of the target application and the screen information of the target screen.

[0115] In one embodiment, before switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing, the processing unit 402 is further configured to perform the following operations:

[0116] If the audio player of the target screen is playing the audio of another application, obtain the audio playback permission of the target screen;

[0117] Adjust the audio playback of the other application according to the audio playback permission, and switch the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing; adjusting the audio playback of the other application includes reducing the volume level of the audio playback of the other application and pausing the audio playback of the other application.

[0118] In one embodiment, the processing unit 402 is further configured to perform the following operations:

[0119] In response to a startup operation of a target application on a target screen, if the remaining hardware resources are less than the hardware resources required to start the target application, release some of the hardware resources allocated to the applications in the playing state to obtain updated remaining hardware resources;

[0120] Allocate the updated remaining hardware resources to the target application to start the target application.

[0121] In one embodiment, after switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing, the processing unit 402 is further configured to perform the following operations:

[0122] Record the audio switching information of the target application in a log file; the log file is used for fault detection of the audio switching operation of the target application; the audio switching information includes the time information of the audio switching operation of the target application, the identity information of the target application, the screen information of the target screen, and the screen information for starting the target application included in the preset database.

[0123] SeeFigure 5 , Figure 5 is a schematic structural diagram of a computer device provided by an embodiment of the present application. The computer device in the embodiment of the present application includes structures such as a power supply module, and includes a processor 501, an input device 502, an output device 503, and a memory 504. Data can be exchanged between the processor 501, the input device 502, the output device 503, and the memory 504, and the corresponding video processing method is implemented by the processor 501.

[0124] The processor 501 may be a central processing unit (CPU). The processor 501 may also be a combination of a CPU and a GPU.

[0125] The input device 502 may include a keyboard, a mouse, a touchpad, a scanner, and a touch screen.

[0126] The output device 503 may include a display, a speaker, headphones, a projector, and a printer.

[0127] The memory 504 may include volatile memory, such as random access memory (RAM); the memory 504 may also include non-volatile memory, such as flash memory, solid-state drive (SSD), etc.; the memory 504 may further include a combination of the above types of memory.

[0128] In one embodiment, the memory 504 is used to store program instructions. The processor 501 may call the program instructions to implement various methods involved in the above embodiments of the present application.

[0129] In the first possible implementation manner, in response to a start operation of a target application on a target screen, the processor 501 of the computer device calls program instructions stored in the memory 504 to obtain the identity information of the target application and the screen information of the target screen;

[0130] Check whether the identity information of the target application exists in a preset database; the preset database includes the identity information of at least one application and the screen information for starting each application;

[0131] If the identity information of the target application exists in the preset database, and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, then switch the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing.

[0132] In one embodiment, in response to a startup operation of a target application on a target screen, the processor 501 is further configured to obtain the identity information of the target application and the screen information of the target screen, including:

[0133] Receiving string information through an information transmission interface, where the string information is sent by invoking an information transmission method;

[0134] Parsing the string information to obtain the identity information of the target application and the screen information of the target screen.

[0135] In one embodiment, before switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing, the processor 501 is further configured to perform the following operations:

[0136] If the audio player of the target screen is playing the audio of another application, obtain the audio playback permission of the target screen;

[0137] Adjust the audio playback of the other application according to the audio playback permission, and switch the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing; adjusting the audio playback of the other application includes reducing the volume level of the audio playback of the other application and pausing the audio playback of the other application.

[0138] In one embodiment, the processor 501 is further configured to perform the following operations:

[0139] In response to a startup operation of a target application on a target screen, if the remaining hardware resources are less than the hardware resources required to start the target application, release some of the hardware resources allocated to the applications in the playing state to obtain updated remaining hardware resources;

[0140] Allocate the updated remaining hardware resources to the target application to start the target application.

[0141] In one embodiment, after switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing, the processor 501 is further configured to perform the following operations:

[0142] Record the audio switching information of the target application in a log file; the log file is used for fault detection of the audio switching operation of the target application; the audio switching information includes the time information of the audio switching operation of the target application, the identity information of the target application, the screen information of the target screen, and the screen information for starting the target application included in the preset database.

[0143] An embodiment of the present application further provides a computer-readable storage medium, which can be used to store the computer program used in the audio processing method in the embodiment. It includes a program designed to execute the above-mentioned switching of the application audio playback position.

[0144] The above-mentioned computer-readable storage medium includes, but is not limited to, flash memory, hard disk, and solid-state drive.

[0145] An embodiment of the present application further provides a computer program product. When the computer program product is run on a computer device, it can execute the method for switching the application audio playback position described above.

[0146] Those of ordinary skill in the art can realize that the modules and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be implemented by electronic hardware, or a combination of computer applications and electronic hardware. Whether these functions are executed in hardware or application mode depends on the specific application and design constraints of the technical method. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0147] In the above embodiments, it can be implemented in whole or in part by applications, hardware, firmware, or any combination thereof. When implemented using an application, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (such as a solid-state drive (SSD)).

[0148] The above is only the specific implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the embodiments of the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims.

Claims

1. An audio processing method, characterized in that, Including: In response to the start operation of the target application on the target screen, obtain the identity information of the target application and the screen information of the target screen; Check whether the identity information of the target application exists in the preset database; The preset database includes the identity information of at least one application and the screen information for starting each application; If the identity information of the target application exists in the preset database and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, then switch the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playback.

2. The method according to claim 1, wherein The obtaining the identity information of the target application and the screen information of the target screen in response to the start operation of the target application on the target screen includes: Receive string information through an information transmission interface, and the string information is sent by invoking an information transmission method; Parse the string information to obtain the identity information of the target application and the screen information of the target screen.

3. The method according to claim 2, characterized in that The string information is obtained by assembling the identity information of the target application and the screen information of the target screen by invoking an information assembly method, and the identity information of the target application and the screen information of the target screen are obtained by the activity start module in response to the start operation of the target application on the target screen.

4. The method according to claim 1, wherein Before switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playback, the method further includes: If the audio player of the target screen is playing the audio of another application, obtain the audio playback permission of the target screen; Adjust the audio playback of the other application according to the audio playback permission, and switch the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playback; the adjusting the audio playback of the other application includes reducing the volume level of the audio playback of the other application and pausing the audio playback of the other application.

5. The method according to claim 1, wherein The method further includes: In response to the start operation of the target application on the target screen, if the remaining hardware resources are less than the hardware resources required to start the target application, release some of the hardware resources allocated to the applications in the playback state to obtain the updated remaining hardware resources; Allocate the updated remaining hardware resources to the target application to start the target application.

6. The method according to claim 1, characterized in that After switching the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playback, it further includes: Record the audio switching information of the target application in a log file; the log file is used for fault detection of the audio switching operation of the target application; the audio switching information includes the time information of the audio switching operation of the target application, the identity information of the target application, the screen information of the target screen, and the screen information for starting the target application included in the preset database.

7. An audio processing device, characterized in that, Comprising: An obtaining unit, in response to a start operation of the target application on the target screen, the obtaining unit obtains the identity information of the target application and the screen information of the target screen; A processing unit, configured to search in a preset database for the existence of the identity information of the target application; The preset database includes the identity information of at least one application and the screen information for starting each application; If the identity information of the target application exists in the preset database, and the screen information for starting the target application included in the preset database is different from the screen information of the target screen, then the processing unit switches the audio of the target application from the audio player of the screen corresponding to the screen information for starting the target application included in the preset database to the audio player of the target screen for playing.

8. A computer device, characterized in that, Comprising a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions, when executed by a processor, cause the processor to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images