Comprehensive audio management system for mobile devices

Abstract

The comprehensive audio management system for mobile devices enables efficient audio output management on mobile devices by integrating a dynamic graphical user interface (GUI) and multi-point connectivity. Users can route audio from different applications, such as WhatsApp, to various hardware outputs (e.g., Bluetooth devices, phone speakers) through a drag-and-drop interface. The operating system (OS) facilitates this routing using APIs like AudioManager or AVAudioSession, while ensuring the synchronization of multiple audio streams across connected devices. It also supports independent volume control for each output device, with adjustments made via a GUI-based interface or hardware volume buttons. The system further incorporates a hardware abstraction layer (HAL) to manage audio streams seamlessly, optimizing playback and enhancing the user experience.

Description

DescriptionTitle of Invention : Comprehensive Audio Management System for Mobile DevicesTechnical Field

[0001] The present invention relates to audio management systems, specifically to a comprehensive system designed for mobile devices that enhances user control over audio output and volume management.Background Art

[0002] Mobile devices today serve as primary sources for audio playback, including music, calls, and notifications. However, current systems often lack intuitive controls for managing multiple audio outputs and levels. Users frequently face challenges when attempting to switch between different audio sources or adjust individual volumes, leading to a suboptimal user experience. Thus, there is a need for an improved audio management system that allows users to efficiently control audio playback and output across various applications.

[0003] There are several programs and methods available for managing multiple audio outputs on mobile devices:

[0004] Samsung Voice Assistant: Available on Samsung devices, this app allows users to customize the audio output for different applications. For example, you can play music through Bluetooth speakers while keeping notifications on the phone speaker.

[0005] Google Phone App: On Android devices, the Google Phone app enables users to switch audio devices directly within the app when answering calls, providing a seamless transition between different audio outputs.

[0006] Automate and JamesDSP (Rootless DSP): These apps are designed to manage audio output for various apps on Android devices. Automate allows users to set up complex automation for handling audio routing, while JamesDSP offers advanced audio processing capabilities to enhance audio output management.

[0007] These tools provide adaptable solutions for efficiently managing multiple audio outputs on mobile devices, ensuring an optimal audio experience.

[0008] There are some patents related to audio management systems designed for mobile devices that enhance user control over audio output and volume management:

[0009] US.Pat.App. No. US20140372109 describes a system that correlates audio output volume with received audio input, allowing adaptive volume control based on context, such as environmental noise or user settings. It integrates with mobile devices to manage volume for different outputs (e.g., headphones, speakers, or Bluetooth devices), providing seamless transitions and customizable presets.

[0010] US.Pat.App. No. US20160173049 discloses an advanced audio output device incorporates sensors to detect ambient sounds and deviations, such as noises or attempts to capture the user's attention (e.g., their name being called). A controller analyzes these sounds and adjusts audio settings accordingly, either lowering the volume or pausing playback in response to recognized events. The device uses stored audio preferences and recognition data, built from past interactions, to determine appropriate adjustments. It evaluates the probability of a user’s preference for specific settings based on the detected audio and applies changes when the likelihood exceeds a defined threshold, ensuring a responsive and personalized audio experience.Summary of Invention

[0011] The system described in the present document comprises a graphical display on the mobile device's screen, featuring a visual indicator that represents the current sound level. The indicator is filled to visually indicate the active sound level, utilizing graphical frameworks such as Android's Canvas or iOS’s Core Graphics.

[0012] An "Audio Output" button is provided to allow users to select their desired sound output method (e.g., phone speaker for WhatsApp, Bluetooth device for music). This functionality is facilitated through APIs such as AudioManager for Android and AVAudioSession for iOS.

[0013] The system supports individual volume adjustments for different audio sources. Users can access volume controls via icons that open a dedicated interface featuring:• Slider Controls for precise volume adjustments.• Gesture Recognition enabling users to swipe left or right to change volume levels.• Button-Based Control utilizing hardware volume buttons after selecting the volume icon.

[0014] The invention incorporates a haptic feedback feature that provides vibration alerts when applications connect or disconnect from audio outputs, enhancing user awareness of state changes. This feedback is managed through APIs such as Vibrator on Android and UHmpactFeedbackGenerator on iOS.

[0015] The system allows simultaneous playback of different audio files through multiple outputs if supported by device hardware and operating system capabilities. It manages synchronization and routing of multiple audio streams to ensure a coherent audio experience.

[0016] The architecture includes a System on Chip (SoC) where the CPU handles performance and audio file processing, converting compressed formats into raw data. A Digital Signal Processor (DSP) enhances audio quality through filtering and noise reduction, optimizing signals before conversion to analog.

[0017] The operating system plays a critical role in managing audio paths, determining output methods (Bluetooth, speaker, headphones), and facilitating user interactions through the graphical user interfaceTechnical Problem

[0018] One common technical challenge in mobile phones involves managing audio playback from different outputs. This issue arises when a user plays music through a car stereo or Bluetooth speaker while simultaneously trying to listen to a voice message on apps like Telegram or WhatsApp or watch a video on Instagram. In such cases, either all sounds are merged and played through a single output, or one sound is interrupted when another begins, preventing the user from effectively listening to both audio sources.

[0019] From a hardware perspective, modern devices are capable of supporting simultaneous audio output through multiple channels. This invention proposes an intelligent sound management system that enables users to select specific audio outputs for different applications. For instance, music from the Music app couldbe streamed via Bluetooth, while notifications or voice messages from WhatsApp are played through the phone's built-in speaker. This functionality enhances the user experience by offering seamless control over audio sources and outputs, allowing users to switch between sounds effortlessly.Solution to Problem

[0020] The present invention significantly enhances the audio experience on mobile devices by providing intuitive controls for managing sound levels and outputs alongside robust feedback mechanisms. By leveraging advanced audio APIs and incorporating both hardware and software solutions, it ensures high-quality audio playback tailored to user preferences. This system not only meets current user needs but also sets the foundation for future innovations in mobile audio management.

[0021] To utilize the product, a sound must already be playing on the mobile device. When audio is initiated in another application, a prompt is automatically displayed, offering two options: Replace the current sound, where the new audio replaces the existing playback and stops the previous sound, or Play sound from a different output, which allows the current audio (e.g., music on a Bluetooth speaker) to continue uninterrupted while the new audio plays from an alternate output (e.g., the phone speaker), enabling simultaneous playback. The interface, designed with a drag-and-drop feature, ensures effortless configuration of audio outputs, and additional enhancements such as bordered interface components or tooltips can be incorporated to improve clarity and usability.Advantageous Effects of Invention

[0022] One of the primary advantages of this product is its ability to allow users to play multiple audio streams simultaneously. For instance, users can listen to music from a Bluetooth speaker while also receiving voice messages on WhatsApp without having to stop the music. Unlike other devices where the voice message would interrupt the music, this product seamlessly integrates both audio streams, ensuring a continuous and enjoyable listening experience.Brief Description of Drawings

[0023] [Fig. 1 shows flowchart of the audio management process.

[0024] Fig. 2 shows Schematic diagram of audio management system. ]Description of Embodiments

[0025] The present invention details the operation of communication software or extensions, such as Bluetooth, in conjunction with software or mobile applications or platforms like WhatsApp. These platforms facilitate the exchange and upload of various audio and video files. For instance, as illustrated in the Fig.1 , a scenario is depicted where music is being played via Bluetooth device 1 . when the user interacts with the audio function of another application or social network (e.g., WhatsApp), the application's icon dynamically appears on the screen, surrounded by several available output options displayed along the screen edges. This system enhances user interaction by offering intuitive and seamless control over audio output routing.

[0026] If the users want to connect WhatsApp to the phone's speaker, they must hold their finger on the WhatsApp icon and drag it to the phone's speaker symbol to establish a connection. This action triggers a software-level interaction within the operating system (OS), specifically utilizing the Android or iOS Ul framework to recognize the drag-and-drop gesture. The OS then routes the audio output from WhatsApp to the phone's speaker using the AudioManager API (for Android) or AVFoundation framework (for iOS). As a result, the music or audio will start playing from the mobile phone's speaker, utilizing the built-in DAC (Digital-to- Analog Converter) to convert the digital audio signal to an analog one that can be played through the speaker.

[0027] If the user wants to connect to Bluetooth 1 , the device's Bluetooth stack (implemented in the OS) will handle the connection switch. This involves stopping the current audio stream playing through Bluetooth 1 by sending a disconnect command via A2DP (Advanced Audio Distribution Profile) and re-establishing a new audio stream from WhatsApp. This change in playback is managed by the OS’s Bluetooth service, which ensures the audio stream is routed through the connected Bluetooth device (Bluetooth 1 ). The A2DP profile is crucial here as it allows high-quality audio streaming over Bluetooth.

[0028] When connecting to Bluetooth 2, the OS must manage multiple Bluetooth connections. Modern mobile OSs support multi-point connectivity, enabling simultaneous audio streams to multiple Bluetooth devices. The AudioManager (Android) or CoreAudio (iOS) framework will handle these connections, ensuringboth Bluetooth 1 and Bluetooth 2 receive the audio stream. However, this requires the hardware Bluetooth chipset to support multi-point connections, and the software stack must be capable of managing these simultaneous connections.

[0029] In situations where different audio files are played through multiple output devices, the OS utilizes the AudioManager API to assign different audio streams to different hardware outputs. Each output device (speakers, Bluetooth 1 , Bluetooth 2) is treated as a separate audio endpoint, allowing for discrete control over each audio stream. The OS manages this via a combination of hardware abstraction layers (HAL) and audio policies defined within the OS to ensure proper routing and synchronization of the audio streams.

[0030] Overall, the process involves intricate interactions between the software stack (OS, audio framework, Bluetooth stack) and the hardware components (speaker, Bluetooth chipsets, DACs) to ensure seamless audio playback and connectivity.

[0031] In one embodiment of this invention, a display icon is designed on the mobile screen, half of which is filled to indicate the current sound level, while the other half remains empty. This visual representation is created using the device’s graphical user interface (GUI), leveraging frameworks like Android's Canvas or iOS’s Core Graphics. Beneath this rectangle, there is an "Audio Output" option, implemented as a button element within the GUI. When the user clicks this option, it triggers a function in the software that allows the user to specify the sound output. For example, selecting this option might use the AudioManager API (on Android) or AVAudioSession (on iOS) to route WhatsApp sound to the phone’s speaker while directing music playback to a connected Bluetooth device.

[0032] The user can individually adjust the volume of each audio source by interacting with icons representing WhatsApp and music. Clicking on these icons initiates an event listener that opens a volume control interface. This interface employs either slider controls or gesture recognition to allow the user to increase or decrease the volume. The adjustments are managed by APIs like AudioManager (for Android) or AVAudioSession (for iOS) which handle the audio output configuration.

[0033] This invention describes two methods for controlling volume:

[0034] Gesture-Based Control: By clicking on the volume icon, the user can swipe left or right to decrease or increase the volume. This is managed by the touch event handling mechanism provided by the mobile OS, capturing swipe gestures and translating them into volume adjustments through the relevant audio APIs.

[0035] Button-Based Control: After clicking on the volume icon, the user can use the hardware volume buttons of the phone to adjust the sound output. The software intercepts these hardware button events and applies the volume changes accordingly, again using the underlying audio framework APIs.

[0036] Furthermore, this invention encompasses features for connecting and disconnecting applications to different audio outputs. When an application connects to an audio output or disconnects, the phone provides haptic feedback through a small vibration. This haptic feedback is controlled by the device’s vibration motor, and the software employs APIs such as Vibrator (Android) or UHmpactFeedbackGenerator (iOS) to trigger these vibrations. This feedback mechanism ensures the user is informed that the connection has been successfully established or terminated.

[0037] In addition to the core functionality, the invention allows for extended capabilities, such as playing different audio files through multiple output hardware simultaneously, if supported by the device's hardware and the OS's audio routing capabilities. This involves managing multiple audio streams and ensuring they are correctly routed and synchronized, leveraging advanced features of the audio APIs and the hardware’s multi-channel audio capabilities.

[0038] The invention pertains to an advanced audio management system integrated into a mobile device, encompassing hardware and software components to optimize audio processing, routing, and playback. The System on Chip (SoC) - CPU serves as the main processing unit, managing device performance and handling audio files by converting them from compressed digital formats to raw digital data. The Digital Signal Processor (DSP) enhances audio quality through filtering, noise reduction, and other enhancements, optimizing digital signals before they are converted to analog. The Operating System (OS) is responsible for managing and controlling audio paths, deciding the output path (Bluetooth, speaker, headphones), and facilitating the user interface.

[0039] The Audio Codec Chip converts digital signals to analog and vice versa, playing a crucial role in digital audio systems. The Digital-to-Analog Converter (DAC) further processes these digital signals into analog signals for playback. The Amplifier boosts the analog signals post-DAC, making them suitable for output through speakers or headphones. The Bluetooth Controller manages the transmission of digital audio signals to Bluetooth devices, while the Bluetooth Codec compresses these signals for efficient transmission, utilizing codecs such as SBC, aptX, and LDAC for varying quality levels.

[0040] The invention also includes a phone speaker that converts analog audio signals into audible sound for phone calls or media playback. A headphone jack transmits analog signals to wired headphones, facilitating audio playback. An earpiece speaker, designed for telephone conversations, plays amplified analog audio, ensuring clear communication. This comprehensive audio management system ensures high-quality audio processing and playback across various output devices, integrating sophisticated hardware and software mechanisms to deliver an optimal auditory experience.

[0041] On other embodiments of invention some alternatives can be substituted or added to system:

[0042] Advanced visual indicators for sound levels can be implemented using frameworks like OpenGL ES for smooth animations and adaptive elements such as dynamic waveforms or spectrograms that respond to audio intensity. These enhancements improve the visual experience and interactivity for users across different platforms.

[0043] The system could integrate universal audio routing with tools like Flutter or React Native, ensuring broader compatibility. Additional features such as voice commands, auto-detection of optimal outputs, and context-aware recommendations improve user convenience and streamline audio management.

[0044] The system can use neural networks for sound separation, enabling seamless management of multiple audio streams. Features like multi-zone audio allow specific outputs for designated areas, providing a highly tailored listening experience.

[0045] Deep integration with audio frameworks like WASAPI or PulseAudio expands platform compatibility, while developer APIs enable third-party apps to interact with the system. These features promote a robust ecosystem for audio management.Industrial Applicability

[0046] The comprehensive audio management system for mobile devices has broad industrial applicability across several sectors:

[0047] 1 . Consumer Electronics: Enhances audio capabilities in smartphones and tablets, improving user experience.

[0048] 2. Telecommunications: Provides better audio control in mobile applications, increasing customer contentment.

[0049] 3. Entertainment: Offers improved audio management for streaming services and media applications.

[0050] 4. Gaming: Allows customization of audio settings for enhanced gaming experiences.

[0051] 5. Healthcare: Improves audio clarity in medical devices for effective communication.

[0052] 6. Automotive: Facilitates seamless audio management in vehicles for safety.

[0053] 7. Smart Home Devices: Enables cohesive audio control across various smart home systems.

[0054] 8. Education: Enhances audio management in e-learning platforms for clearer communication.

Claims

1. 1 . A comprehensive audio management system for mobile devices comprising: a graphical user interface configured to display application icons dynamically when an audio function is activated; a drag-and-drop mechanism, enabling users to select and route audio output from an application to a desired hardware output; an operating system (OS) configured to interpret user gestures and manage audio routing using an audio framework, wherein the OS uses APIs such as AudioManager (for Android) or AVAudioSession (for iOS) to facilitate audio output transitions; a multi-point connectivity framework configured to support concurrent audio streams to multiple Bluetooth devices; an audio policy management module within the OS to assign discrete audio streams to multiple hardware endpoints, ensuring proper synchronization and playback; a set of hardware components, comprising a Digital-to-Analog Converter (DAC) and Bluetooth stack, for processing and transmitting audio signals to selected output devices; a GUI-based volume control interface for adjusting audio levels independently for each connected output device; and a hardware abstraction layer (HAL) facilitating seamless routing of multiple audio streamsutilizing a System on Chip architecture and Digital Signal Processor.

2. The system of claim 1 , wherein the drag-and-drop mechanism is further configured to provide visual feedback, including highlighting available hardware outputs upon user interaction.

3. The system of claim 1 , wherein the OS provides haptic feedback via APIs such as Vibrator (Android) or UHmpactFeedbackGenerator (iOS) when an audio connection is established or terminated.

4. The system of claim 1 , wherein the Bluetooth stack utilizes the Advanced Audio Distribution Profile (A2DP) to manage high-quality audio streaming to Bluetooth devices.

5. The system of claim 1 , wherein the audio policy management module leverages multi-channel audio capabilities of the hardware to maintain distinct audio outputs for each connected device.

6. The system of claim 1 , wherein the GUI-based volume control interface supports gesture-based control, enabling users to adjust volume by swiping left or right on the screen.

7. The system of claim 1 , further comprising hardware volume buttons integrated with the audio framework, allowing manual adjustment of audio levels for specific outputs.

8. The system of claim 1 , wherein the simultaneous audio output is optimized through compression and transmission protocols, such as SBC, aptX, or LDAC, depending on the connected hardware’s capabilities.

9. The system of claim 1 , wherein a visual sound level indicator is implemented on the GUI, displaying current sound levels with interactive features for adjusting audio outputs.

Images