A low-delay media processing method under an Android system with a domestic ARM chip
By building a direct native hardware interface and JNI bridge layer in the Android system, the high-latency Android standard media service framework is bypassed, enabling low-latency streaming media processing. This solves the problem of excessive latency in traditional solutions, improving system efficiency and expanding the range of device applications.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU XINGYI ELECTRONICS TECH CO LTD
- Filing Date
- 2026-01-09
- Publication Date
- 2026-06-05
Smart Images
Figure CN122160420A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer multimedia processing technology, and in particular to a low-latency media processing method under an Android system equipped with a domestically produced ARM chip; specifically, it is a method for achieving high-performance, low-latency streaming media data output under an Android system equipped with a domestically produced ARM chip. Background Technology
[0002] With the rapid development of mobile internet and multimedia technologies, Android devices equipped with domestically produced ARM chips are increasingly widely used in fields such as real-time communication, cloud gaming, and professional audio processing. One of the core multimedia functions of these devices is to achieve efficient decoding, encoding, and output of streaming media data, and its performance directly affects the user experience in real-time interactive scenarios. In the Android system, the core requirement for streaming media processing is to quickly and accurately transmit media data from applications to the hardware to perform related operations in order to achieve smooth audio and video output.
[0003] Traditional Android streaming media processing solutions primarily rely on standard application programming interfaces (APIs) provided by the system, such as MediaPlayer, MediaCodec, and AudioTrack. Their working logic involves submitting media data to the MediaServer system service within the Android framework layer (which internally includes core modules like AudioFlinger). The media service first processes and mixes the data, then interacts with drivers in the Linux kernel through the Hardware Abstraction Layer (HAL), ultimately allowing the hardware to decode or output the media data. Specifically, the complete data path in the traditional solution is "Application -> Java Framework -> System Service (Inter-Process Communication) -> HAL -> Kernel Driver -> Hardware," with each layer working together to achieve media processing functionality.
[0004] However, traditional streaming media processing solutions have significant technical drawbacks: First, the data path is lengthy and multi-layered, with each layer incurring additional time consumption, including multiple memory copies, thread switching, and inter-process communication (IPC) operations. The cumulative latency of these steps typically exceeds 100 milliseconds. Second, the standard framework is designed with functional completeness and compatibility in mind, without optimization for low-latency scenarios, and cannot dynamically adjust the data processing flow according to the transmission requirements of real-time streaming media. Furthermore, cross-process communication and multiple data copies not only increase latency but also consume more system resources, indirectly affecting the overall operating efficiency of the device.
[0005] In other words, the standard media framework of the traditional Android system has too long a data path and too many intermediate processing links, resulting in excessively high latency in streaming media output. This makes it unable to meet the instant response requirements of professional-grade real-time applications such as real-time communication and cloud gaming, limiting the application expansion of Android devices equipped with domestic ARM chips in the field of high-performance multimedia, and also restricting the improvement of the experience of related real-time streaming media applications. Summary of the Invention
[0006] The main purpose of this application is to provide a low-latency media processing method for an Android system equipped with a domestically produced ARM chip. It aims to solve the problem of excessive latency caused by the excessively long data path and too many processing steps in the existing Android standard media framework, and to provide a method that can bypass high-latency system services and establish a direct and efficient data path from the application to the hardware, thereby achieving ultra-low latency output of streaming media data.
[0007] To achieve the above objectives, the first aspect of this invention provides a low-latency media processing method for an Android system equipped with a domestically produced ARM chip, comprising the following steps:
[0008] S1. Construct native layer hardware interface: In the native layer of Android application, integrate and link the underlying media library that can directly interact with the hardware driver, and write a wrapper module for calling the functions of the underlying media library.
[0009] S2. Establish a JNI bridging layer: Write bridging functions through the Java native interface JNI to establish a communication bridge between the Java layer and the native layer, so as to expose the functionality of the encapsulated module to the Java application layer;
[0010] S3. Provide a Java adapter interface and execute the call: Create an adapter class that provides methods corresponding to the Android standard media API functions; when the application calls the methods of the adapter class, the call is passed to the encapsulation module through the JNI bridge layer, and then the data processing task is directly submitted to the hardware driver of the kernel layer for execution through the underlying media library;
[0011] The method constructs a data path from the Java application layer through the JNI bridging layer, native layer encapsulation modules, low-level media libraries, and finally to the kernel layer hardware driver, thereby bypassing the standard media service framework of the Android system.
[0012] Preferably, step S1 includes:
[0013] S11. Integrate the underlying media library: Integrate the underlying media library and its header files provided by the chip manufacturer into the native layer development environment of the Android application; the native layer development environment is a Native C / C++ environment;
[0014] S12. Create a wrapper module: By writing a C++ wrapper class, the functions provided by the underlying media library are called, thereby providing an interface for hardware initialization, data submission and status query.
[0015] Preferably, step S2 includes:
[0016] S21. Write JNI bridge functions: Based on the Java native interface specification, write bridge functions for calling functions in the encapsulated module;
[0017] S22. Declare native methods: Declare the JNI bridging function as a native method of a Java-layer specific class, thereby exposing the hardware control capabilities of the native layer to the upper-layer Java code.
[0018] Preferably, step S3 includes:
[0019] S31. Create a Java adapter class: Create an adapter class in the Java application layer. This adapter class provides methods that correspond to the functionality of the Android standard MediaCodec class.
[0020] S32. Execute low-latency calls: When the application calls the public method of the adapter class, the call triggers the encapsulation module to call the underlying media library via the corresponding JNI bridge function, submitting the data processing task to the hardware driver in the kernel layer for execution.
[0021] Preferably, the methods provided by the adapter class that correspond to the functions of the Android standard MediaCodec class include configure, start, queueInputBuffer, and dequeueOutputBuffer. The calling logic of the methods is compatible with the Android standard MediaCodec class, and internally, the native layer encapsulation module is called through the JNI bridge layer.
[0022] Preferably, the function signature of the bridging function follows JNI naming rules, and the Java layer specific class is the RKMediaCodec class.
[0023] Preferably, the standard media service framework includes MediaServer and / or AudioFlinger services in the Android system.
[0024] The second aspect of the present invention provides an electronic device equipped with a domestically produced ARM chip and an Android system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the low-latency media processing method as described in the first aspect.
[0025] A third aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the low-latency media processing method as described in the first aspect.
[0026] A fourth aspect of the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the low-latency media processing method as described in the first aspect.
[0027] One or more technical solutions proposed in this application have at least the following technical effects:
[0028] 1. Achieve ultra-low latency streaming media output: This invention constructs a direct data path of "application → JNI → native layer → kernel driver → hardware", completely bypassing the high-latency system services, cross-process communication, and multiple memory copying links in the Android standard media framework. This significantly reduces the streaming media output latency from over 100 milliseconds in traditional solutions to below 20 milliseconds, precisely meeting the core requirements of professional-grade real-time applications such as real-time communication, cloud gaming, and professional audio processing.
[0029] 2. Significantly improve system operating efficiency: By avoiding unnecessary intermediate processing steps and multiple memory copy operations in the Android framework, this method significantly reduces CPU load and improves the overall operating efficiency of the system, especially suitable for resource-constrained mobile device scenarios.
[0030] 3. Provides a high-performance solution: This invention provides a new and effective technical path for the development of high-performance real-time applications on the Android platform, breaking through the inherent bottleneck of the Android system in real-time performance. It enables Android devices equipped with domestically produced ARM chips to achieve real-time media processing performance that was previously only achievable on desktop operating systems or dedicated hardware systems, greatly improving the application scope and market competitiveness of Android devices with domestically produced ARM chips. Attached Figure Description
[0031] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0032] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0033] Figure 1This is a schematic diagram of the system software layered architecture involved in the first embodiment of the low-latency media processing method of this application;
[0034] Figure 2 This is a flowchart illustrating the steps of the first embodiment of a low-latency media processing method for an Android system equipped with a domestically produced ARM chip, as described in this application.
[0035] Figure 3 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the low-latency media processing method under the Android system equipped with a domestically produced ARM chip in the second embodiment of this application.
[0036] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0037] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0038] It should be noted that in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0039] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0040] Domestic ARM chips / System-on-a-Chip (SoC): refers to integrated circuit chips designed by Chinese companies that integrate multiple core functions such as a central processing unit, a graphics processing unit, and a video processing unit, such as Rockchip's RK3568 and RK3588 chips.
[0041] Android: An open-source mobile operating system based on the Linux kernel.
[0042] Application Programming Interface (API): A set of predefined functions, protocols, and tools provided by a computer operating system or software library for application developers to call in order to implement specific functionalities.
[0043] Media Player instances (MediaPlayer), MediaCodec, and AudioTrack: These are standard application programming interfaces (APIs) provided by the Android system for media processing, used for high-level media playback, low-level audio and video encoding / decoding, and audio output, respectively.
[0044] MediaServer: A core background service process in the Android system, responsible for managing and processing all media-related tasks in the system, such as audio mixing and routing (through its internal AudioFlinger module), video decoding and synthesis, etc.
[0045] Hardware Abstraction Layer (HAL): A software layer located between the operating system kernel and upper-level software, providing upper-level software with a standard, hardware-independent interface to access hardware.
[0046] Java Native Interface (JNI): A programming framework that allows Java code running in the Java Virtual Machine (JVM) to interact with native applications and libraries written in other programming languages such as C / C++.
[0047] Native Layer: In the Android system, this refers to the program code layer written in native programming languages such as C / C++ that can directly interact with the Linux kernel.
[0048] Video Processing Unit (VPU): Also known as a hardware decoder or hardware encoder, it is a dedicated hardware circuit in a SoC specifically designed for efficient processing of compressed video data.
[0049] Media Processing Platform Library: refers to the underlying native library provided by chip manufacturers (such as Rockchip in this example) that allows developers to directly access the functions of their hardware (such as VPU), such as libmpp.so mentioned in the embodiments of this invention.
[0050] ioctl (input / output control): A system call provided by the Linux operating system kernel that allows user-space applications to send control commands to device drivers to perform specific operations on the hardware.
[0051] This application provides a solution that bypasses the Android standard media service framework by constructing a hardware interface that allows direct interaction between the native layer and the hardware driver. This eliminates latency issues caused by cross-process communication, multiple memory copies, and thread switching, simplifying the streaming media data transmission path. By establishing a JNI bridge layer and providing a Java adapter interface compatible with the Android standard media API, upper-layer applications can easily call low-latency processing capabilities without significant modifications, ensuring high efficiency and compatibility of data transmission. Furthermore, through direct interaction between the underlying media library and the kernel driver, the submission and response chain for data processing tasks is optimized, enabling precise control of streaming media output latency and further improving real-time processing performance. Ultimately, this solution addresses the problem of excessive latency in the traditional Android media framework, improves the real-time performance of streaming media output and system operating efficiency, and meets the high-performance requirements of professional-grade real-time applications such as real-time communication and cloud gaming.
[0052] Reference Figures 1 to 2 , Figure 1 This is a schematic diagram of the layered architecture of the system software involved in the low-latency media processing method of this application; Figure 1 This diagram illustrates the software layered architecture involved in the low-latency media processing method of this invention, as well as the data flow path that differs from the Android standard framework. The diagram mainly includes four layers: the Java application layer, the JNI bridging layer, the native C / C++ layer (i.e., the native layer), and the kernel layer.
[0053] The Java application layer includes modules responsible for specific business logic (e.g., VideoDecoder.java) and adapter interface classes implemented in the Java layer by this invention (e.g., RKMediaCodec.java).
[0054] The JNI bridging layer includes source files that implement JNI bridging functions (such as native-lib.cpp), which are responsible for connecting the Java layer and the native C / C++ layer.
[0055] The native C / C++ layer includes C++ wrapper modules (such as rk_mpp.cpp) for encapsulating calls to underlying libraries and underlying media libraries (such as libmpp.so) provided by chip manufacturers.
[0056] The kernel layer includes the operating system drivers and physical hardware units (such as Linux kernel drivers / VPU hardware) that ultimately perform the decoding task.
[0057] Figure 2 This is a flowchart illustrating the steps of the first embodiment of a low-latency media processing method for an Android system equipped with a domestically produced ARM chip, as described in this application.
[0058] The first embodiment of this invention takes the implementation of low-latency H.264 video hardware decoding on the Rockchip RK3588 chip as an example to describe the method steps of this invention in detail. The low-latency media processing method under an Android system equipped with a domestically produced ARM chip includes steps S1 to S3:
[0059] Step S1: Construct native layer hardware interface: In the native layer of Android application, integrate and link the underlying media library that can directly interact with the hardware driver, and write a wrapper module for calling the functions of the underlying media library.
[0060] Step S1 includes:
[0061] S11. Integrate the underlying media library: Integrate the underlying media library and its header files provided by the chip manufacturer into the native layer development environment of the Android application; the native layer development environment is a Native C / C++ environment;
[0062] S12. Create a wrapper module: By writing a C++ wrapper class, the functions provided by the underlying media library are called, thereby providing an interface for hardware initialization, data submission and status query.
[0063] In this embodiment, this step is implemented by integrating the media processing platform library (libmpp.so) and its header files (rk_mpi.h, etc.) provided by Rockchip into the jniLibs directory of the Android project. Then, a C++ wrapper class RKMpp (such as...) is written. Figure 1 The class rk_mpp.cpp is responsible for calling the underlying media processing interfaces provided by libmpp.so, such as functions like mpp_create(), mpp_init(), and mpp_decode_put_packet().
[0064] Step S2: Establish a JNI bridging layer: Write bridging functions through the Java native interface JNI to establish a communication bridge between the Java layer and the native layer, so as to expose the functionality of the encapsulated module to the Java application layer;
[0065] Step S2 includes:
[0066] S21. Write JNI bridge functions: Based on the Java Native Interface (JNI) specification, write bridge functions for calling functions in the encapsulated module;
[0067] S22. Declare native methods: Declare the JNI bridging function as a native method of a Java-layer specific class, thereby exposing the hardware control capabilities of the native layer to the upper-layer Java code.
[0068] In this embodiment, this step is implemented as follows:
[0069] In the `native-lib.cpp` file, write a series of JNI functions corresponding to the member functions of the `RKMpp` class and declare them as native methods of the `RKMediaCodec` Java class. Their function signatures will follow JNI naming rules. For example, the function `Java_com_example_mediacodecdemo_RKMediaCodec_native_queueInputBuffer` will receive parameters from the Java application layer and call the `queueInputBuffer` method of the `RKMpp` object. Here, `com_example_mediacodecdemo_RKMediaCodec` represents the full path to the Java class that calls this native method.
[0070] Step S3: Provide a Java adapter interface and execute the call: Create an adapter class that provides methods corresponding to the Android standard media API functions; when the application calls the methods of the adapter class, the call is passed to the encapsulation module through the JNI bridge layer, and then the data processing task is directly submitted to the kernel layer hardware driver for execution through the underlying media library;
[0071] Step S3 includes:
[0072] S31. Create a Java adapter class: Create an adapter class in the Java application layer. This adapter class provides methods that correspond to the functionality of the Android standard MediaCodec class.
[0073] S32. Execute low-latency calls: When the application calls the public method of the adapter class, the call triggers the encapsulation module to call the underlying media library via the corresponding JNI bridge function, and submits the data processing task to the kernel layer hardware / driver for execution through the ioctl system call.
[0074] In this embodiment, this step is implemented as follows:
[0075] The implementation method for this step is: create a Java class RKMediaCodec (e.g., Figure 1 As shown, this class provides an API highly similar to the Android standard MediaCodec. The application's business logic (such as VideoDecoder.java) dispatches decoding tasks by calling Java methods such as rkMediaCodec.queueInputBuffer(data).
[0076] This core technical solution bypasses high-latency system services in the Android standard media framework by constructing a low-latency data path with an ultra-short data path from application to hardware: "Application (Java -> JNI -> Native C / C++) -> Kernel Driver -> Hardware". The specific working principle is as follows:
[0077] First, the application's business logic (at the Java application layer) no longer directly calls the Android standard media framework's API, but instead calls methods in the Java adapter interface class (such as RKMediaCodec) provided by this invention that are similar to the standard API.
[0078] When a Java application calls a method of this adapter class, the call is passed to the native C / C++ layer through a pre-established JNI bridge (i.e., a bridging function in the JNI bridging layer).
[0079] At the native C / C++ layer, the JNI bridging function further calls functions provided by C++ wrapper classes (such as RKMpp). These C++ wrapper classes have directly integrated the underlying media libraries provided by the chip manufacturer (such as libmpp.so), so they can directly call the functions provided by the media library to perform operations such as hardware initialization and data submission.
[0080] Ultimately, the underlying media library interacts directly with the kernel-level drivers and hardware (Linux kernel drivers / VPU hardware) through system calls such as ioctl, submitting tasks to the physical hardware for execution.
[0081] In this way, the data path completely bypasses high-latency system services such as Android MediaServer, as well as the associated inter-process communication (IPC) and data copying processes. Due to the significant reduction in the data path and the elimination of intermediate processing steps, the processing latency from the application to the hardware is significantly reduced, thus solving the problem of excessive latency in existing technologies.
[0082] As described above, this solution bypasses high-latency stages such as the Android MediaServer system service and inter-process communication (IPC) by constructing a direct data path from the Java application layer to the JNI bridge layer, then to the native C / C++ layer, and finally to the kernel hardware driver. This eliminates the latency losses caused by multi-level links, multiple memory copies, and thread switching in traditional solutions, thereby achieving ultra-low latency streaming media output performance. Taking H.264 video hardware decoding on the Rockchip RK3588 chip as an example, the streaming media output latency of traditional Android media frameworks is usually greater than 100 milliseconds. However, this solution, relying on the direct interaction between the libmpp.so underlying media library and the VPU hardware driver, can reduce the latency to below 20 milliseconds, achieving an order-of-magnitude latency optimization. This fully meets the needs of professional-grade real-time applications with extremely high time response requirements, such as real-time communication, cloud gaming, and professional audio processing, and solves the core problem that traditional solutions cause poor user experience or even make such applications unusable due to excessive latency. Meanwhile, by streamlining the data processing chain, this approach avoids unnecessary intermediate processing steps, cross-process communication, and multiple memory copies inherent in the standard Android media framework. This significantly reduces the CPU load during media data processing and decreases memory bandwidth usage, thereby substantially improving overall system efficiency. Taking the implementation method of "C++ wrapper classes directly calling libmpp.so, and JNI bridge layer forwarding call instructions" as an example, the system does not need to allocate additional computing resources for multi-level media data forwarding and processing. The CPU can then dedicate more computing power to application business logic or other tasks. This not only improves the device's multitasking capabilities and system smoothness but also reduces unnecessary resource consumption. This helps improve the system stability and battery life of Android devices equipped with domestically produced ARM chips, making it particularly suitable for resource-constrained mobile device scenarios. Furthermore, this case provides a new and effective technical path for the development of high-performance real-time applications on the Android platform: On the one hand, through the design of Java adapter classes (such as RKMediaCodec) compatible with the Android standard MediaCodec API, upper-layer applications can easily call low-latency processing capabilities without significantly modifying their business logic, reducing development and adaptation costs; on the other hand, by leveraging the direct interaction between the native layer and hardware drivers, Android devices can achieve real-time media processing performance previously only achievable on desktop systems or dedicated hardware systems. This breaks through the inherent bottleneck of Android's real-time performance, enabling Android devices equipped with domestically produced ARM chips to expand beyond consumer-grade applications to professional fields such as professional audio workstations, high-precision industrial control interfaces, and medical image processing systems, greatly enhancing the application scope and market competitiveness of Android devices using domestically produced ARM chips.
[0083] Preferably, the adapter class provides methods corresponding to the Android standard MediaCodec class functionality, including configure, start, queueInputBuffer, and dequeueOutputBuffer. The calling logic of these methods is compatible with the Android standard MediaCodec class, and internally, it uses a JNI bridge layer to directly call the native layer encapsulated module. In this way, the adapter class is compatible with the core methods and calling logic of the Android standard MediaCodec, allowing developers to access low-latency capabilities simply by replacing the interface call object without refactoring the upper-layer business logic, significantly reducing application adaptation costs. The standardized calling logic avoids business anomalies caused by interface changes, and the internal design of directly calling the JNI bridge layer ensures the accuracy and operational stability of the underlying low-latency path. This further limitation of the methods corresponding to the MediaCodec class functionality achieves the goal of low-latency streaming media processing while remaining compatible with the existing Android media development ecosystem, allowing the low-latency solution to seamlessly integrate into the Android application system and facilitating technology deployment and promotion.
[0084] Preferably, the function signature of the bridging function follows JNI naming conventions, and the Java layer-specific class is the RKMediaCodec class. This ensures that the bridging function signature conforms to JNI naming conventions, guaranteeing a unique and accurate mapping between the Java application layer and the native layer calls, avoiding call confusion or errors caused by non-standard naming, and ensuring the stability of cross-layer communication. Specifying the RKMediaCodec class as the call carrier clarifies the exposure entry point for native layer functionality, simplifies the development and debugging process, forms a unified link with the adapter class logic, and improves the standardization and maintainability of the solution.
[0085] Specifically, the standard media service framework includes MediaServer and / or AudioFlinger services in the Android system.
[0086] In summary, this solution discloses a low-latency media processing method for Android systems equipped with domestically produced ARM chips, belonging to the field of computer multimedia processing technology. Addressing the problem that the traditional Android standard media framework suffers from excessively long data paths and numerous intermediate steps, resulting in streaming media output latency exceeding 100 milliseconds and failing to meet the demands of professional scenarios such as real-time communication and cloud gaming, this application proposes a solution. Its core is to construct a direct data path of "Java application layer → JNI bridging layer → native C / C++ layer → kernel layer hardware driver," bypassing high-latency system services such as Android MediaServer and / or AudioFlinger. Specifically, this is achieved through three steps: integrating the underlying media library and writing encapsulation modules, establishing a JNI bridging layer to expose native layer functions, and creating an adapter class compatible with the Android standard MediaCodec API for application calls. This solution reduces streaming media output latency to below 20 milliseconds, significantly reducing CPU load and memory bandwidth usage. Furthermore, upper-layer applications can adapt without significant modifications, breaking through the real-time performance bottleneck of the Android system, expanding the professional application scenarios of Android devices using domestically produced ARM chips, and improving system operating efficiency and market competitiveness.
[0087] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the water source heating system control method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.
[0088] A second embodiment of this application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the low-latency media processing method electronic device described in the above embodiments.
[0089] The following is for reference. Figure 3 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0090] like Figure 3 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. While electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.
[0091] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0092] The electronic device provided in this application adopts the low-latency media processing method electronic device in the above embodiments. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the low-latency media processing method electronic device provided in the above embodiments. Moreover, other technical features of the electronic device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.
[0093] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0094] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0095] The third embodiment of this application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the low-latency media processing method electronic device in the above embodiments.
[0096] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.
[0097] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.
[0098] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to perform the functions defined in the methods of the embodiments disclosed in this application.
[0099] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0101] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0102] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the electronic device of the low-latency media processing method described above. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the electronic device of the low-latency media processing method provided in the above embodiments, and will not be repeated here.
[0103] The fourth embodiment of this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the low-latency media processing method electronic device as described above.
[0104] Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the low-latency media processing method electronic device provided in the above embodiments, and will not be repeated here.
[0105] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. A low-latency media processing method for an Android system equipped with a domestically produced ARM chip, characterized in that, The method includes the following steps: S1. Construct native layer hardware interface: In the native layer of Android application, integrate and link the underlying media library that can directly interact with the hardware driver, and write a wrapper module for calling the functions of the underlying media library. S2. Establish a JNI bridging layer: Write bridging functions through the Java native interface JNI to establish a communication bridge between the Java layer and the native layer, so as to expose the functionality of the encapsulated module to the Java application layer; S3. Provide a Java adapter interface and execute the call: Create an adapter class that provides methods corresponding to the Android standard media API functions; when the application calls the methods of the adapter class, the call is passed to the encapsulation module through the JNI bridge layer, and then the data processing task is directly submitted to the hardware driver of the kernel layer for execution through the underlying media library; The method constructs a data path from the Java application layer through the JNI bridging layer, native layer encapsulation module, and underlying media library to the kernel layer hardware driver, thereby bypassing the standard media service framework of the Android system.
2. The low-latency media processing method according to claim 1, characterized in that, Step S1 includes: S11. Integrate the underlying media library: Integrate the underlying media library and its header files provided by the chip manufacturer into the native layer development environment of the Android application; the native layer development environment is a Native C / C++ environment; S12. Create a wrapper module: By writing a C++ wrapper class, the functions provided by the underlying media library are called, thereby providing an interface for hardware initialization, data submission and status query.
3. The low-latency media processing method according to claim 1 or 2, characterized in that, Step S2 includes: S21. Write JNI bridge functions: Based on the Java native interface specification, write bridge functions for calling functions in the encapsulated module; S22. Declare native methods: Declare the JNI bridging function as a native method of a Java-layer specific class, thereby exposing the hardware control capabilities of the native layer to the upper-layer Java code.
4. The low-latency media processing method according to claim 3, characterized in that, Step S3 includes: S31. Create a Java adapter class: Create an adapter class in the Java application layer. This adapter class provides methods that correspond to the functionality of the Android standard MediaCodec class. S32. Execute low-latency calls: When the application calls the public method of the adapter class, the call triggers the encapsulation module to call the underlying media library via the corresponding JNI bridge function, submitting the data processing task to the hardware driver in the kernel layer for execution.
5. The low-latency media processing method according to claim 4, characterized in that, The adapter class provides methods corresponding to the functions of the Android standard MediaCodec class, including configure, start, queueInputBuffer, and dequeueOutputBuffer. The calling logic of these methods is compatible with the Android standard MediaCodec class, and internally, the native layer encapsulation module is called through the JNI bridge layer.
6. The low-latency media processing method according to claim 3, characterized in that, The function signature of the bridging function follows JNI naming rules, and the Java layer specific class is the RKMediaCodec class.
7. The low-latency media processing method according to claim 1, characterized in that, The standard media service framework includes MediaServer and / or AudioFlinger services in the Android system.
8. An electronic device, characterized in that, The electronic device is equipped with a domestically produced ARM chip and an Android system, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the steps of the low-latency media processing method as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the steps of the low-latency media processing method as described in any one of claims 1 to 7.
10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the low-latency media processing method as described in any one of claims 1 to 7.