An HDMI audio acquisition method based on AUDIOTRACK

Abstract

This application provides an HDMI audio acquisition method based on AUDIOTRACK. The method includes: obtaining a Java Virtual Machine instance in the Java native interface initialization function of the native library, obtaining a reference to the target callback method in the Java layer based on the Java Virtual Machine instance, responding to the start of audio acquisition, calling the target audio acquisition method in the Native layer to initialize the HDMI capture card and configure the corresponding acquisition parameters, acquiring the corresponding audio data from the HDMI capture card, calling the set callback function in the Native layer, and transmitting the audio data in real time to the target player of the Android system running in the Java layer based on the callback function, receiving the audio data through the target player of the Android system, and outputting the audio data to the audio device after resampling and clock synchronization processing by the target player.

Description

Technical Field

[0001] This application relates to the field of embedded audio and video processing technology, and more specifically, to an HDMI audio acquisition method based on AUDIOTRACK. Background Technology

[0002] The core of HDMI audio acquisition technology lies in the lossless or high-quality separation, extraction, and digital embedding of audio data streams from HDMI signals.

[0003] Currently, existing HDMI audio acquisition systems typically employ a native audio playback solution, which involves directly acquiring and playing audio data from hardware devices via underlying audio libraries such as tinyalsa.

[0004] However, in existing native audio playback solutions, the HDMI capture card and the audio playback device use different hardware clock sources. After prolonged operation, clock deviations gradually accumulate, leading to problems such as noise and stuttering in audio playback. Even though there are existing solutions that use software algorithms for clock synchronization, they are computationally complex, consume a lot of CPU, and produce unsatisfactory results. Furthermore, native audio playback solutions require handling sampling rate conversion and clock synchronization themselves, which is complex and prone to errors. In addition, system stability and compatibility are also relatively poor. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide an HDMI audio acquisition method based on AUDIOTRACK. This method transmits audio data acquired at the Native layer to the Android system's AudioTrack for playback in real time, utilizing system-level resampling and clock synchronization capabilities. It eliminates the need for manual handling of sample rate conversion and clock synchronization, thus solving problems such as noise and stuttering in HDMI audio acquisition. The method is simpler to implement and less prone to errors. Furthermore, it addresses the need to restart the device after prolonged operation and the inconsistency in playback quality caused by differences in clock characteristics across different hardware platforms, improving system stability and compatibility.

[0006] In a first aspect, embodiments of this application provide an HDMI audio acquisition method based on AUDIOTRACK, the method comprising: In the Java native interface initialization function of the local library, a Java Virtual Machine instance is obtained, and a reference to the target callback method of the Java layer is obtained based on the Java Virtual Machine instance; wherein, the target callback method represents the callback method corresponding to the core of the Android system used to play raw audio data; In response to the start of audio acquisition, the target audio acquisition method is called in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters, and the corresponding audio data is acquired from the HDMI acquisition card. In the Native layer, a callback function is set, and based on the callback function, the audio data is transmitted in real time to the target player of the Android system running in the Java layer. The audio data is received by the target player of the Android system, and the target player performs resampling and clock synchronization processing on the audio data before outputting it to the audio device.

[0007] In one possible implementation, obtaining a reference to the Java layer target callback method based on the Java Virtual Machine instance includes: Based on the Java Virtual Machine instance, obtain a class reference representing the target class in the Java layer; Based on the class reference, obtain the target callback method in the target class.

[0008] In one possible implementation, the step of calling and setting a callback function at the Native layer, and then transmitting the audio data in real time to the target player running on the Android system in the Java layer based on the callback function, includes: In response to the HDMI capture card enabling the target audio data loading mode, a setup callback function is called in the Native layer, and the initialization of the Native playback device is skipped; The audio data is continuously read from the HDMI capture card in the Native layer, and the audio data is passed to the Java layer based on the callback function; Create a Java array, copy the audio data from the Native layer into the Java array, and then pass the audio data to the target player.

[0009] In one possible implementation, the step of continuously reading the audio data from the HDMI capture card at the Native layer and passing the audio data to the Java layer based on the callback function includes: The audio data is continuously read from the HDMI capture card at the Native layer, and the callback function is checked after each successful read. If the callback function is set, then the callback function is called to pass the audio data to the Java layer.

[0010] In one possible implementation, the step of creating a Java array, copying the audio data from the Native layer into the Java array, and passing the audio data to the target player includes: In the event-driven function, obtain the JNI environment and create a Java array based on the JNI environment; The audio data from the Native layer is copied into the Java array, and a static method in the Java layer is called to pass the audio data to the target player.

[0011] In one possible implementation, the multi-dimensional processing of the audio data includes: The audio data is written to the corresponding target buffer; wherein the target buffer is at least four times the size of the minimum buffer. The audio data is processed in multiple dimensions based on the target buffer.

[0012] In one possible implementation, the target audio data loading mode corresponds to a first playback mode, the playback mode further includes a second playback mode, and the method further includes: The mode flag for the playback mode is set via a configuration method and takes effect upon the next startup; During system operation, the corresponding playback mode is selected based on the mode flag to switch to the corresponding playback mode.

[0013] Secondly, embodiments of this application also provide an HDMI audio acquisition device based on AUDIOTRACK, the device comprising: The acquisition module is used to acquire a Java Virtual Machine instance in the Java native interface initialization function of the local library, and to acquire a reference to the target callback method of the Java layer based on the Java Virtual Machine instance; wherein, the target callback method represents the callback method corresponding to the core of the Android system used to play raw audio data; The acquisition module is used to respond to the start of audio acquisition by calling the target audio acquisition method in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters, and then acquiring the corresponding audio data from the HDMI acquisition card. The callback module is used to call and set a callback function in the Native layer, and then transmit the audio data to the target player of the Android system running in the Java layer in real time based on the callback function. The processing module is used to receive the audio data through the target player of the Android system, and the target player performs resampling and clock synchronization processing on the audio data before outputting it to the audio device.

[0014] In one possible implementation, the acquisition module is specifically used for: Based on the Java Virtual Machine instance, obtain a class reference representing the target class in the Java layer; Based on the class reference, obtain the target callback method in the target class.

[0015] In one possible implementation, the callback module is specifically used for: In response to the HDMI capture card enabling the target audio data loading mode, a setup callback function is called in the Native layer, and the initialization of the Native playback device is skipped; The audio data is continuously read from the HDMI capture card in the Native layer, and the audio data is passed to the Java layer based on the callback function; Create a Java array, copy the audio data from the Native layer into the Java array, and then pass the audio data to the target player.

[0016] In one possible implementation, the callback module is specifically used for: The audio data is continuously read from the HDMI capture card at the Native layer, and the callback function is checked after each successful read. If the callback function is set, then the callback function is called to pass the audio data to the Java layer.

[0017] In one possible implementation, the callback module is specifically used for: In the event-driven function, obtain the JNI environment and create a Java array based on the JNI environment; The audio data from the Native layer is copied into the Java array, and a static method in the Java layer is called to pass the audio data to the target player.

[0018] In one possible implementation, the processing module is specifically used for: The audio data is written to the corresponding target buffer; wherein the target buffer is at least four times the size of the minimum buffer. The audio data is processed in multiple dimensions based on the target buffer.

[0019] In one possible implementation, the target audio data loading mode corresponds to a first playback mode, and the playback mode further includes a second playback mode; the device further includes: The settings module is used to set the mode flag of the playback mode through a configuration method, and the setting will take effect on the next startup. The switching module is used to select the corresponding playback mode based on the mode flag during system operation, so as to switch to the corresponding playback mode.

[0020] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the HDMI audio acquisition method as described in any of the first aspects.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the HDMI audio acquisition method described in any one of the first aspects.

[0022] This application provides an HDMI audio acquisition method based on AUDIOTRACK. In the Java native interface initialization function of the local library, a Java Virtual Machine instance is obtained, and a reference to the target callback method in the Java layer is obtained based on the Java Virtual Machine instance. In response to starting audio acquisition, the target audio acquisition method is called in the Native layer to initialize the HDMI capture card and configure the corresponding acquisition parameters. The corresponding audio data is acquired from the HDMI capture card. A callback function is called in the Native layer, and the audio data is transmitted in real-time to the target player running on the Android system in the Java layer. The target player in the Android system receives the audio data, performs resampling and clock synchronization processing, and then outputs the audio data to the audio device. This application, by transmitting the audio data acquired in the Native layer to the Android system's AudioTrack playback in real-time, utilizes system-level resampling and clock synchronization capabilities, and eliminates the need for manual handling of sampling rate conversion and clock synchronization. This solves problems such as noise and stuttering in HDMI audio acquisition, making implementation simpler and less prone to errors. Furthermore, it also solves the problems of needing to restart the device after long-term operation and the inconsistent playback effects caused by differences in clock characteristics between different hardware platforms, improving system stability and compatibility.

[0023] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of an HDMI audio acquisition method provided according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an HDMI audio acquisition device provided according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0027] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0029] Considering the background technology of HDMI audio acquisition, the core lies in the lossless or high-quality separation, extraction and digital embedding of audio data streams from HDMI signals.

[0030] Currently, existing HDMI audio acquisition systems typically employ a native audio playback solution, which involves directly acquiring and playing audio data from hardware devices via underlying audio libraries such as tinyalsa.

[0031] However, in existing native audio playback solutions, the HDMI capture card and the audio playback device use different hardware clock sources. After prolonged operation, clock deviations gradually accumulate, leading to problems such as noise and stuttering in audio playback. Even though there are existing solutions that use software algorithms for clock synchronization, they are computationally complex, consume a lot of CPU, and produce unsatisfactory results. Furthermore, native audio playback solutions require handling sampling rate conversion and clock synchronization themselves, which is complex and prone to errors. In addition, system stability and compatibility are also relatively poor.

[0032] To address this issue, this application provides an HDMI audio acquisition method based on AUDIOTRACK. By transmitting audio data acquired at the native layer to the Android system's AudioTrack for playback in real time, it leverages system-level resampling and clock synchronization capabilities. Furthermore, it eliminates the need for manual handling of sample rate conversion and clock synchronization, resolving issues such as noise and stuttering in HDMI audio acquisition. This method is simpler to implement and less prone to errors. Additionally, it resolves the need to restart the device after prolonged operation and addresses inconsistencies in playback quality caused by differences in clock characteristics across different hardware platforms, thus improving system stability and compatibility.

[0033] Figure 1 This is a flowchart of an HDMI audio acquisition method provided according to an embodiment of this application. Figure 1 As shown, the HDMI audio acquisition method of this application embodiment may specifically include: S101. In the Java native interface initialization function of the local library, obtain the Java Virtual Machine instance and obtain the reference to the target callback method of the Java layer based on the Java Virtual Machine instance.

[0034] S102. In response to starting audio acquisition, the target audio acquisition method is called in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters, and the corresponding audio data is acquired from the HDMI acquisition card.

[0035] S103. Call the callback function in the Native layer and transmit the audio data to the target player of the Android system running in the Java layer in real time based on the callback function.

[0036] S104. Audio data is received through the target player of the Android system, and the target player performs resampling and clock synchronization processing on the audio data before outputting it to the audio device.

[0037] The aforementioned HDMI audio acquisition method utilizes system-level resampling and clock synchronization capabilities by transmitting audio data acquired at the Native layer to the Android system's AudioTrack playback in real time. This eliminates the need for manual sampling rate conversion and clock synchronization, resolving issues such as noise and stuttering in HDMI audio acquisition. The method is simpler to implement and less prone to errors. Furthermore, it addresses the need to restart the device after prolonged operation and the inconsistencies in playback quality caused by differences in clock characteristics across different hardware platforms, thus improving system stability and compatibility.

[0038] The exemplary steps described above in the embodiments of this application are illustrated below with specific examples: S101, in the Java native interface initialization function of the native library, a Java Virtual Machine instance is obtained, and a reference to the target callback method in the Java layer is obtained based on the Java Virtual Machine instance. In this embodiment, the target callback method represents the callback method (i.e., the AudioTrack callback method) corresponding to the core of the Android system used to play raw audio data. In the Java native interface initialization function of the native library, a reference to the target callback method in the Java layer is obtained based on the Java Virtual Machine instance. Specifically, for example, during the initialization phase, the JavaVM instance is obtained in the JNI_OnLoad function, and a reference to the AudioTrack callback method in the Java layer is obtained through FindClass and GetStaticMethodID, and the method ID is cached for later use.

[0039] Optionally, when obtaining a reference to the target callback method in the Java layer based on the Java Virtual Machine instance, a class reference representing the target class in the Java layer is obtained based on the Java Virtual Machine instance; the target callback method in the target class is then obtained based on the class reference. Additionally, the method identifier corresponding to the target callback method is obtained and globally cached for subsequent callback invocations.

[0040] S102, in response to starting audio acquisition, calls the target audio acquisition method in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters, and acquires the corresponding audio data from the HDMI acquisition card.

[0041] In this embodiment, the acquisition parameters include at least sampling rate, bit depth, and channels. When audio acquisition is started, the target audio acquisition method is called in the Native layer to initialize the HDMI capture card and configure the acquisition parameters. Then, audio data is acquired through the HDMI capture card. The audio data can be PCM audio data. For example, when audio acquisition is started, the Native layer calls the CaptureMgr::dealaudio() method to initialize the audio acquisition device (i.e., the HDMI capture card), configures the sampling parameters (e.g., sampling rate 48000Hz, bit depth 16bit, channels 2), and acquires audio data from the device.

[0042] S103 calls a callback function in the Native layer and transmits audio data in real time to the target player running in the Android system in the Java layer based on the callback function.

[0043] In this embodiment, the target player can be an AudioTrack player, which refers to an Android AudioTrack player running in the Java layer. The native layer calls a callback function to transmit audio data to the Android system's AudioTrack player running in the Java layer in real time. Specifically, for example, the native layer calls CTriAudio::setAudioTrackCallback() to set a callback function, transmitting the audio data collected by the native layer to the Android system's AudioTrack in real time.

[0044] In some implementations, in response to the HDMI capture card enabling the target audio data loading mode, a setup callback function is called in the Native layer, and the initialization of the Native playback device is skipped. Audio data is continuously read from the HDMI capture card in the Native layer and passed to the Java layer based on the callback function. A Java array is created, the audio data from the Native layer is copied into the Java array, and the audio data is then passed to the target player. Here, the target audio data loading mode corresponds to the AudioTrack mode.

[0045] Optionally, the Native layer continuously reads audio data from the HDMI capture card, and after each successful read, checks whether a callback function has been set; if the callback function has been set, the callback function is called to pass the audio data to the Java layer.

[0046] Optionally, the JNI environment can be obtained in the event-driven function, and a Java array can be created based on the JNI environment; the audio data from the Native layer can be copied into the Java array, and a static method of the Java layer can be called to pass the audio data to the target player.

[0047] Specifically, after configuring the sampling parameters, the system checks whether the HDMI capture card has enabled the target audio data loading mode, i.e., AudioTrack mode. If AudioTrack mode is enabled, the Native layer calls CTriAudio::setAudioTrackCallback() to set the callback function and skips the initialization of the Native playback device in the init() method to avoid resource conflicts. The Native layer continuously reads audio data from the HDMI capture card in the CTriAudio::work_block() method. After each successful read, if the AudioTrack callback function is detected as set, the callback function is called to pass the audio data to the Java layer.

[0048] Continuing, in the audioDataCallback() function, the JNI environment is first obtained through JavaVM::GetEnv() or AttachCurrentThread() to ensure thread safety; then a Java byte array is created, and the audio data from the native layer is copied into the array; finally, the static method onAudioData() in the Java layer is called to pass the data to the AudioTrack player.

[0049] S104 receives audio data through the target player of the Android system, and the target player performs resampling and clock synchronization processing on the audio data before outputting it to the audio device.

[0050] In this embodiment of the application, audio data is received by the target player of the Android system, and the target player performs resampling and clock synchronization processing on the audio data before outputting it to the audio device.

[0051] Optionally, when performing multi-dimensional processing on the audio data, the audio data is written to a corresponding target buffer; multi-dimensional processing is then performed on the audio data based on the target buffer. The size of the target buffer is at least four times the size of the minimum buffer.

[0052] It should be noted that the minimum buffer size is obtained; the size of the AudioTrack target buffer is determined based on the minimum buffer size; wherein, the AudioTrack is at least 4 times the minimum buffer size, that is, greater than or equal to 4 times; for example, the target buffer size is 4 times the minimum value to provide sufficient fault tolerance.

[0053] Specifically, the Java layer AudioTrack player receives audio data and writes it to the AudioTrack buffer using the AudioTrack.write() method. The Android system automatically performs resampling, clock synchronization, and mixing processing before finally outputting the data to the audio device.

[0054] The HDMI audio acquisition method provided in this application obtains a Java Virtual Machine instance in the Java native interface initialization function of the local library, and obtains a reference to the target callback method in the Java layer based on the Java Virtual Machine instance. In response to starting audio acquisition, the target audio acquisition method is called in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters. The corresponding audio data is acquired from the HDMI acquisition card. A callback function is called in the Native layer, and the audio data is transmitted in real time to the target player of the Android system running in the Java layer based on the callback function. The target player of the Android system receives the audio data, and after resampling and clock synchronization processing, it outputs the audio data to the audio device. The HDMI audio acquisition method of this application, by transmitting the audio data acquired in the Native layer to the AudioTrack playback of the Android system in real time, utilizes the system-level resampling and clock synchronization capabilities, and eliminates the need to handle sampling rate conversion and clock synchronization manually. This solves the problems of noise and stuttering in HDMI audio acquisition, making it simpler to implement and less prone to errors. In addition, it also solves the problem of needing to restart the device after long-term operation and the problem of inconsistent playback effects caused by differences in clock characteristics of different hardware platforms, improving system stability and compatibility.

[0055] Furthermore, the target audio data loading mode corresponds to the first playback mode (i.e., AudioTrack mode), and the playback mode also includes a second playback mode (Native playback mode); the mode flag of the playback mode is set through the configuration method and takes effect on the next startup; during system runtime, the corresponding playback mode is selected based on the mode flag to switch to the corresponding playback mode.

[0056] Specifically, the system supports switching between AudioTrack mode and Native playback mode at runtime. The mode flag is set through the NativeLib.setUseAudioTrack() method and takes effect on the next startup.

[0057] In summary, the HDMI audio acquisition method of this application corresponds to a three-layer architecture: an audio acquisition layer, a JNI callback layer, and an AudioTrack playback layer. The audio acquisition layer refers to the audio acquisition module running in the Native (C++) layer, which acquires PCM audio data from the HDMI capture card using low-level libraries such as tinyalsa. The JNI callback layer refers to the Java Native Interface layer, used to implement data transfer and function calls between the Native and Java layers, including JavaVM management, JNI environment acquisition, method ID caching, and thread safety management. The AudioTrack playback layer refers to the Android AudioTrack player running in the Java layer, used to receive audio data from the Native layer callback and play it through the Android system's audio services; the system automatically handles resampling and clock synchronization.

[0058] In short, the core of this application lies in its three-layer architecture design, which transmits audio data collected from the Native layer to the Android system's AudioTrack in real time. It leverages system-level capabilities to handle clock synchronization without requiring the implementation of complex synchronization algorithms. Cross-layer data transfer is achieved using the JNI callback mechanism, and JavaVM thread management ensures safety in a multi-threaded environment. In AudioTrack mode, Native playback device initialization is skipped to avoid resource conflicts while maintaining compatibility with Native mode. The system automatically handles resampling and clock synchronization, completely eliminating noise issues caused by hardware clock skew.

[0059] Therefore, this application combines Native layer audio acquisition with Android AudioTrack playback, leveraging system-level resampling and clock synchronization capabilities, achieving the following beneficial effects: 1. Noise reduction: The Android system automatically handles clock synchronization, resolving noise issues caused by accumulated hardware clock deviations. 2. Improved stability: System-level audio processing has undergone thorough testing and optimization, ensuring stable long-term operation without requiring device restarts. 3. Reduced implementation complexity: Eliminating the need to implement resampling and clock synchronization algorithms reduces code volume and maintenance costs. 4. Enhanced compatibility: AudioTrack is an Android standard API, supporting all Android devices and offering excellent compatibility. 5. Dynamic switching support: Dynamic switching between AudioTrack mode and Native mode allows for flexible adaptation to different scenarios.

[0060] This application completely solves the noise problem of HDMI audio acquisition by transmitting the audio data collected by the Native layer to the Android system's AudioTrack for playback in real time, and by utilizing system-level resampling and clock synchronization capabilities, thereby improving system stability and compatibility.

[0061] Figure 2 This is a structural schematic diagram of an HDMI audio acquisition device provided according to an embodiment of this application; as shown below. Figure 2 As shown, the HDMI audio acquisition device 200 of this application embodiment may specifically include: The acquisition module 201 is used to obtain a Java Virtual Machine instance in the Java native interface initialization function of the local library, and obtain a reference to the target callback method of the Java layer based on the Java Virtual Machine instance; wherein, the target callback method represents the callback method corresponding to the core of the Android system used to play raw audio data.

[0062] The acquisition module 202 is used to respond to the start of audio acquisition by calling the target audio acquisition method in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters, and to acquire the corresponding audio data from the HDMI acquisition card.

[0063] Callback module 203 is used to call and set a callback function in the Native layer, and then transmit audio data in real time to the target player running in the Android system in the Java layer based on the callback function.

[0064] The processing module 204 is used to receive audio data through the target player of the Android system, and output the audio data to the audio device after the target player performs resampling and clock synchronization processing on the audio data.

[0065] In one possible implementation, the acquisition module is specifically used for: Based on a Java Virtual Machine instance, obtain a class reference representing the target class in the Java layer; Based on class references, retrieve the target callback method in the target class.

[0066] In one possible implementation, the callback module is specifically used for: In response to the HDMI capture card enabling the target audio data loading mode, the setup callback function is called in the Native layer, and the initialization of the Native playback device is skipped; The Native layer continuously reads audio data from the HDMI capture card and passes the audio data to the Java layer based on a callback function; Create a Java array, copy the audio data from the Native layer into the Java array, and then pass the audio data to the target player.

[0067] In one possible implementation, the callback module is specifically used for: The Native layer continuously reads audio data from the HDMI capture card, and checks whether the callback function has been set after each successful read. If a callback function is set, the callback function will be called to pass the audio data to the Java layer.

[0068] In one possible implementation, the callback module is specifically used for: In the event-driven function, obtain the JNI environment and create a Java array based on the JNI environment; The audio data from the Native layer is copied into a Java array, and a static method in the Java layer is called to pass the audio data to the target player.

[0069] In one possible implementation, the processing module is specifically used for: The audio data is written to the corresponding target buffer; the size of the target buffer is at least four times the size of the minimum buffer. Multi-dimensional processing of audio data is performed based on the target buffer.

[0070] In one possible implementation, the target audio data loading mode corresponds to a first playback mode, and the playback mode further includes a second playback mode; the device further includes: The settings module is used to set the mode flag of the playback mode through configuration methods, and the setting will take effect on the next startup; The switching module is used to select the corresponding playback mode based on the mode flag during system operation, so as to switch to the corresponding playback mode.

[0071] The HDMI audio acquisition device provided in this application obtains a Java Virtual Machine instance in the Java native interface initialization function of the local library, and obtains a reference to the target callback method in the Java layer based on the Java Virtual Machine instance. In response to starting audio acquisition, it calls the target audio acquisition method in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters. It acquires the corresponding audio data from the HDMI acquisition card, calls the set callback function in the Native layer, and transmits the audio data to the target player of the Android system running in the Java layer in real time based on the callback function. The target player of the Android system receives the audio data, and outputs it to the audio device after resampling and clock synchronization. The HDMI audio acquisition device of this application, by transmitting the audio data acquired in the Native layer to the AudioTrack playback of the Android system in real time, utilizes the system-level resampling and clock synchronization capabilities, and does not require handling the sampling rate conversion and clock synchronization itself. This solves the problems of noise and stuttering in HDMI audio acquisition, making the implementation simpler and less prone to errors. In addition, it also solves the problem of needing to restart the device after long-term operation, and the problem of inconsistent playback effects caused by the differences in clock characteristics of different hardware platforms, improving system stability and compatibility.

[0072] like Figure 3 As shown in the embodiment of this application, an electronic device 300 includes a processor 301, a memory 302, and a bus. The memory 302 stores machine-readable instructions executable by the processor 301. When the electronic device is running, the processor 301 communicates with the memory 302 via the bus, and the processor 301 executes the machine-readable instructions to perform the steps of the HDMI audio acquisition method described above.

[0073] Specifically, the memory 302 and processor 301 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 301 runs the computer program stored in the memory 302, it can execute the HDMI audio acquisition method described above.

[0074] Corresponding to the above-described HDMI audio acquisition method, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described HDMI audio acquisition method.

[0075] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0076] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0077] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0078] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the deployment methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0079] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for HDMI audio acquisition based on AUDIO TRACK, characterized in that, The method includes: In the Java native interface initialization function of the local library, a Java Virtual Machine instance is obtained, and a reference to the target callback method of the Java layer is obtained based on the Java Virtual Machine instance; wherein, the target callback method represents the callback method corresponding to the core of the Android system used to play raw audio data; In response to the start of audio acquisition, the target audio acquisition method is called in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters, and the corresponding audio data is acquired from the HDMI acquisition card. In the Native layer, a callback function is set, and based on the callback function, the audio data is transmitted in real time to the target player of the Android system running in the Java layer. The audio data is received by the target player of the Android system, and the target player performs resampling and clock synchronization processing on the audio data before outputting it to the audio device.

2. The method according to claim 1, characterized in that, The step of obtaining a reference to the target callback method in the Java layer based on the Java Virtual Machine instance includes: Based on the Java Virtual Machine instance, obtain a class reference representing the target class in the Java layer; Based on the class reference, obtain the target callback method in the target class.

3. The method according to claim 1, characterized in that, The step of calling and setting a callback function at the Native layer, and then transmitting the audio data in real time to the target player running on the Android system in the Java layer based on the callback function, includes: In response to the HDMI capture card enabling the target audio data loading mode, a setup callback function is called in the Native layer, and the initialization of the Native playback device is skipped; The audio data is continuously read from the HDMI capture card in the Native layer, and the audio data is passed to the Java layer based on the callback function; Create a Java array, copy the audio data from the Native layer into the Java array, and then pass the audio data to the target player.

4. The method according to claim 3, characterized in that, The step of continuously reading the audio data from the HDMI capture card at the Native layer and passing the audio data to the Java layer based on the callback function includes: The audio data is continuously read from the HDMI capture card at the Native layer, and the callback function is checked after each successful read. If the callback function is set, then the callback function is called to pass the audio data to the Java layer.

5. The method according to claim 4, characterized in that, The step of creating a Java array, copying the audio data from the native layer into the Java array, and passing the audio data to the target player includes: In the event-driven function, obtain the JNI environment and create a Java array based on the JNI environment; The audio data from the Native layer is copied into the Java array, and a static method in the Java layer is called to pass the audio data to the target player.

6. The method according to claim 1, characterized in that, The multi-dimensional processing of the audio data includes: The audio data is written to the corresponding target buffer; wherein the size of the target buffer is at least four times the size of the minimum buffer. The audio data is processed in multiple dimensions based on the target buffer.

7. The method according to claim 3, characterized in that, The target audio data loading mode corresponds to a first playback mode, and the playback mode further includes a second playback mode. The method further includes: The mode flag for the playback mode is set via a configuration method and takes effect upon the next startup; During system operation, the corresponding playback mode is selected based on the mode flag to switch to the corresponding playback mode.

8. An HDMI audio acquisition device based on AUDIO TRACK, characterized in that, The device includes: The acquisition module is used to acquire a Java Virtual Machine instance in the Java native interface initialization function of the local library, and to acquire a reference to the target callback method of the Java layer based on the Java Virtual Machine instance; wherein, the target callback method represents the callback method corresponding to the core of the Android system used to play raw audio data; The acquisition module is used to respond to the start of audio acquisition by calling the target audio acquisition method in the Native layer to initialize the HDMI acquisition card and configure the corresponding acquisition parameters, and then acquiring the corresponding audio data from the HDMI acquisition card. The callback module is used to call and set a callback function in the Native layer, and then transmit the audio data to the target player of the Android system running in the Java layer in real time based on the callback function. The processing module is used to receive the audio data through the target player of the Android system, and the target player performs resampling and clock synchronization processing on the audio data before outputting it to the audio device.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the HDMI audio acquisition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the HDMI audio acquisition method as described in any one of claims 1 to 7.

Images