A method for seamless sharing a player across components based on a native layer of a honkong system

Abstract

This invention relates to the field of video playback interaction technology, specifically a method for seamless cross-component sharing of a player based on the HarmonyOS native layer. The method includes steps S1: the HarmonyOS client preloads the native dynamic library file of the player proxy class; step S2: the player proxy class adapts to off-screen rendering windows and supports on-screen rendering windows; step S3: the player dynamically switches rendering windows; step S4: the player proxy class manages the player; and step S5: the player proxy class implements cross-component player transfer. This invention has the following advantages: it greatly simplifies the development logic of the player in complex scenarios involving multi-component switching; it solves the problem of player initiation depending on the creation of XComponent, making preloading scenarios simpler.

Description

Technical Field

[0001] This invention relates to the field of video playback interaction technology, specifically a method for seamless cross-component sharing of a player based on the native layer of the HarmonyOS system. Background Technology

[0002] As video playback interactions become increasingly complex in HarmonyOS clients, users can swipe up and down in a playlist, switch between the playlist and video details, and the player may switch between multiple components or pages. In preloading scenarios, the player needs to pre-decode in UI-less environments. HarmonyOS application layers use ArkUI, meaning developers cannot directly access UI element objects or interfere with UI element object reuse logic. Player rendering relies on ArkUI node XComponents. In these complex user interactions, the existing AVPlayer does not meet the requirements.

[0003] In video multimedia playback scenarios on the HarmonyOS platform, using existing AVPlayer or other players requires adding an XComponent node to ArkUI after player creation. Furthermore, the player can only be manipulated and its SurfaceID (rendering window ID) bound after the XComponent node has fully loaded. Starting playback before the XComponent is created will result in an error, preventing the player from performing off-screen rendering, which is unsuitable for pre-loaded playback scenarios. In scenarios involving multiple components and multiple pages sharing a player, developers need to maintain the lifecycle of the XComponent on each page, increasing development complexity and the error rate to some extent. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method for seamless cross-component sharing of a media player based on the HarmonyOS native layer, thereby resolving the problems mentioned in the background section.

[0005] To achieve the above objectives, the present invention provides a method for seamless cross-component sharing of a media player based on the HarmonyOS native layer, comprising: Step S1: The HarmonyOS client preloads the native dynamic library file of the player proxy class; Step S2: The player proxy class adapts to off-screen rendering windows and supports on-screen rendering windows; Step S3: The player dynamically switches the rendering window; Step S4: The player's proxy class manages the player; Step S5: The player proxy class enables cross-component transfer of the player.

[0006] Specifically, step S1 includes: S1-1: When a HarmonyOS application is cold-started, after the application UI process is instantiated, windowStage.loadContent() is called to load the target page, and an invisible XComponent component is created on that page; S1-2: Set the dynamic library file name and loading callback method for the player proxy class of the XComponent component, and preload the dynamic library into the native layer memory; S1-3: After the dynamic library is loaded, the dynamic library loading object playerProxyNativeObj of the player proxy class is obtained through the loading callback method and stored in the player management class ProxyPlayerManager in the ArkTS layer, thus completing the association between the ArkTS layer and the Native layer player proxy. S1-4: Call the initialization method of ProxyPlayerManager to set the number of buffers and playback start parameters. In playerProxyNativeObj in the native layer, initialize the player queue and rendering window queue, save the player and rendering window references in key-value pairs, and expose the operation methods to ProxyPlayerManager through the Napi bridge.

[0007] Specifically, the XComponent component mentioned in step S1-1 is in an invisible state and is only used for preloading Native dynamic libraries.

[0008] Specifically, step S2 includes: S2-1: The player is based on the native layer and encapsulates FFmpeg and HarmonyOS hardware decoder AVCodec. It uses OHNativeWindow as the rendering window and divides the rendering window into a rendering window that supports on-screen display and an off-screen rendering window, which are managed separately. S2-2: If the business layer passes in the unique ID of XComponent, the proxy class obtains the rendering window based on the ID, injects the lifecycle callback function, senses the window state change, obtains OHNativeWindow after the window is successfully loaded, and stores it in the rendering window queue with the unique ID of XComponent as the key; S2-3: If the business layer does not pass in the unique ID of XComponent, the proxy class creates OHNativeWindow as an off-screen rendering window in the native layer through OH_NativeImage_AcquireNativeWindow(). S2-4: Set the OHNativeWindow and window type obtained in step S2-2 or S2-3 into the player to complete the rendering window binding.

[0009] Specifically, in step S2-1, the rendering windows that support being displayed are included in the rendering window queue for lifecycle management, while off-screen rendering windows do not require lifecycle management.

[0010] Specifically, step S3 includes: S3-1: The player's decoding thread and UI thread are independent of each other. When switching rendering windows, the UI thread initiates the request, and the decoding thread performs the binding operation. S3-2: The player proxy layer issues a switching request on the UI thread, passes in the target rendering window reference and saves it to the player's temporary variable, and adds a switching flag; S3-3: Before decoding the next frame, the decoding thread reads the switching marker and the target rendering window reference, and then binds the rendering layer to the decoder to continue the decoding process.

[0011] Specifically, step S4 includes: S4-1: The ArkTS business layer uses ProxyPlayerManager to perform player creation, query, destruction, and rendering window binding and unbinding operations based on the player ID, thereby being aware of the rendering layer's lifecycle. S4-2: The business layer determines a unique player ID based on the playback scenario and playback resources. The proxy class queries the player queue for the player corresponding to the ID. If the player is not found, a new player is created, the start-up parameters are set, and the player is stored in the queue. If the player is found, it is returned directly. S4-3: The business layer initiates play, pause, stop and other operations through the player ID, and the proxy class locates the player based on the ID and executes the corresponding action; S4-4: The business layer passes in the ID of XComponent and the ID of the player. The proxy class automatically completes the binding between the player and the rendering layer and saves the rendering window lifecycle callback in the native layer. S4-5: Register a listener in the onMemoryLevel() method callback of the application UIAbility. When memory is insufficient, iterate through the player queue and release player objects that are not playing in the foreground.

[0012] Specifically, in step S4-2, the player queue is shared globally within the application, supporting multiple components and multiple pages to share the same player object.

[0013] Specifically, step S5 includes: S5-1: When passing data across components, the business layer records and passes a unique player ID. The cross-components include components within a page and components between pages. S5-2: If the current component is about to be destroyed, the proxy class passes in the player ID, unbinds the current component's rendering layer from the player, releases the corresponding rendering window, creates an off-screen window and binds it to the player in off-screen rendering mode; if the current component does not need to be destroyed, it does not need to be unbound in advance. S5-3: After the receiving component obtains the player ID, it passes the ID and the XComponentID of the current component to the proxy class. The proxy class completes the seamless binding between the player and the new rendering window based on step S3.

[0014] Specifically, during the player's switching rendering window process in step S5-3, the decoder is not rebuilt to maintain the continuity of the playback state.

[0015] The beneficial effects of this invention are: By preloading the native dynamic library of the player proxy, the player proxy class manages the player and rendering window in a unified manner and adapts to the dynamic switching of the rendering window. Developers only need to use the player ID and rendering window ID to perform rendering binding or unbinding, playback behavior control and other logic through the proxy layer, which greatly simplifies the development logic of the player in multi-component switching in complex scenarios. It can also use the off-screen rendering window generated by the proxy class to start playback without UI, solve the problem that the player starts playback depends on the creation of XComponent, and make the preloading scenario simple. Attached Figure Description

[0016] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of a method for seamless cross-component sharing of a player based on the native layer of the HarmonyOS system in this invention; Figure 2 This is a flowchart illustrating the characteristics of the Native dynamic library file for the pre-loading player proxy class in the HarmonyOS client in this invention; Figure 3 This is a flowchart illustrating the features of the player proxy class in this invention regarding the adaptation method for off-screen rendering windows and the support for sending rendering windows. Figure 4 This is a flowchart illustrating the features of the player's adaptation method for dynamically switching rendering windows in this invention; Figure 5 A flowchart for managing player characteristics for a player proxy class; Figure 6 This is a flowchart illustrating the process of passing player features across components using a player proxy class in this invention. Detailed Implementation

[0017] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0018] Please see Figures 1-6 This invention provides a technical solution: a method for seamless cross-component sharing of a player based on the HarmonyOS native layer, comprising: S1. Characteristics of HarmonyOS client preloading player proxy class's native dynamic library file: S1-1. In the HarmonyOS application cold start scenario, after the application UI process is instantiated, windowStage.loadContent() is called to load the corresponding target page for the main window, and an invisible XComponent is created in the page. S1-2, Set the name of the dynamic library file of the player proxy class to be loaded and the callback method to listen for loading for the XComponent created in S1-1 above, and start preloading the dynamic library of the player proxy class into the native layer memory; S1-3. After the dynamic library is loaded, obtain the dynamic library loading object playerProxyNativeObj of the player proxy class in the loading callback method of S1-2XComponent above, and save it in the player management class ProxyPlayerManager in the ArkTS layer to complete the association between the player proxy of the ArkTS layer and the Native layer. S1-4. After completing the association in S1-3 above, call the ProxyPlayerManager initialization method in the ArkTS layer to set the relevant configurations (number of buffers, playback start parameters) in the player proxy in the ArkTS layer. In playerProxyNativeObj in the native layer, initialize the player queue and rendering window queue, and use key-value pairs to store references to the player and rendering windows. The player queue is shared globally within the application and uses a NATI bridge to expose the provided methods for operating the player and rendering windows to ProxyPlayerManager. Developers only need to obtain this proxy class to operate on multiple players and multiple windows.

[0019] S2, Features of the player proxy class's methods for adapting to off-screen rendering windows and supporting on-screen rendering window adaptation: S2-1: The player uses FFmpeg and HarmonyOS hardware decoder AVCodec, encapsulated on a native layer, and uses OHNativeWindow as the rendering window, managing window types accordingly. The player's internal business logic categorizes rendering windows into two types: one type supports on-screen display, whose reference is stored in the window queue created in S1-4, and the player's proxy class needs to manage the lifecycle of this type of window; the other type is off-screen rendering, which does not support on-screen display and does not require lifecycle management. When creating a player object using the player proxy class, players can choose between the two rendering methods mentioned above. The ArkTS business layer can determine whether a ready rendering layer is available and select the appropriate rendering method. S2-2. In the scenario of creating and using the player in the player proxy class, if the business layer passes in the unique ID of XComponent, it is regarded as starting the playback with a rendering window. In the native layer, the proxy class will obtain the rendering window according to the passed unique rendering ID and perform lifecycle management on the rendering window. First, the proxy layer actively injects the rendering window lifecycle callback function. The native layer perceives the changes in the rendering window state. In the successful loading callback of the rendering window, OHNativeWindow is obtained and the unique ID of XComponent is used as the key to save it in the window queue created in [S1-4]. S2-3. In the above-mentioned S2-2 player proxy class creation and player use scenario, if the business layer does not pass in the unique ID of XComponent, it is considered to be off-screen playback. The player does not need to wait for the UI layer to finish loading before starting playback. The proxy class uses OH_NativeImage_AcquireNativeWindow() in the native layer to create OHNativeWindow as the rendering layer for off-screen rendering. S2-4. In both rendering methods S2-2 and S2-3 above, the OHNativeWindow entity responsible for rendering is obtained. The business layer calls the binding method to set the rendering window and window type in the player, thus completing the rendering window binding process.

[0020] S3. Features of the player's adaptation method for dynamically switching rendering windows: S3-1. Based on the two playback rendering methods S2-2 and S2-3 mentioned above, the business layer only needs to determine whether to pass in the ID of XComponent. Everything else is handled by the native layer for window management. In order to achieve a seamless switching effect, the decoder in the player cannot be rebuilt. The decoding thread and the user's UI thread (the player proxy class also runs on the UI thread) are two independent threads. When the player proxy class requests to switch rendering windows, it cannot operate directly on the UI thread and needs to switch to the decoding thread. S3-2. Based on the thread model of S3-1, in the scenario of switching rendering layers, the player proxy layer issues a switching request in the UI thread, passing in the rendering window reference as a parameter. The rendering window reference is stored in a temporary variable inside the player, and a flag for requesting to switch rendering windows is added. At this time, the switching request is ready. S3-3: The player's internal decoding thread continuously loops through decoding. Before decoding the next frame, it reads the request to switch the rendering layer flag from S3-2, reads the rendering window reference saved by S3-2, and synchronously binds the rendering layer and the decoder. After the binding is completed, the decoding process continues.

[0021] S4. Player proxy class manages player features: S4-1, the HarmonyOS client ArkTS business layer obtains the initialized player management class object ProxyPlayerManager through S1-4. In this proxy class, developers can create, query, and destroy players based on the unique player ID. They can also bind and unbind rendering windows and perceive the rendering layer lifecycle through rendering window ID and player ID. S4-2: The business layer first determines a unique player ID based on the playback scenario and playback resources, and uses the player proxy class to query the player queue in S1-4. If the player object corresponding to the ID is not found in the player queue of the S1-4 proxy class, a new player object is created, and the start playback parameters in the proxy class are set in the player object. The reference of the newly created player object is stored in the player queue of S1-4. If the player object corresponding to the ID is found in the player queue of the S1-4 proxy class, it is directly returned to the business caller of ArkTS. S4-3. Based on S4-2, the business layer has already created a player object based on the player ID. The business layer can then use this ID to perform a series of playback operations, including operations such as (start playback, pause playback, stop playback, etc.). After receiving the operation execution command, the player proxy layer passes in the player ID as a parameter, finds the player based on the ID, and executes the corresponding action. S4-4. Based on the player created by S4-2, the business layer passes the ID of the XComponent to be rendered and the player ID to the player proxy layer. The native layer player proxy automatically binds the player and the rendering layer according to the characteristics of S2-2 and S3-2, and saves the lifecycle of the rendering window injected in S2-2 in the native layer proxy class to complete the lifecycle awareness of XComponent. S4-5. Register a callback in the onMemoryLevel() method of UIAbility when the application starts. In the case of insufficient memory, the system will call back this method. The player proxy class will traverse the player queue in the S1-4 proxy class according to the callback of this method, find the player object that is not in the playback state, stop and release the player.

[0022] S5, Passing Player Features Across Components Using a Player Proxy Class: S5-1. During playback, a unique player ID can be used for cross-component passing. Cross-component passing can be distinguished between intra-page components and inter-page components. The business layer only needs to record the player ID to be passed and pass the player ID string to the next component. S5-2. After passing the player ID in S5-1, distinguish whether the current page component is about to be destroyed. If the current component is about to be destroyed, use the player proxy class to pass in the player ID, unbind the current component's rendering layer from the player, and release the rendering layer window saved in S1-4. Use S2-3 to create an off-screen window and bind the player in off-screen rendering mode to complete the separation of the player and the old component window. If the current component does not need to be destroyed, there is no need to unbind it from the player in advance. S5-3: When the next component receives the player ID, it uses the S4-4 method in the player proxy class to pass the player ID and the current component's XComponentID as parameters. The player proxy class seamlessly switches the rendering layer process in the native layer based on S3 to complete the binding between the player and the rendering window.

[0023] The “Native layer” refers to the low-level code written in C / C++. It interacts with the upper-layer application (ArkTS) through the Native API to implement functions that have high requirements for performance or system resources, such as high-performance computing, hardware access, graphics rendering, audio and video processing, and encryption.

[0024] The XComponent component, as a rendering component, can be used for EGL / OpenGLES and media data writing. It renders the screen by using the "NativeWindow" held by the XComponent, and is usually used to meet the more complex custom rendering needs of developers.

[0025] By preloading the native dynamic library of the player proxy, the player proxy class manages the player and rendering window in a unified manner and adapts to the dynamic switching of the rendering window. Developers only need to use the player ID and rendering window ID to perform rendering binding or unbinding, playback behavior control and other logic through the proxy layer, which greatly simplifies the development logic of the player in multi-component switching in complex scenarios. It can also use the off-screen rendering window generated by the proxy class to start playback without UI, solve the problem that the player starts playback depends on the creation of XComponent, and make the preloading scenario simple.

[0026] By uniformly managing all players required within the application, the UI business layer's operations on the players are delegated. A unique player ID is determined based on the playback scenario and playback resources, and this player ID value is used to locate the application's unique player object. All business layers only need to save the player ID to control playback behavior and operate on the player based on the native layer. Developers in the ArkUI layer do not need to care about the lifecycle of the player and rendering window; they only need to record the rendering window ID and player ID to associate them. The player proxy class supports the creation of off-screen rendering windows, allowing off-screen playback to start in advance in scenarios without XComponent. The player proxy senses application exit and insufficient memory callbacks, proactively releasing player objects that are not playing in the foreground. In multi-page switching scenarios, the business layer only needs to pass the player ID to pass the player across multiple pages, and can seamlessly bind or unbind the player and rendering window.

[0027] Although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for seamless cross-component sharing of a media player based on the HarmonyOS native layer, characterized in that, include: Step S1: The HarmonyOS client preloads the native dynamic library file of the player proxy class; Step S2: The player proxy class adapts to off-screen rendering windows and supports on-screen rendering windows; Step S3: The player dynamically switches the rendering window; Step S4: The player's proxy class manages the player; Step S5: The player proxy class enables cross-component transfer of the player.

2. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 1, characterized in that, Step S1 specifically includes: S1-1: When a HarmonyOS application is cold-started, after the application UI process is instantiated, windowStage.loadContent() is called to load the target page, and an invisible XComponent component is created on that page; S1-2: Set the dynamic library file name and loading callback method for the player proxy class of the XComponent component, and preload the dynamic library into the native layer memory; S1-3: After the dynamic library is loaded, the dynamic library loading object playerProxyNativeObj of the player proxy class is obtained through the loading callback method and stored in the player management class ProxyPlayerManager in the ArkTS layer, thus completing the association between the ArkTS layer and the Native layer player proxy. S1-4: Call the initialization method of ProxyPlayerManager to set the number of buffers and playback start parameters. In playerProxyNativeObj in the native layer, initialize the player queue and rendering window queue, save the player and rendering window references in key-value pairs, and expose the operation methods to ProxyPlayerManager through the Napi bridge.

3. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 2, characterized in that, The XComponent component mentioned in step S1-1 is in an invisible state and is only used for preloading Native dynamic libraries.

4. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 1, characterized in that, Step S2 specifically includes: S2-1: The player is based on the native layer and encapsulates FFmpeg and HarmonyOS hardware decoder AVCodec. It uses OHNativeWindow as the rendering window and divides the rendering window into a rendering window that supports on-screen display and an off-screen rendering window, which are managed separately. S2-2: If the business layer passes in the unique ID of XComponent, the proxy class obtains the rendering window based on the ID, injects the lifecycle callback function, senses the window state change, obtains OHNativeWindow after the window is successfully loaded, and stores it in the rendering window queue with the unique ID of XComponent as the key; S2-3: If the business layer does not pass in the unique ID of XComponent, the proxy class creates OHNativeWindow as an off-screen rendering window in the native layer through OH_NativeImage_AcquireNativeWindow(). S2-4: Set the OHNativeWindow and window type obtained in step S2-2 or S2-3 into the player to complete the rendering window binding.

5. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 4, characterized in that, In step S2-1, the rendering windows that support being sent to the screen are included in the rendering window queue for lifecycle management, while off-screen rendering windows do not require lifecycle management.

6. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 1, characterized in that, Step S3 specifically includes: S3-1: The player's decoding thread and UI thread are independent of each other. When switching rendering windows, the UI thread initiates the request, and the decoding thread performs the binding operation. S3-2: The player proxy layer issues a switching request on the UI thread, passes in the target rendering window reference and saves it to the player's temporary variable, and adds a switching flag; S3-3: Before decoding the next frame, the decoding thread reads the switching marker and the target rendering window reference, and then binds the rendering layer to the decoder to continue the decoding process.

7. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 1, characterized in that, Step S4 specifically includes: S4-1: The ArkTS business layer uses ProxyPlayerManager to perform player creation, query, destruction, and rendering window binding and unbinding operations based on the player ID, thereby being aware of the rendering layer's lifecycle. S4-2: The business layer determines a unique player ID based on the playback scenario and playback resources. The proxy class queries the player queue for the player corresponding to the ID. If the player is not found, a new player is created, the start-up parameters are set, and the player is stored in the queue. If the player is found, it is returned directly. S4-3: The business layer initiates play, pause, stop and other operations through the player ID, and the proxy class locates the player based on the ID and executes the corresponding action; S4-4: The business layer passes in the ID of XComponent and the ID of the player. The proxy class automatically completes the binding between the player and the rendering layer and saves the rendering window lifecycle callback in the native layer. S4-5: Register a listener in the onMemoryLevel() method callback of the application UIAbility. When memory is insufficient, iterate through the player queue and release player objects that are not playing in the foreground.

8. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 7, characterized in that, In step S4-2, the player queue is shared globally within the application, supporting multiple components and multiple pages to share the same player object.

9. The method for seamless cross-component sharing of a player based on the HarmonyOS native layer according to claim 1, characterized in that, Step S5 specifically includes: S5-1: When passing data across components, the business layer records and passes a unique player ID. The cross-components include components within a page and components between pages. S5-2: If the current component is about to be destroyed, the proxy class passes in the player ID, unbinds the current component's rendering layer from the player, releases the corresponding rendering window, creates an off-screen window and binds it to the player in off-screen rendering mode; if the current component does not need to be destroyed, it does not need to be unbound in advance. S5-3: After the receiving component obtains the player ID, it passes the ID and the XComponentID of the current component to the proxy class. The proxy class completes the seamless binding between the player and the new rendering window based on step S3.

10. A method for seamless cross-component sharing of a player based on the native layer of the HarmonyOS system according to claim 9, characterized in that, During the player's switching rendering window process in step S5-3, the decoder is not rebuilt to maintain the continuity of the playback state.

Images