Cross-platform video decoding method and device

Abstract

The invention provides a cross-platform video decoding method and device, computing equipment and a computer readable storage medium, and the method is combined with a factory mode to completely package bottom layer differences of all platform types, and an application layer developer does not need to care about the bottom layer platform differences. The optimal implementation can be automatically selected on different engines and platforms through a uniform API (Application Program Interface); during rendering, a single-frame memory management strategy is used, and only one frame which is currently displayed is reserved in a memory; and in the aspect of performance, the advantages of a multi-thread platform and the parallel computing characteristics of the GPU are fully utilized. According to the embodiment of the invention, through deep fusion of a cross-platform soft decoding architecture, single-frame memory optimization, multi-thread parallel, GPU acceleration and other technologies, perfect balance of cross-platform, high performance and low memory is realized, and the development efficiency of the whole project is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a cross-platform video decoding method and apparatus, computing device and computer-readable storage medium. Background Technology

[0002] With the widespread adoption of digital media, video content has become a core function of various applications, especially games and social applications. To achieve smooth playback and reduce device power consumption, using hardware decoders to process H.264, H.265, and other video streams has become the mainstream technical solution in the industry. However, existing technologies face many inherent defects in achieving cross-platform video playback due to the fragmentation of the technological ecosystem:

[0003] Hardware heterogeneity: The hardware decoder capabilities of different devices vary. Older or low-end devices lack support for efficient codec formats such as H.265, or have limited decoding capabilities, resulting in playback failures or a poor user experience.

[0004] Platform-level restrictions: For security and resource control reasons, various mini-game platforms strictly restrict or completely prohibit applications from directly calling hardware decoding interfaces in their "sandbox" environment, forcing developers to adopt software decoding solutions.

[0005] Web layer standards diverge: Major browsers show significant differences in their support for different codec formats. Content providers must prepare video sources in multiple formats to reach all users. Meanwhile, the video playback APIs of mainstream game engines such as Unity, Unreal Engine, and Cocos Creator are all different. Developers who want to implement cross-engine video functionality must perform repeated adaptation development for each engine, resulting in high R&D and maintenance costs.

[0006] In summary, existing technical solutions suffer from insurmountable compatibility gaps at the hardware, platform, web, and engine levels. This combination of problems forces developers to repeatedly develop video playback functionality to adapt to various platforms when creating the same product. The lack of a unified interface design and significant differences in the technology stacks of different platforms lead to low development efficiency and high maintenance costs. Therefore, building a highly efficient and unified video playback solution that can overcome all barriers and achieve "develop once, run on multiple platforms" has become a pressing technical challenge for the industry. Summary of the Invention

[0007] In view of this, embodiments of this application provide a cross-platform video decoding method and apparatus, computing device and computer-readable storage medium to address the technical deficiencies existing in the prior art.

[0008] According to a first aspect of the embodiments of this application, a cross-platform video decoding method is provided, comprising:

[0009] In response to a video playback request from the application layer, the current platform type is detected, and an adaptive decoding runtime environment is created based on the current platform type. The current platform type includes, but is not limited to, native operating system platforms, mini-game platforms, and web browser or editor environments.

[0010] The software decoder is initialized in the created decoding runtime environment;

[0011] The decoding process is initiated based on the created decoding runtime environment to complete the decoding and rendering of the video. During decoding, the decoding timing is controlled based on the frame timestamp to achieve single-frame memory optimization.

[0012] According to a second aspect of the embodiments of this application, a cross-platform video decoding apparatus is provided, comprising:

[0013] A creation unit is used to detect the current platform type in response to a video playback call request from the application layer, and create an adaptive decoding runtime environment based on the current platform type; the current platform type includes, but is not limited to: native operating system platform, mini-game platform, and web browser or editor environment;

[0014] An initialization unit is used to initialize the software decoder in the created decoding runtime environment;

[0015] The decoding unit is used to start the decoding process according to the created decoding runtime environment to complete the decoding and rendering of the video. During decoding, the decoding timing is controlled according to the frame timestamp to achieve single-frame memory optimization.

[0016] According to a third aspect of the present application, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the instructions to implement the steps of the cross-platform video decoding method.

[0017] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided that stores computer instructions, which, when executed by a processor, implement the steps of the cross-platform video decoding method.

[0018] The embodiments of this application eliminate the reliance on device hardware decoding capabilities, resolving the decoder differences between different brands and models of devices. Furthermore, by incorporating a factory pattern, all underlying differences between operating system platforms and mini-game environments are completely encapsulated. Application layer developers do not need to concern themselves with underlying platform differences to achieve a consistent video playback experience on mainstream game engines, significantly reducing project adaptation costs and maintenance complexity. A unified API interface lowers the learning curve, allowing for automatic selection of the best implementation across different engines and platforms without requiring application layer code modifications during development.

[0019] In terms of performance and memory management, a "single-frame memory management" strategy is used, where only the currently displayed frame is kept in memory. This supports simultaneous playback of multiple videos without causing memory overflow, allowing for smooth playback of high-definition videos even on low-end devices. Regarding performance, an independent decoding thread is enabled on platforms that support Workers, achieving zero-copy data sharing with the main thread through SharedArrayBuffer. Simultaneously, computationally intensive YUV-to-RGB conversion tasks are offloaded to GPU shaders for parallel processing, freeing the CPU from heavy pixel calculations.

[0020] This application embodiment achieves a perfect balance of cross-platform compatibility, high performance, and low memory usage through the deep integration of cross-platform software decoding architecture, single-frame memory optimization, multi-threaded parallelism, and GPU acceleration technologies, greatly improving the efficiency of developers. Attached Figure Description

[0021] Figure 1 This is a structural block diagram of the computing device provided in the embodiments of this application;

[0022] Figure 2 This is a flowchart illustrating a cross-platform video decoding method provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the structure of a cross-platform video decoding device provided in an embodiment of this application. Detailed Implementation

[0024] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0025] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a,” “the,” and “the” used in one or more embodiments of this application and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "in response to a determination".

[0027] This application provides a cross-platform video decoding method and apparatus, computing device and computer-readable storage medium, which will be described in detail in the following embodiments.

[0028] Figure 1 A structural block diagram of a computing device 100 according to an embodiment of this application is shown. The components of the computing device 100 include, but are not limited to, a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and a database 150 is used to store data.

[0029] The computing device 100 also includes an access device 140, which enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 140 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0030] In one embodiment of this application, the aforementioned components of the computing device 100 and Figure 1 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 1The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0031] The computing device 100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 100 can also be a mobile or stationary server.

[0032] Figure 2 The flowchart of a cross-platform video decoding method provided in this application is shown, including steps 202 to 206.

[0033] Step 202: In response to the video playback call request from the application layer, detect the current platform type; create an adaptive decoding runtime environment based on the current platform type.

[0034] In this embodiment of the application, when the application initiates a video playback request, in response to the request, the type of the current platform is first detected, and then a corresponding decoding runtime is created according to the different platform types.

[0035] In one feasible implementation, when an application initiates a video playback request, it calls the playback function at the application layer through a unified video playback component interface. This video playback component interface is implemented using the factory pattern, thereby automatically adapting to different platforms and engines without requiring the application layer to modify the code.

[0036] Furthermore, the development engine uses a factory class to detect the current platform type, which includes, but is not limited to:

[0037] Native operating system platforms: Android, iOS, HarmonyOS Next;

[0038] Mini-game / Quick game platforms: WeChat / Douyin / Alipay mini-games / Huawei / OPPO / VIVO / Xiaomi / Honor;

[0039] And a web browser environment or an editor environment.

[0040] Furthermore, the appropriate decoding runtime is selected based on the different platform types.

[0041] Specifically, if the current platform is a native operating system platform, an instance of a C++ wrapper class is created through JSB (JavaScript-to-Native) to directly call the software decoder, such as the C++ interface of the FFmpeg static library;

[0042] If the current platform is a mini-game platform or a web browser, the software decoder, such as libraries like FFmpeg, Dav1d, and libvpx, is called via WebAssembly. Specifically, the C++ code of the software decoder is pre-compiled into a .wasm high-performance file and a .js glue file, and the JS file is used to call the .wasm file.

[0043] Further steps include checking whether the current mini-game platform supports multi-threading. If it does, a Worker instance is created to load the .wasm file; if not, a single-threaded WASM solution is used to load the .wasm file on the main thread.

[0044] For web browsers, the system checks whether they support SharedArrayBuffer technology. If they do, a multi-threaded approach is used to create Worker instances; otherwise, a single-threaded WASM approach is used.

[0045] Those skilled in the art should know that different development languages ​​are used in different development engines, and different platform types also have different characteristics. Different APIs can be called appropriately to complete the detection of the current platform type. The specific process will not be elaborated here.

[0046] Step 204: Initialize the software decoder in the created runtime environment.

[0047] In this embodiment, the software decoder's library files are pre-trimmed and optimized before initializing the software decoder in the runtime environment. Specifically, for example, when configuring the compilation options for the software encoder FFmpeg, AV1 or VP9 decoders are enabled to replace H.264 and other encodings, and unnecessary codecs, audio processing modules, network protocols, etc., are removed, and the library size is compressed, reducing the size of the FFmpeg library from tens of MB to several MB.

[0048] Furthermore, when initializing the software decoder in the created runtime environment, the following is included:

[0049] (1) Open the video file. Obtain the path to the video file or network stream based on the video playback request and open the video file, for example, by reading it directly from the file system path or by loading it through the game engine's asset management system.

[0050] (2) Read video information and parse video parameters: parse the video header according to the video file to obtain the format, resolution, encoding method, etc.; further parse the video parameters to obtain the width, height, frame rate, total number of frames, duration, etc. of the video based on the read video information.

[0051] (3) Locate and create a decoder instance: Based on the encoding method, locate the corresponding decoder and initialize the decoder instance according to the video parameters. Furthermore, by setting the parameters of the decoder context or initializing a format converter, the decoded frame data is converted into a specific format such as YUV420P for cross-platform processing.

[0052] Further, the YUV texture dimensions (full Y size, half UV size) are calculated based on the obtained video parameters, and a GPU texture buffer is created. In this step, the required dimensions for the Y, U, and V textures are calculated based on the parsed video width and height, providing accurate data for creating the texture buffer. When creating the GPU texture buffer, video memory space is requested from the GPU via the graphics API based on the YUV texture dimensions.

[0053] If the current platform type is a multi-threaded mini-game platform or a web browser, complete the above initialization process (1)-(3) in the first worker and establish an independent WASM software decoder instance. After the first worker completes the software decoder initialization, it sends the obtained video information to the main thread. After receiving the message, the main thread executes the above steps of creating a GPU texture buffer, calculates the YUV texture size according to the obtained video parameters and creates three textures: Y, U, and V, and sends the IDs (handles) of these three textures back to the first worker. In subsequent decoding, these texture IDs are used to update the data.

[0054] If the current platform is a native operating system platform that supports multithreading, a separate background thread is used to complete the initialization of the software decoder, and the main thread or a dedicated rendering thread is used to create the texture buffer.

[0055] If the current platform type is a platform that does not support multithreading, the initialization process of the software decoder and the process of creating GPU texture objects are both completed in the main thread.

[0056] Step 206: Start the decoding process based on the created runtime environment to complete the video decoding and rendering.

[0057] In this step, the video is decoded based on the decoder instance initialized by the runtime environment in step 204.

[0058] If the current platform type is a multi-threaded mini-game platform or a web browser, the WASM software decoder instance runs on the first Worker instance created. The main thread is responsible for game logic and rendering and is not blocked by decoding work.

[0059] Before the first Worker instance starts decoding, the main thread creates a SharedArrayBuffer instance and sends a reference to the SharedArrayBuffer to the first Worker instance via postMessage.

[0060] When the first Worker instance receives the signal to start decoding sent by the main thread, it calls the software decoder instance to begin decoding:

[0061] In the initial state, the frame ready flag on the SharedArrayBuffer instance is 0. The first Worker instance decodes the next frame of the video data and copies the decoded Y, U, and V data to the preset memory location in the SharedArrayBuffer instance.

[0062] The first Worker instance sets the frame ready flag on the SharedArrayBuffer instance to 1 through an atomic operation and then enters a loop to wait.

[0063] When the main thread detects that the frame ready flag is 1 in the Update loop, it reads the decoded data from the SharedArrayBuffer and renders it.

[0064] After the main thread finishes rendering the current frame, it sets the frame ready flag back to 0 and wakes up the first Worker instance.

[0065] After the first Worker instance is awakened, it continues decoding the next frame.

[0066] Preferably, in this embodiment of the application, a single-frame memory strategy is adopted, which retains only the currently displayed frame of YUV data in memory, and ensures smooth playback through precise timing control and fast decoding.

[0067] Specifically, the actual frame interval time deltaTime is passed into the Update loop of the main thread and accumulated, and then the current frame index currentFrame that should be displayed is calculated based on the accumulated time and the frame interval time frameInterval of the video source.

[0068] DeltaTime is the actual time elapsed from the previous frame to the current frame in the game engine (or rendering loop). It is a variable, and its value may differ for each frame, depending on the application's performance, system load, etc. It can be calculated from changes in timestamps.

[0069] `frameInterval` is the duration for which each frame of a currently playing video source should be displayed. It is a constant determined by the video's frame rate (FPS), typically `frameInterval = 1 / videoFPS`.

[0070] Here is an example of code:

[0071] update(deltaTime):

[0072] accumulatedTime + = deltaTime

[0073] currentFrame=floor(accumulatedTime / frameInterval)

[0074] Furthermore, the current frame index (currentFrame) that should be displayed is compared with the frame index (frameIndex) of the last displayed frame. If they are different, the next frame is decoded, and the YUV data of the next frame is put into memory.

[0075] With the above-mentioned single-frame memory rendering method, only 1.5MB of memory is required for 1024x1024 resolution in YUV420P format. Compared with the 4MB or multi-frame caching scheme of traditional PNG format, the memory usage is greatly reduced. This allows multiple videos to be played at the same time without causing memory overflow, and the memory usage is stable and controllable and does not increase with the playback time, thus enabling smooth playback of high-definition videos on low-end devices.

[0076] Preferably, after the main thread retrieves the next frame of YUV data from memory, it completes the rendering in the following manner:

[0077] The main thread uploads YUV data to the GPU memory and updates the GPU texture; the GPU converts YUV to RGB through shaders and renders it, avoiding the performance overhead of CPU pixel-by-pixel conversion.

[0078] Specifically, the conversion from YUV color space to RGB color space is performed using the GPU's parallel computing capabilities within the GPU shader. One feasible conversion method is as follows:

[0079] R = Y + 1.402 * (V - 128)

[0080] G=Y-0.344*(U-128)-0.714*(V-128)

[0081] B = Y + 1.772 * (U - 128)

[0082] If the current platform type is a WASM platform type that supports single thread, such as a fast game platform or a web environment that does not support Workers, then the software decoder instance compiled by WASM is directly called in the main thread, and a single-frame memory strategy is adopted for on-demand decoding. Similarly, the decoding timing is precisely controlled by the frame timestamp method mentioned above.

[0083] If the current platform is a native operating system platform and does not support multithreading, the software decoder instance is called directly via JSB in the main thread, and a single-frame memory strategy is used in the update loop for on-demand decoding. The decoding timing is precisely controlled using the frame timestamp method described above. If multithreading is supported, a separate C++ decoding thread is created, and condition variables and mutexes are used to achieve thread synchronization.

[0084] In the embodiments described above, by fully adopting a software decoding strategy, the reliance on hardware decoding capabilities of devices is eliminated, fundamentally solving the problem of decoder differences between different brands and models of devices. Furthermore, by combining a factory pattern, all underlying differences between operating system platforms and mini-game environments are completely encapsulated. Application layer developers do not need to worry about underlying platform differences to obtain a consistent video playback experience on mainstream game engines. This "develop once, run on all platforms" capability achieves 100% compatibility, greatly reducing project adaptation costs and maintenance difficulty. The unified API interface used also reduces the learning cost; during development, no application layer code modification is required to automatically select the best implementation on different engines and platforms, shortening the adaptation cycle from weeks to days.

[0085] In terms of performance and memory management, a "single-frame memory management" strategy is used, retaining only the currently displayed frame in memory. Combined with the adoption of the YUV420P color format, memory usage is reduced by 60%, supporting simultaneous playback of multiple videos without memory overflow, and enabling smooth playback of high-definition videos even on low-end devices. Regarding performance, the solution fully utilizes modern multi-core architectures, enabling independent decoding threads on platforms supporting Workers, and achieving zero-copy data sharing with the main thread through SharedArrayBuffer, completely avoiding serialization overhead. Simultaneously, the computationally intensive YUV-to-RGB conversion task is offloaded to the GPU shaders for parallel processing, freeing the CPU from heavy pixel calculations, resulting in an overall performance improvement of over 40% compared to traditional solutions.

[0086] Therefore, this application embodiment achieves a perfect balance of cross-platform compatibility, high performance, and low memory usage through the deep integration of cross-platform software decoding architecture, single-frame memory optimization, multi-threaded parallelism, and GPU acceleration technologies, greatly improving the efficiency of developers.

[0087] Corresponding to the above method embodiments, this application also provides an embodiment of a cross-platform video decoding device, such as... Figure 3 As shown, the device includes:

[0088] A creation unit is used to detect the current platform type in response to a video playback call request from the application layer, and create an adaptive decoding runtime environment based on the current platform type; the current platform type includes, but is not limited to: native operating system platform, mini-game platform, and web browser or editor environment;

[0089] An initialization unit is used to initialize the software decoder in the created decoding runtime environment;

[0090] The decoding unit is used to start the decoding process according to the created decoding runtime environment to complete the decoding and rendering of the video. During decoding, the decoding timing is controlled according to the frame timestamp to achieve single-frame memory optimization.

[0091] The above is an illustrative scheme of a cross-platform video decoding device according to this embodiment. It should be noted that the technical solution of this cross-platform video decoding device and the technical solution of the cross-platform video decoding method described above belong to the same concept. For details not described in detail in the technical solution of this cross-platform video decoding device, please refer to the description of the technical solution of the cross-platform video decoding method described above.

[0092] In one embodiment of this application, a computing device is also provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the instructions to implement the steps of the cross-platform video decoding method described above.

[0093] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the cross-platform video decoding method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the cross-platform video decoding method described above.

[0094] An embodiment of this application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the cross-platform video decoding method as described above.

[0095] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the cross-platform video decoding method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the cross-platform video decoding method described above.

[0096] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0097] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0098] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A cross-platform video decoding method, characterized in that, include: In response to a video playback request from the application layer, the current platform type is detected, and an adaptive decoding runtime environment is created based on the current platform type. The current platform type includes, but is not limited to, native operating system platforms, mini-game platforms, and web browser or editor environments. The software decoder is initialized in the created decoding runtime environment; The decoding process is initiated based on the created decoding runtime environment to complete the decoding and rendering of the video. During decoding, the decoding timing is controlled based on the frame timestamp to achieve single-frame memory optimization.

2. The method according to claim 1, wherein, Creating an adaptive decoding runtime environment based on the current platform type includes: If the current platform is a native operating system platform, an instance of a C++ wrapper class is created through JSB to call the software decoder; If the current platform is a mini-game platform or a web browser, the software decoder is called through WebAssembly. The software decoder is pre-compiled into a .wasm file and a .js file, and the .js file calls the .wasm file.

3. The method according to claim 2, wherein, If the current platform is a mini-game platform or a web browser, the software decoder call via WebAssembly includes: Check if the current platform supports multithreading. If it does, create the first Worker instance to load the .wasm file. If it does not support multithreading, use a single-threaded approach and load the .wasm file on the main thread.

4. The method according to claim 3, wherein, Initializing the software decoder in the created decoding runtime environment includes: Obtain the video path according to the video playback call request; read the video information and parse it to obtain the video parameters; find the corresponding decoder according to the obtained video encoding method and initialize the decoder instance according to the video parameters; The texture size is calculated based on the video parameters, and a GPU texture buffer is created.

5. The method according to claim 3, wherein, The method also includes: If the current platform type is a multi-threaded mini-game platform or a web browser, a WASM software decoder instance is created in the first Worker instance; the acquired video information is sent to the main thread, and the main thread executes the step of creating a GPU texture buffer after receiving the message.

6. The method according to claim 5, wherein, The decoding process, which is initiated based on the created decoding runtime environment, completes the video decoding and rendering, including: If the current platform type is a multi-threaded mini-game platform or a web browser, the main thread creates a SharedArrayBuffer instance and sends a reference to the SharedArrayBuffer instance to the first Worker instance; In the initial state, the frame ready flag on the SharedArrayBuffer instance is 0. The first Worker instance decodes the next frame of the video data and copies the decoded next frame data to a preset memory location in the SharedArrayBuffer instance. The first Worker instance sets the frame ready flag to 1 through an atomic operation and then enters a loop to wait; When the main thread detects that the frame is ready as 1 in the Update loop, it reads the decoded data from the SharedArrayBuffer instance and renders it. After the main thread finishes rendering the current frame, it sets the frame ready flag back to 0 and wakes up the first Worker instance. After the first Worker instance is woken up, it continues to decode the next frame.

7. The method according to claim 1, wherein, During decoding, single-frame memory optimization is achieved by controlling the decoding timing based on the frame timestamp, including: In the Update loop of the main thread, the actual frame interval time is passed in and accumulated. Then, the frame index that should be displayed now is calculated based on the accumulated time and the frame interval time of the video source. The frame index that should be displayed now is compared with the frame index that was displayed last time. If the two are different, the decoder instance is notified to decode the next frame. Then, the data of the next frame obtained by decoding is written into memory, and the main thread is notified to complete the rendering.

8. The method according to claim 6, wherein, The method also includes: The main thread uploads the YUV data obtained from the SharedArrayBuffer instance to the GPU memory to update the GPU texture; the GPU converts the YUV color space to the RGB color space through the shader and renders the display.

9. The method according to claim 2, wherein, Before creating an adaptive decoding runtime environment based on the current platform type, the following is included: The library files of the software decoder are trimmed and optimized, and unnecessary modules are removed when compiling the software decoder.

10. A cross-platform video decoding device, characterized in that, include: A creation unit is used to detect the current platform type in response to a video playback call request from the application layer, and create an adaptive decoding runtime environment based on the current platform type; the current platform type includes, but is not limited to: native operating system platform, mini-game platform, and web browser or editor environment; An initialization unit is used to initialize the software decoder in the created decoding runtime environment; The decoding unit is used to start the decoding process according to the created decoding runtime environment to complete the decoding and rendering of the video. During decoding, the decoding timing is controlled according to the frame timestamp to achieve single-frame memory optimization.

11. A computing device, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the instructions, it implements the steps of the method according to any one of claims 1-9.

12. A computer-readable storage medium storing computer instructions, characterized in that, When executed by the processor, this instruction implements the steps of the method according to any one of claims 1-9.

Images