Multi-version FFmpeg Switching Method, Device and Medium

Abstract

An embodiment of the present invention discloses a multi-version FFmpeg switching method, device, and medium. The method includes: after an upper-layer application initiates an FFmpeg function call, intercept the FFmpeg function call by using the modified environment variable pointing to the middle-layer library; when it is detected that there is a video processing sub-function in the intercepted FFmpeg function, use the created global thread-local variable to record whether each process is in a video processing state respectively; determine the process for video processing according to the local variable, and when the subsequent video processing sub-function is called by the process, execute video processing by using the sub-function of the video processing version FFmpeg function. This not only ensures the flexibility of video processing but also supports seamless switching between versions, thereby improving the overall applicability of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of RISC-V system multimedia processing, and particularly to a method, device and medium for switching multiple versions of FFmpeg. Background Art

[0002] With the rapid development of the RISC-V architecture, more and more systems and devices are beginning to adopt RISC-V processors. However, RISC-V systems face some compatibility challenges in multimedia processing, especially in video decoding. Currently, many RISC-V systems rely on a video processing unit (VPU) for video hardware decoding, and the VPU usually requires a specific version of FFmpeg to cooperate with it. FFmpeg, as an open-source multimedia processing tool, is widely used in scenarios such as audio and video encoding, decoding, and conversion.

[0003] Existing solutions require the FFmpeg version in the system to be strictly matched with the hard decoding plugin, which makes it easy to have incompatibility problems when updating the FFmpeg version or different applications depend on different versions. Especially on the RISC-V architecture, the flexibility and adaptability of the system are limited, and users cannot freely switch the FFmpeg version when using hardware-accelerated decoding, causing difficulties in development and deployment. Summary of the Invention

[0004] Embodiments of the present invention provide a method, device and medium for switching multiple versions of FFmpeg to solve the technical problem that the FFmpeg version in the RISC-V system cannot be freely switched in the prior art.

[0005] In a first aspect, embodiments of the present invention provide a method for switching multiple versions of FFmpeg, including:

[0006] After an upper-layer application initiates an FFmpeg function call, intercept the FFmpeg function call by using an environment variable modified to point to an intermediate-layer library;

[0007] When a video processing sub-function exists in the intercepted FFmpeg function, use the created global thread local variable to record whether each process is in a video processing state respectively;

[0008] Determine the process for processing video according to the local variable, and when the process subsequently calls the video processing sub-function, execute video processing by using a sub-function of the FFmpeg function of the video processing version.

[0009] In a second aspect, embodiments of the present invention further provide a device for switching multiple versions of FFmpeg, including:

[0010] An interception module, which is used to intercept FFmpeg function calls by using the modified environment variable pointing to the middle-layer library after the upper-layer application initiates an FFmpeg function call;

[0011] A recording module, which is used to use the created global thread-local variable to record whether each process is in the video processing state respectively when a video processing sub-function exists in the intercepted FFmpeg function;

[0012] Call the process for video processing determined according to the local variable, and when the video processing sub-function is subsequently called by the process, execute video processing by using the sub-function of the FFmpeg function of the video processing version.

[0013] In a third aspect, an embodiment of the present invention further provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the multi-version FFmpeg switching method provided in the above embodiment when executed by a computer processor.

[0014] The multi-version FFmpeg switching method, device and medium provided by the embodiments of the present invention intercept FFmpeg function calls by using the modified environment variable pointing to the middle-layer library after the upper-layer application initiates an FFmpeg function call; when a video processing sub-function exists in the intercepted FFmpeg function, use the created global thread-local variable to record whether each process is in the video processing state respectively; determine the process for video processing according to the local variable, and when the video processing sub-function is subsequently called by the process, execute video processing by using the sub-function of the FFmpeg function of the video processing version. By using the newly added middle layer, a unified interface is defined to intercept FFmepg function calls, and thread-local variables are used to enable each thread to record its own state, preventing interference during multi-threaded access. Subsequently, when calling the video processing function, it can be determined whether the current is video processing. If it is video processing, the FFmepg of the video processing version is called, otherwise the system version FFmepg is called. This not only ensures the flexibility of video processing but also supports seamless switching between versions, thereby improving the overall applicability of the system. Description of the Drawings

[0015] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects and advantages of the present invention will become more obvious:

[0016] Figure 1 is a flowchart of the multi-version FFmpeg switching method provided by Embodiment 1 of the present invention;

[0017] Figure 2 is a flowchart of the multi-version FFmpeg switching method provided by Embodiment 2 of the present invention;

[0018] Figure 3 It is a schematic structural diagram of the multi-version FFmpeg switching device provided in the third embodiment of the present invention. Detailed implementation manners

[0019] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the drawings rather than all the structures.

[0020] Embodiment 1

[0021] Figure 1 It is a flowchart of the multi-version FFmpeg switching method provided in the first embodiment of the present invention. This embodiment can be directed to the situation where the FFmpeg version in the RISC-V system can be freely switched according to the hard decoding plug-in. This method can be executed by the multi-version FFmpeg switching device, and specifically includes the following steps:

[0022] Step 110, after the upper-layer application initiates an FFmpeg function call, use the modified environment variable pointing to the middle-layer library to intercept the FFmpeg function call.

[0023] In the prior art, video hardware decoding under the RISC-V architecture depends on a specific version of FFmpeg and its plug-ins. However, the FFmpeg version used by the desktop environment components in the system is often inconsistent with the FFmpeg version required for hard decoding, resulting in the inability of hard decoding to work properly.

[0024] Due to the inability to flexibly call the FFmpeg version that adapts to the hard decoding requirements, the efficiency of video hard decoding in the prior art is often difficult to reach the optimum. When the system cannot load the FFmpeg version that matches the VPU hardware, it will fallback to the software decoding method, resulting in a slower decoding speed, increased resource consumption, and the inability to fully utilize the performance advantages of hardware acceleration.

[0025] Although the problem can be partially solved by the method of upgrading and overwriting currently, it may damage the compatibility with the video hard decoding plug-in.

[0026] Therefore, in this embodiment, a compatibility layer is designed first. Exemplarily, the compatibility layer may include: a middle layer that can define a unified interface to intercept FFmepg function calls. When an upper-layer application, that is, at the operating system level, needs to call FFmepg for audio or video processing, the middle layer is used to intercept FFmpeg function calls. Optionally, the environment variable LD_PRELOAD= / usr / local / libmy_FFmpeg_intercept.so can be set and written into / etc / profile to make the environment variable take effect permanently, so that the middle-layer library libmy_FFmpeg_intercept.so can receive and intercept the calls of the upper-layer application to FFmpeg functions.

[0027] Step 120, when it is intercepted that there is a video processing sub-function in the FFmpeg function, use the created global thread-local variable to record whether each process is in the video processing state respectively.

[0028] FFmpeg is an open-source computer program that can be used to record, convert digital audio and video, and convert them into streams, including a large number of sub-functions. Use its sub-functions to determine whether it is currently used for video processing. Exemplarily, video sub-functions such as avformat_open_input, avformat_find_stream_info, av_find_best_stream, avcodec_find_decoder, avcodec_open2, av_read_frame, avcodec_send_packet, avcodec_receive_frame, av_packet_unref, av_frame_free, avcodec_free_context, avformat_close_input can be used to determine the current video processing state.

[0029] Since there are multiple processes in the system, if all of them are intercepted and processed, it will affect the processing of other processes. At the same time, since FFmpeg may be used by multiple threads, in this embodiment, independent states are saved for each thread call, so that each thread can use different versions of FFmpeg respectively and does not affect other processes using FFmpeg functions.

[0030] Exemplarily, a global thread-local variable static __thread int is_video_processing can be created first to record whether each thread is currently in the state of processing video. After creation, the variable is initialized, that is, the variable of each thread is initially set to 0. Then it is judged whether each thread calls the video processing sub-function. When a thread calls the video processing sub-function, the variable of the corresponding thread is modified to 1.

[0031] Optionally, the judgment of whether the thread calls the video processing sub-function may include: judging whether the thread calls the FFmpeg multimedia stream processing sub-function. When calling, read the url parameter of the FFmpeg multimedia stream processing sub-function, and determine whether the processing object of the FFmpeg multimedia stream processing sub-function is the file to be processed or the data format is video, and the video attributes. Use the url parameter to judge whether the thread calls the video processing sub-function, and the corresponding FFmpeg version.

[0032] The FFmpeg multimedia stream processing sub-function avformat_open_input is the initial function for video processing. Its function is to open the video to be processed. By judging whether the format passed in through its char *url parameter is video, if it is video, set is_video_processing to 1.

[0033] Further, when there are multiple versions, the video type of the char *url parameter can be used to further determine the corresponding FFmpeg version to achieve adaptation to multiple FFmpeg versions. Further, the most suitable FFmpeg version can also be selected in combination with the application program. Optionally, determine the application to which the process belongs according to the process, find the video processing format and decoding version of the application, and determine the corresponding FFmpeg version according to the video processing format, decoding version and char *url parameter.

[0034] Step 130, determine the process of processing video according to the local variable, and when the video processing sub-function is called subsequently in the process, execute video processing by using the sub-function of the FFmpeg function of the video processing version.

[0035] Exemplarily, the process with is_video_processing set to 1 can be found, and when the video processing sub-function is called subsequently in it, execute video processing by using the sub-function of the FFmpeg function of the video processing version. So that the video processing version of FFmpeg is strictly matched with the hardware decoding plugin to improve the video processing efficiency. When it is not video, directly call the original function with the same name of the system version of FFmpeg through dlsym.

[0036] In a preferred embodiment of the present embodiment, the method may further include the following steps: after the system version is normally installed, decompress the deb package of the video processing version, and modify the file path to a path different from the system version; use the dpkg command to repackage and install the decompressed deb package after modifying the path; redefine and declare all function interfaces of different versions of FFmpeg in the middle layer library, with the same name as the original FFmpeg function interfaces, for intercepting function calls. By using the above method, multiple versions of FFmpeg can coexist. At the same time, all function interfaces of different versions of FFmpeg are redefined and declared in the middle layer library to facilitate the video processing process to conveniently call the FFmpeg functions of each version.

[0037] Embodiment 2

[0038] Figure 2 FIG. is a schematic flowchart of a multi-version FFmpeg switching method provided in Embodiment 2 of the present invention. This embodiment is optimized based on the above embodiment. When the video processing sub-function is subsequently called in the process, the video processing is performed by using the sub-function of the FFmpeg function of the video processing version. Specifically, it is optimized as follows: call the custom open FFmpeg version function load_custom_FFmpeg() function to read the FFmpeg of the video processing version, and use the dynamic link function dlsym to call the functions of the FFmpeg of the video processing version. And the method may further include the following steps: when the process intercepts the FFmpeg multimedia stream cleanup and close sub-function, modify the variable of the process in the global thread local variable to the initial value. Call the custom close FFmpeg version function unload_custom_FFmpeg() to close the FFmpeg of the video processing version, and use the dynamic link function dlsym to call the original function of the same name of the system version FFmpeg.

[0039] See Figure 2 , the multi-version FFmpeg switching method includes:

[0040] Step 210, when the upper-layer application initiates an FFmpeg function call, intercept the FFmpeg function call by using the modified environment variable pointing to the middle layer library.

[0041] Step 220, when the upper-layer application initiates an FFmpeg function call, intercept the FFmpeg function call by using the modified environment variable pointing to the middle layer library.

[0042] Step 230: Determine the process for processing the video based on local variables. When the video processing sub-function is subsequently called in the process, call the custom function load_custom_FFmpeg() to read the FFmpeg version for video processing, and use the function of the FFmpeg version for video processing called through the dynamic link function dlsym.

[0043] In this embodiment, a custom load_custom_FFmpeg() function can be provided in the middle layer to open the specified FFmpeg version of the process. Use the function of the FFmpeg version for video processing called through the dynamic link function dlsym. Use the dynamic link function dlsym to call the function in the library corresponding to the FFmpeg version for video processing.

[0044] Step 240: When the process intercepts the FFmpeg multimedia stream cleanup and close sub-function, modify the variable of the process in the global thread local variable to its initial value.

[0045] In this embodiment, avformat_close_input is the FFmpeg multimedia stream cleanup and close sub-function, which can determine that the process has completed video processing. Therefore, the variable of the process in the global thread local variable can be modified to its initial value, that is, set to 0. This is to facilitate the call of the original FFmpeg version.

[0046] Step 250: Call the custom function unload_custom_FFmpeg() to close the FFmpeg version for video processing, and use the dynamic link function dlsym to call the original function with the same name of the system version of FFmpeg.

[0047] Exemplarily, the custom function unload_custom_FFmpeg() added in the middle layer can be used to close the FFmpeg version for video processing. And use the dynamic link function dlsym to call the original function with the same name of the system version of FFmpeg.

[0048] In this embodiment, when the video processing sub-function is subsequently called in the process, the sub-function using the video processing version of the FFmpeg function is used to perform video processing. Specifically, the optimization is as follows: The custom function load_custom_FFmpeg() for opening the FFmpeg version is called to read the video processing version of FFmpeg, and the functions of the video processing version of FFmpeg are called using the dynamic linking function dlsym. Moreover, the method can further include the following steps: When the process intercepts the FFmpeg multimedia stream cleaning and closing sub-function, the variable of the process in the global thread local variable is modified to the initial value. The custom function unload_custom_FFmpeg() for closing the FFmpeg version is called to close the video processing version of FFmpeg, and the original function with the same name of the system version of FFmpeg is called using the dynamic linking function dlsym. In the above manner, the sub-function using the FFmpeg function of the video processing version is used to perform video processing. And by using the FFmpeg multimedia stream cleaning and closing sub-function therein, the variable of the process is modified, and the custom function unload_custom_FFmpeg() newly added in the middle layer is called to close the video processing version of FFmpeg, and the original function with the same name of the system version of FFmpeg is called using the dynamic linking function dlsym. This avoids affecting the system version of FFmpeg.

[0049] Embodiment 3

[0050] Figure 3 It is a schematic structural diagram of the multi-version FFmpeg switching device provided in Embodiment 3 of the present invention. Refer to Figure 3 , the multi-version FFmpeg switching device includes:

[0051] An interception module 310, configured to intercept the FFmpeg function call by using the modified environment variable pointing to the middle layer library when the upper-layer application initiates an FFmpeg function call;

[0052] A recording module 320, configured to use the created global thread local variable to record whether each process is in the video processing state respectively when a video processing sub-function exists in the intercepted FFmpeg function;

[0053] An execution module, configured to determine the process for video processing according to the local variable, and when the video processing sub-function is subsequently called in the process, use the sub-function of the video processing version of the FFmpeg function to perform video processing.

[0054] The multi-version FFmpeg switching device provided in this embodiment intercepts FFmpeg function calls by using the modified environment variable pointing to the middle layer library after the upper-layer application initiates an FFmpeg function call. When it intercepts that there is a video processing sub-function in the FFmpeg function, it uses the created global thread-local variable to record whether each process is in the video processing state respectively. It determines the process for video processing according to the local variable, and when the process subsequently calls the video processing sub-function, it uses the sub-function of the FFmpeg function for video processing version to perform video processing. By using the newly added middle layer, it defines a unified interface, intercepts FFmepg function calls, uses thread-local variables to enable each thread to record its own state, prevents interference during multi-threaded access, and can then determine whether it is currently for video processing when calling the video processing function. If it is for video processing, it calls the FFmepg for video processing version, otherwise it calls the system version FFmepg. This not only ensures the flexibility of video processing but also supports seamless switching between versions, thereby enhancing the overall applicability of the system.

[0055] Based on the above embodiments, the recording module includes:

[0056] A creation unit for creating a global thread-local variable static__thread int is_video_processing;

[0057] An initialization unit for initializing the variable of each thread to 0;

[0058] A modification unit for determining whether a thread calls a video processing sub-function, and when the thread processes and calls the video processing sub-function, modifying the variable of the corresponding thread to 1.

[0059] Based on the above embodiments, the modification unit is used for:

[0060] Determining whether a thread calls an FFmpeg multimedia stream processing sub-function. When calling, it reads the url parameter of the FFmpeg multimedia stream processing sub-function, determines whether the processing object of the FFmpeg multimedia stream processing sub-function is the processed file or the data format is video, and the video attributes, and uses the url parameter to determine whether the thread calls the video processing sub-function and the corresponding FFmpeg version.

[0061] Based on the above embodiments, the device further includes:

[0062] A variable modification module for modifying the variable of the process in the global thread-local variable to the initial value when the process intercepts the FFmpeg multimedia stream cleaning and closing sub-function.

[0063] Based on the above embodiments, the device further includes:

[0064] A closing module, configured to call the custom function unload_custom_FFmpeg() for closing the FFmpeg version for video processing to close the FFmpeg version for video processing, and call the original function with the same name of the system version FFmpeg by using the dynamic link function dlsym.

[0065] Based on the above embodiments, the execution module includes:

[0066] A calling unit, configured to call the custom function load_custom_FFmpeg() for opening the FFmpeg version to read the FFmpeg version for video processing, and call the functions of the FFmpeg version for video processing by using the dynamic link function dlsym.

[0067] Based on the above embodiments, the interception module includes:

[0068] A setting unit, configured to set the environment variable LD_PRELOAD= / usr / local / libmy_FFmpeg_intercept.so, and write it into / etc / profile to make the environment variable take effect permanently. The intermediate library libmy_FFmpeg_intercept.so receives and intercepts the calls of the upper-layer application to the FFmpeg functions.

[0069] Based on the above embodiments, the device further includes:

[0070] A path modification module, configured to, after the system version is normally installed, decompress the deb package of the FFmpeg version for video processing, and modify the file path to a path different from that of the system version;

[0071] A repackaging module, configured to use the dpkg command to repackage and install the decompressed deb package after modifying the path;

[0072] A redefinition module, configured to redefine and declare all function interfaces of different versions of FFmpeg in the intermediate library, with the same name as the original function interfaces of FFmpeg, for intercepting function calls.

[0073] The multi-version FFmpeg switching device provided by the embodiments of the present invention can execute the multi-version FFmpeg switching method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0074] Embodiment 4

[0075] Embodiment 4 of the present invention further provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute any of the multi-version FFmpeg switching methods provided in the above embodiments when executed by a computer processor.

[0076] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable media may be computer-readable signal media or computer-readable storage media. The computer-readable storage media may, for example, but not be limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, the computer-readable storage media may be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system, device, or component.

[0077] The computer-readable signal media may include data signals propagated in a baseband or as part of a carrier wave, which carry computer-readable program codes. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal media may also be any computer-readable media other than the computer-readable storage media, and this computer-readable media can send, propagate, or transmit a program for use by or combined with an instruction execution system, device, or component.

[0078] The program codes contained on the computer-readable media can be transmitted by any appropriate media, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0079] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or device. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0080] Note that the above is only a preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments may be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for switching multiple versions of FFmpeg, characterized in that, Including: After the upper-layer application initiates an FFmpeg function call, set the environment variable LD_PRELOAD= / usr / local / libmy_FFmpeg_intercept.so and write it into / etc / profile to make the environment variable take effect permanently. The middle-layer library libmy_FFmpeg_intercept.so receives and intercepts the FFmpeg function calls from the upper-layer application; When it is intercepted that there is a video processing sub-function in the FFmpeg function, use the created global thread-local variable to record whether each process is in the video processing state respectively; Determine the process for video processing according to the local variable, and when the process subsequently calls the video processing sub-function, use the sub-function of the video processing version FFmpeg function to perform video processing.

2. The method according to claim 1, wherein The using the created global thread-local variable to record whether each process is in the video processing state respectively includes: Create a global thread-local variable static __thread int is_video_processing; Initialize the variable of each thread to 0; Judge whether the thread calls the video processing sub-function. When the thread calls the video processing sub-function, modify the variable of the corresponding thread to 1.

3. The method according to claim 2, wherein The judging whether the thread calls the video processing sub-function includes: Judge whether the thread calls the FFmpeg multimedia stream processing sub-function. When calling, read the url parameter of the FFmpeg multimedia stream processing sub-function, determine whether the processing object of the FFmpeg multimedia stream processing sub-function is the processed file or the data format is video, as well as the video attributes, use the url parameter to judge whether the thread calls the video processing sub-function, and the corresponding FFmpeg version.

4. The method according to claim 3, wherein The method further includes: When the process intercepts the FFmpeg multimedia stream cleanup and close sub-function, modify the variable of the process in the global thread-local variable to the initial value.

5. The method according to claim 4, wherein The method further includes: Call the custom close FFmpeg version function unload_custom_FFmpeg() to close the video processing version FFmpeg, and use the dynamic link function dlsym to call the original function of the same name of the system version FFmpeg.

6. The method according to claim 3, wherein The when the process subsequently calls the video processing sub-function, using the sub-function of the video processing version FFmpeg function to perform video processing includes: Call the custom open FFmpeg version function load_custom_FFmpeg() to read the video processing version FFmpeg, and use the dynamic link function dlsym to call the function of the video processing version FFmpeg.

7. The method according to claim 1, wherein The method further includes: After the system version is normally installed, decompress the deb package of the video processing version, and modify the file path to a path different from the system version; Use the dpkg command to repackage and install the decompressed deb package after modifying the path; Redefine and declare all function interfaces of different versions of FFmpeg in the middle layer library, with the same names as the original FFmpeg function interfaces, for intercepting function calls.

8. A multi-version FFmpeg switching device, characterized in that, Including: An interception module, which is used to intercept FFmpeg function calls by using the modified environment variable pointing to the middle layer library after the upper layer application initiates an FFmpeg function call. The interception module includes: A setting unit, which is used to set the environment variable LD_PRELOAD= / usr / local / libmy_FFmpeg_intercept.so and write it into / etc / profile to make the environment variable take effect permanently. The middle layer library libmy_FFmpeg_intercept.so receives and intercepts the calls of the upper layer application to FFmpeg functions; A recording module, which is used to record whether each process is in the video processing state respectively by using the created global thread local variable when a video processing sub-function exists in the intercepted FFmpeg function; An execution module, which is used to determine the process for video processing according to the local variable, and when the subsequent video processing sub-function is called by the process, execute video processing by using the sub-function of the FFmpeg function of the video processing version.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the multi-version FFmpeg switching method according to any one of claims 1-7 when executed by a computer processor.

Images