Video hardware accelerated decoding method, apparatus, computer device, storage medium and program product

Abstract

The application relates to a video hardware acceleration decoding method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: receiving a video decoding processing request, determining a target application and a target chip corresponding to the video decoding processing request, determining a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface, determining a hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, obtaining a video coding interface deployed for the hardware abstraction layer, accessing the video coding interface, calling a video processing unit matched with the video coding interface, performing video decoding on video data corresponding to the video decoding processing request, and obtaining video decoding data. Compatibility between the video processing unit of the target chip and the video acceleration hardware driver is achieved by the method, and targeted decoding processing is realized through the video processing unit, so that the video decoding efficiency and quality are further improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video hardware-accelerated decoding method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of computer technology and the widespread application of various video processing applications, in order to improve the processing efficiency of video processing applications, a method has emerged that deploys third-party interfaces on hardware and calls these interfaces to perform video decoding and encoding, thereby using hardware acceleration to improve the efficiency of video decoding and encoding.

[0003] Traditional technologies typically employ a hardware-based VAAPI (Video Acceleration API) framework, which provides interfaces for various video processing applications. These applications can then call the interfaces provided by the VAAPI framework to achieve hardware-accelerated video decoding.

[0004] However, the traditional VAAPI framework supports GPUs (graphics processing units). Since GPUs are primarily designed for graphics processing, they cannot effectively process video data and require significant resources. As a result, traditional video processing methods still suffer from low efficiency and excessive resource consumption. Summary of the Invention

[0005] Therefore, it is necessary to provide a video hardware-accelerated decoding method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the efficiency of video data decoding and reduce resource consumption, in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a video hardware-accelerated decoding method, comprising: receiving a video decoding processing request; determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip, and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, and performing video decoding on the video data corresponding to the video decoding processing request to obtain video decoded data.

[0007] In one embodiment, the method further includes: configuring a compatibility layer for the target chip according to a context processing component deployed by the video acceleration hardware driver to obtain a configured compatibility layer; feeding back context control information corresponding to the video acceleration hardware driver to a context control component deployed by the hardware abstraction layer through the compatibility layer; and configuring the hardware abstraction layer according to the context control information through the context control component to obtain a configured hardware abstraction layer.

[0008] In one embodiment, the step of configuring the hardware abstraction layer (HAL) according to the context control information through the context control component to obtain the configured HAL includes: obtaining configuration information parsing data corresponding to the context control information through the context control component; constructing a configuration file corresponding to the HAL based on the configuration information parsing data; configuring an image buffer array corresponding to the HAL according to the configuration file; and constructing an interaction context object corresponding to the HAL based on the configuration file and the image buffer array to obtain the configured HAL.

[0009] In one embodiment, the step of invoking a video processing unit matched with the video codec interface to perform video decoding on the video data corresponding to the video decoding processing request to obtain video decoded data includes: invoking a video processing unit matched with the video codec interface to obtain the video data corresponding to the video decoding processing request; generating a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer, and determining a target decoding function corresponding to the video data; performing function decoding processing on the candidate rendering list according to the target decoding function to obtain surface decoding data; and decoding and parsing the surface decoding data to obtain video decoded data.

[0010] In one embodiment, generating a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer includes: when the image buffer array in the hardware abstraction layer is in an idle state, setting the surface state of the image buffer array to a rendering state to obtain a rendering buffer array; storing the video data in the rendering buffer array to obtain a target rendering buffer array carrying the video data; and storing the target rendering buffer array in the rendering list to generate a candidate rendering list.

[0011] In one embodiment, the method further includes: deploying a hardware abstraction layer state monitoring function for the hardware abstraction layer, and calling the hardware abstraction layer state monitoring function to monitor the state of the hardware abstraction layer and obtain the data stream state corresponding to the hardware abstraction layer; determining the state of the output buffer corresponding to the image buffer array in the hardware abstraction layer based on the data stream state; when it is determined based on the state of the output buffer that the output buffer has been filled, obtaining the video decoding data corresponding to the image buffer array, and clearing the input buffer and output buffer corresponding to the image buffer array to switch the image buffer array to an idle state.

[0012] Secondly, this application also provides a video hardware acceleration decoding device, comprising: a target application determination module, configured to receive a video decoding processing request and determine a target application and a target chip corresponding to the video decoding processing request; a video acceleration hardware driver determination module, configured to determine a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface; a video codec interface acquisition module, configured to determine a hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver and acquire a video codec interface deployed for the hardware abstraction layer; and a video decoding data acquisition module, configured to access the video codec interface, call a video processing unit matching the video codec interface, perform video decoding on the video data corresponding to the video decoding processing request, and obtain video decoding data.

[0013] Thirdly, this application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: receiving a video decoding processing request; determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip, and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, and performing video decoding on the video data corresponding to the video decoding processing request to obtain video decoded data.

[0014] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the following steps: receiving a video decoding processing request; determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, and performing video decoding on the video data corresponding to the video decoding processing request to obtain video decoded data.

[0015] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps: receiving a video decoding processing request; determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, and performing video decoding on the video data corresponding to the video decoding processing request to obtain video decoded data.

[0016] In the aforementioned video hardware-accelerated decoding method, apparatus, computer device, computer-readable storage medium, and computer program product, by receiving a video decoding processing request, the target application and target chip corresponding to the video decoding processing request are determined, and the video acceleration interface associated with the target application and the video acceleration hardware driver corresponding to the video acceleration interface are determined. Based on the video acceleration hardware driver, the hardware abstraction layer corresponding to the target chip is determined, thereby achieving compatibility between the video processing unit of the target chip and the video acceleration hardware driver, solving the problem that video decoding processing cannot be directly invoked by calling the video processing unit through the video acceleration hardware driver. Furthermore, by accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit matching the video codec interface is invoked to perform video decoding processing on the video data corresponding to the video decoding processing request, obtaining the video decoded data. Compared with the traditional method of using a graphics processor for video decoding, the targeted decoding processing implemented by the video processing unit further improves the efficiency and quality of video decoding. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a diagram illustrating the application environment of a video hardware-accelerated decoding method in one embodiment.

[0019] Figure 2 This is a flowchart illustrating a video hardware-accelerated decoding method in one embodiment;

[0020] Figure 3 This is a schematic diagram of the process of an upper-layer application calling a video processing unit to perform video decoding in one embodiment.

[0021] Figure 4 This is a flowchart illustrating a video hardware-accelerated decoding method in another embodiment;

[0022] Figure 5 This is a schematic diagram of the data transmission process between the video acceleration hardware driver and the hardware abstraction layer in one embodiment.

[0023] Figure 6 This is a schematic diagram of the process for obtaining a configured hardware abstraction layer in one embodiment;

[0024] Figure 7 This is a schematic diagram illustrating the configuration process of the hardware abstraction layer in one embodiment;

[0025] Figure 8 This is a timing diagram illustrating the configuration of the hardware abstraction layer based on the video acceleration interface, video acceleration hardware driver, and hardware abstraction layer in one embodiment.

[0026] Figure 9 This is a schematic diagram of the process for obtaining video decoding data in one embodiment;

[0027] Figure 10 This is a timing diagram illustrating video decoding processing based on a video acceleration interface, a video acceleration hardware driver, and a hardware abstraction layer in one embodiment.

[0028] Figure 11 This is a flowchart illustrating a video hardware-accelerated decoding method in another embodiment;

[0029] Figure 12 This is a schematic diagram illustrating the process of calling the hardware abstraction layer status monitoring function to monitor the status of the hardware abstraction layer in one embodiment.

[0030] Figure 13 This is a structural block diagram of a video hardware-accelerated decoding device in one embodiment;

[0031] Figure 14 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] The video hardware-accelerated decoding method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, portable wearable devices, and aircraft. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Server 104 can be a standalone physical server, a server cluster consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. In this embodiment, terminal 102 and server 104 can be directly or indirectly connected via wired or wireless communication, and this is not limited in this respect. Both terminal 102 and server 104 can be used independently to execute the video hardware-accelerated decoding method provided in this embodiment, or they can work together to execute the same method. For example, taking the collaborative execution of the video hardware-accelerated decoding method by terminal 102 and server 104 as an example, server 104 receives a video decoding processing request from terminal 102 and determines the target application and target chip corresponding to the request. It then further determines the video acceleration interface associated with the target application and the video acceleration hardware driver corresponding to the interface. Further, server 104, based on the video acceleration hardware driver, determines the hardware abstraction layer corresponding to the target chip and obtains the video codec interface deployed for the hardware abstraction layer. By accessing the video codec interface, server 104 calls the video processing unit matching the interface to decode the video data corresponding to the video decoding processing request, thereby obtaining the decoded video data. After obtaining the video decoding data, the server 104 can send the obtained video decoding data back to the terminal device 102 for display or storage.

[0034] In one exemplary embodiment, such as Figure 2 As shown, a video hardware-accelerated decoding method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S202 to S208. Wherein:

[0035] Step S202: Receive a video decoding processing request and determine the target application and target chip corresponding to the video decoding processing request.

[0036] Specifically, the upper-layer applications deployed on the terminal device, such as video editing applications, video playback applications, and browsers, can trigger video decoding processing requests during actual application based on the user's video editing or playback operations. The terminal feeds back the detected video decoding processing requests to the server. After receiving the video decoding processing requests, the server parses the video decoding processing requests to determine the target application and target chip corresponding to the video decoding processing requests.

[0037] The target application can be specifically understood as the upper-layer application deployed on the terminal device that triggered the video decoding processing request, such as video editing applications, video playback applications, and browsers. The target chip, depending on the specific application scenario and its requirements, can be different types of processor chips, integrating components such as a graphics processing unit (GPU), a video decoding unit (VPU), a display control interface, and a video memory controller to enhance computing and multimedia processing capabilities.

[0038] Step S204: Determine the video acceleration interface associated with the target application and the video acceleration hardware driver corresponding to the video acceleration interface.

[0039] Specifically, after the server identifies the target application corresponding to the video decoding processing request, such as a video editing application, a video playback application, or a browser, it further identifies the video acceleration interface associated with the target application. That is, the video acceleration interface (i.e., Video Acceleration API, or VAAPI) that the target application deployed on the terminal can access during operation to implement hardware acceleration, such as which interfaces in the video acceleration interface library.

[0040] After determining the video acceleration interface associated with the target application, the server further determines the video acceleration hardware driver corresponding to the video acceleration interface. For example, it determines the video acceleration hardware driver that can be accessed by the video acceleration interface that needs to be accessed.

[0041] Step S206: Based on the video acceleration hardware driver, determine the hardware abstraction layer corresponding to the target chip, and obtain the video codec interface deployed for the hardware abstraction layer.

[0042] Specifically, the server can determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, such as the hardware abstraction layer pre-configured for processor chip A, and then further obtain the video codec interface deployed for the hardware abstraction layer.

[0043] The hardware abstraction layer pre-configured for the target chip (such as processor chip A) can be understood as a conversion layer that enables access and invocation of the video processing unit from the video acceleration interface. It is equipped with a video codec interface that can be used to access the video processing unit. By accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit can be used to perform video decoding processing.

[0044] Step S208: Access the video codec interface, call the video processing unit that matches the video codec interface, perform video decoding on the video data corresponding to the video decoding processing request, and obtain video decoding data.

[0045] Specifically, by accessing the video codec interface, the server can call the video processing unit that matches the video codec interface, and obtain the video data corresponding to the video decoding processing request, so as to perform video decoding on the video data through the video processing unit to obtain the video decoded data.

[0046] Specifically, by calling the video processing unit, a candidate rendering list can be generated based on the video data and the image buffer array deployed in the hardware abstraction layer. By determining the target decoding function corresponding to the video data, the candidate rendering list is subjected to function decoding processing according to the target decoding function to obtain surface decoding data. Finally, by decoding and parsing the surface decoding data, the video decoding data corresponding to the video data can be obtained.

[0047] In one exemplary embodiment, such as Figure 3 As shown, this provides a schematic diagram of an upper-layer application calling a video processing unit to perform video decoding processing. Figure 3 It can be seen that during the process of upper-layer applications calling the video processing unit for video decoding, the upper-layer applications, including video editing applications, video playback applications, and browsers, can trigger video decoding requests during actual application. After receiving the video decoding request, the server determines the target application corresponding to the video decoding request (e.g., ...). Figure 3 Video editing applications in China), and target chips (such as...) Figure 3 The X100 chip in the middle), and determine the video acceleration interface associated with the target application (e.g., Figure 3 (Some video acceleration interfaces in the VAAPI library), and the corresponding video acceleration hardware drivers (such as...) Figure 3(VA Hardware Driver in the middle).

[0048] Secondly, based on the video acceleration hardware driver, determine the compatibility layer corresponding to the target chip (e.g., Figure 3 The X100_video_dir layer in the middle), and the hardware abstraction layer corresponding to the target chip (e.g., the compatibility layer) is used to determine the hardware abstraction layer corresponding to the target chip. Figure 3 In the VDEV layer (the hardware abstraction layer), a video codec interface is deployed. This interface is used by the video processing unit (VRAM) to perform video decoding. After obtaining the video codec interface deployed by the hardware abstraction layer, one can access the video codec interface to process video data from the video processing unit layer (e.g., the VDEV layer). Figure 3 In the VPU driver layer (i.e., the video processing unit driver layer), the video processing unit that matches the corresponding video codec interface is called to perform video decoding on the video data corresponding to the video decoding processing request, and obtain the video decoded data.

[0049] In the aforementioned video hardware-accelerated decoding method, by receiving a video decoding request, the target application and target chip corresponding to the request are determined. The video acceleration interface associated with the target application and the corresponding video acceleration hardware driver are also identified. Based on the video acceleration hardware driver, the hardware abstraction layer corresponding to the target chip is determined, thereby achieving compatibility between the target chip's video processing unit and the video acceleration hardware driver. This solves the problem of not being able to directly call the video processing unit for video decoding processing by calling the video acceleration hardware driver. Furthermore, by accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit matching the video codec interface is called to perform video decoding processing on the video data corresponding to the video decoding request, obtaining the video decoded data. Compared to the traditional method of using a graphics processor for video decoding, this targeted decoding processing through the video processing unit further improves video decoding efficiency and quality.

[0050] In one exemplary embodiment, such as Figure 4 As shown, a video hardware-accelerated decoding method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S402 to S406. Wherein:

[0051] Step S402: Configure the compatibility layer of the target chip according to the context processing component deployed by the video acceleration hardware driver to obtain the configured compatibility layer.

[0052] Before determining the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, the server needs to pre-configure the hardware abstraction layer and deploy a video codec interface for the hardware abstraction layer. When video decoding is required, the video processing unit can be invoked to perform video decoding by accessing the video codec interface deployed in the hardware abstraction layer.

[0053] Specifically, during the hardware abstraction layer process, the server involves compatibility layer configuration. This is achieved by calling the context processing component deployed by the video acceleration hardware driver to perform compatibility layer configuration, including configuring the compatibility layer and configuring the compatibility layer buffer.

[0054] Specifically, configuring the compatibility layer allows the acquisition of a compatibility layer context. This context enables the transmission of context control information corresponding to the video acceleration hardware driver to the hardware abstraction layer (HAL). Furthermore, configuring the compatibility layer buffer allows the storage of compatibility layer surface data. This data can then be sent to the HAL context component during HAL configuration to facilitate HAL configuration.

[0055] In step S404, the context control information corresponding to the video acceleration hardware driver is fed back to the context control component deployed in the hardware abstraction layer through the compatibility layer.

[0056] Specifically, after configuring the compatibility layer corresponding to the target chip, the server feeds back the context control information corresponding to the video acceleration hardware driver to the context control component deployed in the hardware abstraction layer through the compatibility layer. Specifically, the context control information corresponding to the video acceleration hardware driver is sent to the context control component deployed in the hardware abstraction layer through the compatibility layer context.

[0057] Step S406: Configure the hardware abstraction layer using the context control component based on the context control information to obtain the configured hardware abstraction layer.

[0058] Specifically, the server, through the configured compatibility layer, feeds back the context control information corresponding to the video acceleration hardware driver to the context control component deployed in the hardware abstraction layer. Then, by calling the context control component deployed in the hardware abstraction layer, the server configures the abstraction layer according to the context control information corresponding to the video acceleration hardware driver, and obtains the configured hardware abstraction layer.

[0059] After configuring the hardware abstraction layer, a video codec interface for calling the video processing unit is further deployed on the hardware abstraction layer, so that the video processing unit can be called to perform video decoding processing by accessing the video codec interface.

[0060] In one exemplary embodiment, such as Figure 5 The diagram illustrates a data transmission process between a video acceleration hardware driver and a hardware abstraction layer. Figure 5 It is known that the video acceleration hardware driver deploys a context processing component, which can configure the compatibility layer and the compatibility layer buffer. By configuring the compatibility layer through the context processing component deployed in the video acceleration hardware driver, a compatibility layer context can be obtained. Using this compatibility layer context, context control information corresponding to the video acceleration hardware driver can be sent to the context control component deployed in the hardware abstraction layer. Furthermore, after configuring the compatibility layer buffer through the context processing component deployed in the video acceleration hardware driver, compatibility layer surface data can be stored in the compatibility layer buffer. This compatibility surface data can then be sent to the context component deployed in the hardware abstraction layer during hardware abstraction layer configuration. Finally, the context control component deployed in the hardware abstraction layer configures the hardware abstraction layer based on the context control information and the compatibility layer surface data, thus obtaining the configured hardware abstraction layer.

[0061] In this embodiment, the target chip is configured with a compatibility layer based on the context processing component deployed by the video acceleration hardware driver. This results in a pre-configured compatibility layer. Through this compatibility layer, context control information corresponding to the video acceleration hardware driver is fed back to the context control component deployed by the hardware abstraction layer. The context control component then configures the hardware abstraction layer based on the context control information, resulting in a pre-configured hardware abstraction layer. This achieves compatibility between the target chip's video processing unit and the video acceleration hardware driver, resolving the issue of not being able to directly call the video processing unit for video decoding by calling the video acceleration hardware driver. After obtaining the pre-configured hardware abstraction layer, video decoding can be implemented based on it, improving video decoding efficiency and quality.

[0062] In one exemplary embodiment, such as Figure 6 As shown, the steps to obtain a configured Hardware Abstraction Layer (HAL) are as follows: The HAL is configured using the context control component based on context control information. Specifically, these steps include S602 to S608.

[0063] Step S602: Obtain the configuration information parsing data corresponding to the context control information through the context control component.

[0064] Specifically, the server obtains configuration information parsing data corresponding to the context control information through the context control component deployed in the hardware abstraction layer. Specifically, this data can be parsed from the config file (a file used to store configuration information) provided by the target application such as ffmpeg (i.e., video editing software).

[0065] Step S604: Parse the data based on the configuration information and construct the configuration file corresponding to the hardware abstraction layer.

[0066] Specifically, the server uses the context control component deployed in the hardware abstraction layer to parse data based on configuration information (such as config parsing data) and construct a configuration file corresponding to the hardware abstraction layer (for example, object_config, which can be understood as the object (i.e., hardware abstraction layer) configuration file).

[0067] Step S606: Configure the image buffer array corresponding to the hardware abstraction layer according to the configuration file.

[0068] Specifically, the server configures the image buffer array (e.g., surface array, which can be understood as being used) corresponding to the hardware abstraction layer by the context control component deployed through the hardware abstraction layer, according to the configuration file (e.g., object_config) corresponding to the hardware abstraction layer.

[0069] Step S608: Based on the configuration file and image buffer array, construct the interaction context object corresponding to the hardware abstraction layer, and obtain the configured hardware abstraction layer.

[0070] Specifically, the server uses the context control component deployed in the hardware abstraction layer to construct an interaction context object (e.g., object_context) corresponding to the hardware abstraction layer based on the configuration file (e.g., object_config) and the image buffer array (e.g., surface array). By constructing the configuration file, configuring the image buffer array, and constructing the interaction context object, the server completes the configuration of the hardware abstraction layer and obtains the configured hardware abstraction layer.

[0071] In one exemplary embodiment, such as Figure 7 As shown, a configuration process for a hardware abstraction layer is provided, refer to Figure 7 It can be seen that the configuration process of the hardware abstraction layer specifically involves 0-information initialization, 1-configuration file construction, 2-image cache array construction, 3-interaction context object construction, and 4-image buffer construction, among which:

[0072] P0, 0 - Information Initialization: Before obtaining the configuration information parsing data corresponding to the context control information, the server also needs to process the request data corresponding to the video decoding request (i.e., Figure 7 In request_data), context control information (i.e. Figure 7The initialization process is performed on the sVDEVContext object in the request data. Specifically, the malloc function (i.e., the dynamic memory allocation function) in the request data allocates a stack in the memory of the central processing unit to complete the initialization process of the context control information corresponding to the video acceleration hardware driver.

[0073] Furthermore, the server can obtain configuration information parsing data corresponding to the context control information through the context control component deployed in the hardware abstraction layer. Specifically, this data can be parsed data from the config file (a file used to store configuration information) provided by the target application such as ffmpeg (i.e., video editing software).

[0074] P1, 1-Configuration file construction: The context control component deployed through the hardware abstraction layer parses data based on the configuration information (i.e., config parsing data) and constructs the configuration file corresponding to the hardware abstraction layer (i.e., ...). Figure 7 (object_config in the context). Specifically, the object_config configuration file corresponding to the hardware abstraction layer can be used to further initialize and configure the data streams corresponding to the context control information.

[0075] P2, 2-Image Buffer Array Construction: The server configures the image buffer array corresponding to the Hardware Abstraction Layer (i.e., according to the configuration file) to obtain the image buffer array. Figure 7 (object_surface in the context). (Refer to...) Figure 7 It can be seen that, for the image buffer array object_surface, the method is to create the image buffer array object_surface and then use the image buffer array object_surface to obtain the decoded data according to the context control information corresponding to the hardware abstraction layer.

[0076] Among them, reference Figure 7 It can be seen that, for the image buffer array `object_surface`, the currently rendered video data and the image buffer array `object_surface` can be placed into the candidate rendering list. After rendering is complete, the `render_surface` (i.e., the image buffer array whose surface state is rendering) in the interaction context object (i.e., `object_context`) is cleared. The surface decoding data obtained by decoding the candidate rendering list is stored in the context control information corresponding to the hardware abstraction layer and can be copied to the image buffer array `object_surface`.

[0077] P3.3 Interaction Context Object Construction: The server constructs the interaction context object corresponding to the hardware abstraction layer (i.e., based on the configuration file and image buffer array) according to the configuration file and image buffer array. Figure 7(object_context in the text). (Refer to...) Figure 7 It can be seen that when creating the interaction context object object_context, the handle is obtained and the handle of the interaction context object object_context is saved to the context control information corresponding to the hardware abstraction layer. At the same time, the context object being processed in the current video acceleration hardware driver is also stored in the context control information corresponding to the hardware abstraction layer.

[0078] Among them, reference Figure 7 It can be seen that when creating the interaction context object `object_context`, the configuration information corresponding to the image cache is obtained from the context control information corresponding to the hardware abstraction layer, and this information is used to configure the default parameters of the image cache area. Furthermore, based on the configuration information corresponding to the image cache, the image buffer array `object_surface` can be further populated with array source data.

[0079] P4, 4-Image Buffer Construction: After constructing the interaction context object, the image buffer is further constructed (i.e. Figure 7 The `object_buffer` in the image buffer array is used to store data. The `object_buffer` is used only to store initial data. The `object_surface` array is built before the `object_buffer`, but during image rendering, data from the `object_buffer` is transferred to the corresponding part of the `object_surface` array as input for data decoding. Decoded data is also placed into the corresponding part of the `object_surface` array via a callback function. The `object_surface` array is used for object data management, while the `object_buffer` array is used to temporarily store input data.

[0080] In one exemplary embodiment, such as Figure 8 As shown, this provides a schematic diagram of hardware abstraction layer configuration based on video acceleration interface, video acceleration hardware driver, and hardware abstraction layer. (Refer to...) Figure 8 It can be seen that the process of configuring the hardware abstraction layer includes:

[0081] 1) The target application sends the request data corresponding to the video decoding processing request to the video acceleration hardware driver through the video acceleration interface in the video acceleration interface framework. Specifically, the target application can be a video editing application, the video acceleration interface framework can be the VAAPI framework, the video acceleration hardware driver can be X100_driver, and the hardware abstraction layer can be the VDEV layer. 2) The video acceleration hardware driver allocates and initializes the request data corresponding to the video decoding processing request based on the hardware abstraction layer's registration function table. 3) The video acceleration hardware driver obtains the configuration information parsing data corresponding to the context control information. When the data type of the configuration information parsing data is the target type, the hardware abstraction layer's data stream is initialized. 4) The video acceleration hardware driver constructs the configuration file corresponding to the hardware abstraction layer based on the configuration information parsing data. 5) The video acceleration hardware driver configures the image buffer array corresponding to the hardware abstraction layer based on the configuration file and initializes the image buffer array according to the height and width information in the configuration file. 6) The video acceleration hardware driver constructs the interaction context object corresponding to the hardware abstraction layer based on the configuration file and the image buffer array. 7) The video acceleration hardware driver is based on the Hardware Abstraction Layer (HAL) and deploys HAL status monitoring functions to monitor the data flow status of the HAL. 8) After constructing the interaction context object, the video acceleration hardware driver builds an image buffer to store data, completing the configuration of the HAL.

[0082] In this embodiment, the configuration information parsing data corresponding to the context control information is obtained through the context control component. Based on the configuration information parsing data, a configuration file corresponding to the hardware abstraction layer is constructed. According to the configuration file, an image buffer array corresponding to the hardware abstraction layer is configured. Based on the configuration file and the image buffer array, an interaction context object corresponding to the hardware abstraction layer is constructed, and the configured hardware abstraction layer is obtained. This achieves compatibility between the target chip's video processing unit and the video acceleration hardware driver, solving the problem that the video processing unit cannot be directly invoked for video decoding by calling the video acceleration hardware driver. After obtaining the configured hardware abstraction layer, video decoding can be implemented based on the hardware abstraction layer, improving video decoding efficiency and quality.

[0083] In one exemplary embodiment, such as Figure 9 As shown, the steps for obtaining video decoding data, namely, calling the video processing unit matched with the video encoding / decoding interface to perform video decoding on the video data corresponding to the video decoding processing request, specifically include the following steps S902 to S906. Wherein:

[0084] Step S902: Call the video processing unit that matches the video codec interface to obtain the video data corresponding to the video decoding processing request. Based on the video data and the image buffer array in the hardware abstraction layer, generate a candidate rendering list and determine the target decoding function corresponding to the video data.

[0085] Specifically, the server accesses the video codec interface deployed in the hardware abstraction layer, calls the video processing unit that matches the video codec interface, obtains the video data corresponding to the video decoding processing request, and performs video decoding processing on the video data corresponding to the video decoding processing request to obtain video decoded data.

[0086] Specifically, for the video data corresponding to the video decoding request, a candidate rendering list is generated based on the video data and the image buffer array in the hardware abstraction layer, and then further decoded on the candidate rendering list. After obtaining the video data corresponding to the video decoding request, a target decoding function corresponding to the video data is further determined, and the candidate rendering list is decoded using the target decoding function.

[0087] In one exemplary embodiment, a candidate rendering list is generated based on video data and an image buffer array in the hardware abstraction layer, including:

[0088] When the image buffer array in the hardware abstraction layer is in an idle state, the surface state of the image buffer array is set to the rendering state to obtain the rendering buffer array; the video data is stored in the rendering buffer array to obtain the target rendering buffer array carrying the video data; the target rendering buffer array is stored in the rendering list to generate the candidate rendering list.

[0089] Specifically, when the image buffer array in the hardware abstraction layer is in an idle state, that is, when the image buffer array in the current hardware abstraction layer is in a usable state, the surface state of the image buffer array is set to the rendering state, the rendering buffer array is obtained, and the currently rendered surface is marked as the image buffer array (i.e., the rendering buffer array) with the surface state set to the rendering state in the interaction context object.

[0090] The video data corresponding to the video decoding request can be first stored in the image buffer. When video decoding is required, the video data is copied from the image buffer to the rendering buffer array to obtain the target rendering buffer array carrying the video data. Further, by obtaining a pre-built rendering list and storing the target rendering buffer array in the pre-built rendering list, a candidate rendering list is generated. By decoding the candidate rendering list, the actual required video decoding data can be obtained.

[0091] Step S904: Perform function decoding processing on the candidate rendering list according to the target decoding function to obtain surface decoding data.

[0092] Specifically, the server determines the target decoding function corresponding to the video data and calls the target decoding function to perform function decoding processing on the candidate rendering list according to the target decoding function, thereby obtaining surface decoding data. Simultaneously with obtaining the surface decoding data, i.e., after the function decoding processing is completed, the surface state of the target rendering buffer array in the candidate rendering list is set to an idle state, and the image buffer array (i.e., the rendering buffer array) whose surface state is currently being rendered in the interaction context object is cleared.

[0093] Step S906: Decode and parse the surface decoding data to obtain video decoding data.

[0094] Specifically, the server can obtain video decoded data corresponding to the video data by further decoding and parsing the surface decoded data. Specifically, the further decoding and parsing of the surface decoded data can be achieved by setting the surface decoded data to start in a parsing state and setting the stop point as the start node of the processing sequence.

[0095] Specifically, for the decoding and parsing process, when the surface decoding data is submitted for the first time, a processing node is created to start the sequence and trigger a stop callback event (i.e., to complete the current decoding and parsing process). At the same time, multiple image processing nodes are also created to perform decoding and parsing processing on the surface decoding data to obtain video decoding data.

[0096] In one exemplary embodiment, such as Figure 10 The diagram illustrates a method for implementing video decoding based on a video acceleration interface, video acceleration hardware driver, and hardware abstraction layer. Figure 10 It can be seen that the video decoding process specifically includes:

[0097] P1. The first stage, also known as the start stage, represents the stage of registering the current image buffer array surface to the context interaction object. Specifically: 1) The target application sends the video data corresponding to the video decoding processing request to the video acceleration hardware driver through the video acceleration interface. When sending video data to the video acceleration hardware driver, the image buffer array number and the interaction context object number are also sent to the driver. The video acceleration hardware driver can then determine whether the corresponding image buffer array is idle based on the image buffer array number. 2) When the video acceleration hardware driver determines that the image buffer array in the hardware abstraction layer is idle, it sets the surface state of the image buffer array to the rendering state, obtaining the rendering buffer array. In the interaction context object, it marks the currently rendered surface as the image buffer array with the surface state set to the rendering state (i.e., the rendering buffer array).

[0098] P2. The second stage, the rendering stage, involves copying data from the image buffer to the image buffer array surface. Specifically: 3) The target application sends the interaction context object number and the image buffer number to the video acceleration hardware driver via the video acceleration interface. 4) The video acceleration hardware driver copies the video data from the image buffer corresponding to the image buffer number to the rendering image buffer array corresponding to the interaction context object number, thus obtaining the target rendering buffer array carrying the video data.

[0099] P3. The third stage, the decoding stage, represents the stage where the decoding function in the Hardware Abstraction Layer (VDEV) is called to decode the current image buffer array `surface`. Specifically: 5) The video acceleration hardware driver stores the target rendering buffer array into a pre-configured rendering list, obtaining a candidate rendering list, and calls the target decoding data corresponding to the video data to perform function decoding and parsing on the candidate rendering list to obtain the surface decoding data. 6) The video acceleration hardware driver sets the surface state of the target rendering buffer array in the candidate rendering list to an idle state and clears the currently rendered image buffer array in the interaction context object. 7) The video acceleration hardware driver sets the parsing method for the surface decoding data, with the parsing start point set as the parsing state and the stop point set as the parsing sequence start node. 8) For the hardware abstraction layer, when the surface decoding data is submitted for the first time, the video acceleration hardware driver creates a processing node to trigger a stop callback event (i.e., completes the current decoding and parsing process), and creates multiple image processing nodes to perform decoding and parsing processing on the surface decoding data to obtain the video decoding data.

[0100] In this embodiment, by calling a video processing unit that matches the video codec interface, video data corresponding to the video decoding processing request is obtained. Based on the video data and the image buffer array in the hardware abstraction layer, a candidate rendering list is generated, and a target decoding function corresponding to the video data is determined. According to the target decoding function, the candidate rendering list is subjected to function decoding processing to obtain surface decoding data. The surface decoding data is then decoded and parsed to obtain video decoding data. Compared with the traditional method of using a graphics processor for video decoding, the targeted decoding processing implemented by the video processing unit further improves the efficiency and quality of video decoding.

[0101] In one exemplary embodiment, such as Figure 11 As shown, a video hardware-accelerated decoding method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S1102 to S1106. Wherein:

[0102] Step S1102: For the hardware abstraction layer, deploy the hardware abstraction layer status monitoring function and call the hardware abstraction layer status monitoring function to monitor the status of the hardware abstraction layer and obtain the data flow status corresponding to the hardware abstraction layer.

[0103] Specifically, the server deploys a hardware abstraction layer (HAL) state monitoring function (i.e., the VDEV_Callback function) for the HAL. By calling the HAL state monitoring function, the server monitors the state of the HAL and obtains the data stream state corresponding to the HAL. The data stream state corresponding to the HAL includes the following states: the input buffer of the image buffer array is empty; the output buffer of the image buffer array is full; and the data stream has stopped.

[0104] In this context, an empty input buffer of an image buffer array means that there is currently no new input data to be input and processed. The mapping of the image buffer array to the input address should be removed, and the image buffer array should be discarded, awaiting the next remapping and input data. Conversely, a full output buffer of an image buffer array means that the input data has been decoded by the Hardware Abstraction Layer (VDEV). The decoded data can be copied out, and the output and input areas of the image buffer array should be cleared, putting the image buffer array into a ready-to-use (idle) state, ready to receive the next segment of data for subsequent processing.

[0105] The data stream stopping state specifically includes active stopping and passive stopping. Active stopping can be understood as the process of creating a data stream corresponding to the hardware abstraction layer by specifying that the created type only includes parsing and decoding, and specifying a stopping point in the parsing state, typically set to the processing node that submits the first piece of data. When data is submitted, a stop callback function is triggered, allowing for stream attribute configuration and output within the callback function. Passive stopping, on the other hand, is triggered when a fault or problem occurs, such as a system error involving segment dumping.

[0106] Step S1104: Determine the state of the output buffer corresponding to the image buffer array in the hardware abstraction layer based on the data stream state.

[0107] Specifically, after obtaining the data stream status corresponding to the hardware abstraction layer, the server further determines the status of the output buffer corresponding to the image buffer array in the hardware abstraction layer based on the data stream status, such as determining that the output buffer corresponding to the image buffer array is empty or that the output buffer corresponding to the image buffer array has been filled.

[0108] Step S1106: When it is determined that the output buffer is full based on the state of the output buffer, the video decoding data corresponding to the image buffer array is obtained, and the input buffer and output buffer corresponding to the image buffer array are cleared to switch the image buffer array to an idle state.

[0109] Specifically, when the server determines that the output buffer corresponding to the image buffer array has been filled based on the state of the output buffer, it obtains the video decoding data corresponding to the image buffer array and feeds it back to the terminal for subsequent processing such as display and playback.

[0110] After obtaining the video decoding data corresponding to the image buffer array, the input buffer and output buffer corresponding to the image buffer array are further cleared to switch the image buffer array to an idle state, so that the subsequent processing flow can call the image buffer array for data storage or data encoding and decoding processing, etc.

[0111] In one exemplary embodiment, such as Figure 12 As shown, this provides a process for calling the hardware abstraction layer state monitoring function to perform state monitoring of the hardware abstraction layer, referring to... Figure 12 It can be seen that when the hardware abstraction layer state monitoring function is called to monitor the state of the hardware abstraction layer, the data flow state corresponding to the hardware abstraction layer obtained can specifically include the state where the input buffer of the image buffer array is empty, the state where the output buffer of the image buffer array is full, and the state where the data flow stops. Among them:

[0112] P1. Regarding the state where the input buffer of the image buffer array is empty: Specifically, this means that there is currently no new input data to be input and processed for the image buffer array. The mapping of the image buffer array to the input address should be removed, the image buffer array should be discarded, and it should wait for the next remapping and input data.

[0113] P2. The state where the output buffer of the image buffer array is full: This means that the input data of the image buffer array has been decoded by the hardware abstraction layer VDEV. The decoded data can be copied out from the output buffer of the image buffer array, and the output and input areas of the image buffer array are cleared, so that the image buffer array is in the preparation stage (i.e., an idle state that can be used) and ready to receive the next segment of data for subsequent processing.

[0114] P3. Regarding data stream stopping states: This includes active stopping and passive stopping. Active stopping can be understood as the creation of a data stream corresponding to the hardware abstraction layer, where the creation type is specified to include only parsing and decoding, and a stopping point is specified in the parsing state, typically set to the processing node that submits the first piece of data. When data is submitted, a stop callback function is triggered, allowing for stream attribute configuration and output within the callback function. Passive stopping, on the other hand, is triggered when a fault or problem occurs, such as a system error involving segment dumping.

[0115] Specifically, by analyzing the data stream's stopping state, the cause of the data stream stopping can be determined. For example, if it matches the stopping point specified in the parsing state, i.e., it is due to configuration information, then the reconfiguration function is triggered. At the same time, further processing such as configuring some stream attributes and output content is performed in the callback function.

[0116] In the aforementioned video hardware-accelerated decoding method, a hardware abstraction layer (HAL) state monitoring function is deployed and called to monitor the state of the HAL, obtain the data stream state corresponding to the HAL, and when it is determined from the data stream state that the output buffer corresponding to the image buffer array in the HAL is full, the video decoding data corresponding to the image buffer array is obtained, and the input and output buffers corresponding to the image buffer array are cleared to switch the image buffer array to an idle state. This allows the state of the image buffer array to be modified and adjusted according to the data stream state during actual processing, so as to facilitate subsequent calls and video processing, reduce resource waste, and improve video processing efficiency.

[0117] In one exemplary embodiment, an overall flow of a video hardware-accelerated decoding method is provided, including the following processing steps:

[0118] 1) Receive a video decoding request and determine the target application and target chip corresponding to the request. 2) Determine the video acceleration interface associated with the target application and the corresponding video acceleration hardware driver. 3) Configure the compatibility layer for the target chip based on the context processing component deployed by the video acceleration hardware driver, obtaining the configured compatibility layer. 4) Through the compatibility layer, feed back the context control information corresponding to the video acceleration hardware driver to the context control component deployed in the hardware abstraction layer. 5) Obtain the configuration information parsing data corresponding to the context control information through the context control component. 6) Construct a configuration file corresponding to the hardware abstraction layer based on the parsed configuration information. 7) Configure the image buffer array corresponding to the hardware abstraction layer using the configuration file. 8) Construct the interaction context object corresponding to the hardware abstraction layer using the configuration file and the image buffer array, obtaining the configured hardware abstraction layer. 9) Based on the video acceleration hardware driver, determine the hardware abstraction layer corresponding to the target chip and obtain the video codec interface deployed for the hardware abstraction layer. 10) Access the video codec interface using the video processing unit matching the interface to obtain the video data corresponding to the video decoding request. 11) Deploy a hardware abstraction layer (HAL) state monitoring function. 12) Call the HAL state monitoring function to monitor the state of the HAL and obtain the data stream state corresponding to the HAL. 13) Determine the state of the output buffer corresponding to the image buffer array in the HAL based on the data stream state. 14) When the output buffer is determined to be full based on its state, obtain the video decoding data corresponding to the image buffer array, and clear the input and output buffers corresponding to the image buffer array to switch the image buffer array to an idle state. 15) When the image buffer array in the HAL is in an idle state, set the surface state of the image buffer array to the rendering state to obtain the rendering buffer array. 16) Store the video data in the rendering buffer array to obtain the target rendering buffer array carrying the video data. Store the target rendering buffer array in the rendering list to generate a candidate rendering list. 17) Determine the target decoding function corresponding to the video data. Based on the target decoding function, perform function decoding processing on the candidate rendering list to obtain surface decoding data. 18) Decode and parse the surface decoding data to obtain the video decoding data.

[0119] The aforementioned video hardware-accelerated decoding method achieves compatibility between the target chip's video processing unit and the video acceleration hardware driver, resolving the issue that the video processing unit cannot be directly invoked for video decoding by calling the video acceleration hardware driver. Instead, it accesses the video codec interface deployed in the hardware abstraction layer to invoke the video processing unit that matches the video codec interface, thereby performing video decoding processing on the video data corresponding to the video decoding request and obtaining the video decoded data. Compared to the traditional method of using a graphics processor for video decoding, this targeted decoding processing through the video processing unit further improves video decoding efficiency and quality.

[0120] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0121] Based on the same inventive concept, this application also provides a video hardware acceleration decoding apparatus for implementing the video hardware acceleration decoding method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more video hardware acceleration decoding apparatus embodiments provided below can be found in the limitations of the video hardware acceleration decoding method described above, and will not be repeated here.

[0122] In one exemplary embodiment, such as Figure 13 As shown, a video hardware-accelerated decoding device is provided, including: a target application determination module 1302, a video acceleration hardware driver determination module 1304, a video encoding / decoding interface acquisition module 1306, and a video decoding data acquisition module 1308, wherein:

[0123] The target application determination module 1302 is used to receive a video decoding processing request and determine the target application and target chip corresponding to the video decoding processing request; the video acceleration hardware driver determination module 1304 is used to determine the video acceleration interface associated with the target application and the video acceleration hardware driver corresponding to the video acceleration interface; the video codec interface acquisition module 1306 is used to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver and obtain the video codec interface deployed for the hardware abstraction layer; the video decoding data acquisition module 1308 is used to access the video codec interface, call the video processing unit matching the video codec interface, perform video decoding on the video data corresponding to the video decoding processing request, and obtain the video decoding data.

[0124] In the aforementioned video hardware-accelerated decoding device, by receiving a video decoding processing request, the target application and target chip corresponding to the request are determined. The video acceleration interface associated with the target application and the corresponding video acceleration hardware driver are also identified. Based on the video acceleration hardware driver, the hardware abstraction layer corresponding to the target chip is determined, thereby achieving compatibility between the target chip's video processing unit and the video acceleration hardware driver. This solves the problem of not being able to directly call the video processing unit for video decoding processing by calling the video acceleration hardware driver. Furthermore, by accessing the video codec interface deployed in the hardware abstraction layer, the video processing unit matching the video codec interface is called to perform video decoding processing on the video data corresponding to the video decoding processing request, obtaining the video decoded data. Compared to the traditional method of using a graphics processor for video decoding, this targeted decoding processing through the video processing unit further improves video decoding efficiency and quality.

[0125] In one exemplary embodiment, a video hardware-accelerated decoding device is provided, further comprising a hardware abstraction layer configuration module, configured to: configure a compatibility layer for a target chip according to a context processing component deployed by a video acceleration hardware driver, thereby obtaining a configured compatibility layer; feed back context control information corresponding to the video acceleration hardware driver to a context control component deployed by the hardware abstraction layer through the compatibility layer; and configure the hardware abstraction layer according to the context control information through the context control component, thereby obtaining a configured hardware abstraction layer.

[0126] In an exemplary embodiment, the hardware abstraction layer configuration module is further configured to: obtain configuration information parsing data corresponding to the context control information through the context control component; construct a configuration file corresponding to the hardware abstraction layer based on the configuration information parsing data; configure an image buffer array corresponding to the hardware abstraction layer according to the configuration file; construct an interaction context object corresponding to the hardware abstraction layer based on the configuration file and the image buffer array, and obtain the configured hardware abstraction layer.

[0127] In an exemplary embodiment, the video decoding data acquisition module is further configured to: invoke a video processing unit matching the video encoding / decoding interface to obtain video data corresponding to the video decoding processing request; generate a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer, and determine the target decoding function corresponding to the video data; perform function decoding processing on the candidate rendering list according to the target decoding function to obtain surface decoding data; and perform decoding and parsing on the surface decoding data to obtain video decoding data.

[0128] In an exemplary embodiment, the video decoding data acquisition module is further configured to: when the image buffer array in the hardware abstraction layer is in an idle state, set the surface state of the image buffer array to a rendering state to obtain a rendering buffer array; store the video data in the rendering buffer array to obtain a target rendering buffer array carrying the video data; and store the target rendering buffer array in a rendering list to generate a candidate rendering list.

[0129] In an exemplary embodiment, a video hardware-accelerated decoding device is provided, further comprising a state monitoring module, configured to: deploy a hardware abstraction layer state monitoring function for the hardware abstraction layer, and call the hardware abstraction layer state monitoring function to perform state monitoring on the hardware abstraction layer and obtain the data stream state corresponding to the hardware abstraction layer; determine the state of the output buffer corresponding to the image buffer array in the hardware abstraction layer based on the data stream state; when it is determined based on the state of the output buffer that the output buffer has been filled, obtain the video decoding data corresponding to the image buffer array, and clear the input buffer and output buffer corresponding to the image buffer array to switch the image buffer array to an idle state.

[0130] Each module in the aforementioned video hardware acceleration decoding device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0131] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 14As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data such as target applications, target chips, video acceleration interfaces, hardware abstraction layers, video codec interfaces, video processing units, video data, and video decoding data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a video hardware-accelerated decoding method.

[0132] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0133] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0134] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0135] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0139] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A video hardware-accelerated decoding method, characterized in that, The method includes: Receive a video decoding processing request, and determine the target application and target chip corresponding to the video decoding processing request; Determine the video acceleration interface associated with the target application, and the video acceleration hardware driver corresponding to the video acceleration interface; Based on the video acceleration hardware driver, determine the hardware abstraction layer corresponding to the target chip, and obtain the video codec interface deployed for the hardware abstraction layer; Access the video codec interface, call the video processing unit that matches the video codec interface, perform video decoding on the video data corresponding to the video decoding processing request, and obtain video decoded data; The method further includes: Based on the context processing component deployed by the video acceleration hardware driver, the target chip is configured with a compatibility layer to obtain a configured compatibility layer; through the compatibility layer, the context control information corresponding to the video acceleration hardware driver is fed back to the context control component deployed by the hardware abstraction layer; through the context control component, the hardware abstraction layer is configured based on the context control information to obtain a configured hardware abstraction layer.

2. The method according to claim 1, characterized in that, The step of configuring the hardware abstraction layer (HAL) based on the context control information through the context control component to obtain the configured HAL includes: The context control component is used to obtain configuration information parsing data corresponding to the context control information. Parse the data based on the configuration information and construct a configuration file corresponding to the hardware abstraction layer; According to the configuration file, an image buffer array corresponding to the hardware abstraction layer is configured; Based on the configuration file and the image buffer array, an interaction context object corresponding to the hardware abstraction layer is constructed, and the configured hardware abstraction layer is obtained.

3. The method according to claim 1 or 2, characterized in that, The step of calling a video processing unit that matches the video encoding / decoding interface to perform video decoding on the video data corresponding to the video decoding processing request and obtain video decoded data includes: The video processing unit matching the video codec interface is invoked to obtain the video data corresponding to the video decoding processing request. Based on the video data and the image buffer array in the hardware abstraction layer, a candidate rendering list is generated, and the target decoding function corresponding to the video data is determined. According to the target decoding function, the candidate rendering list is subjected to function decoding processing to obtain surface decoding data; The surface decoding data is decoded and parsed to obtain video decoding data.

4. The method according to claim 3, characterized in that, The process of generating a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer includes: When the image buffer array in the hardware abstraction layer is in an idle state, the surface state of the image buffer array is set to the rendering state to obtain the rendering buffer array; The video data is stored in the rendering buffer array to obtain a target rendering buffer array carrying the video data; The target rendering buffer array is stored in the rendering list to generate a candidate rendering list.

5. The method according to claim 1 or 2, characterized in that, The method further includes: For the hardware abstraction layer, a hardware abstraction layer status monitoring function is deployed and called to monitor the status of the hardware abstraction layer and obtain the data flow status corresponding to the hardware abstraction layer. Based on the data stream state, determine the state of the output buffer corresponding to the image buffer array in the hardware abstraction layer; When it is determined that the output buffer is full based on the state of the output buffer, the video decoding data corresponding to the image buffer array is obtained, and the input buffer and output buffer corresponding to the image buffer array are cleared to switch the image buffer array to an idle state.

6. A video hardware-accelerated decoding device, characterized in that, The device includes: The target application determination module is used to receive a video decoding processing request and determine the target application and target chip corresponding to the video decoding processing request. A video acceleration hardware driver determination module is used to determine the video acceleration interface associated with the target application and the video acceleration hardware driver corresponding to the video acceleration interface. The video codec interface acquisition module is used to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, and to acquire the video codec interface deployed for the hardware abstraction layer. The video decoding data acquisition module is used to access the video encoding and decoding interface, call the video processing unit that matches the video encoding and decoding interface, perform video decoding on the video data corresponding to the video decoding processing request, and obtain video decoding data. The device further includes a hardware abstraction layer configuration module, configured to: configure the target chip for compatibility layer according to the context processing component deployed by the video acceleration hardware driver, and obtain a configured compatibility layer; feed back the context control information corresponding to the video acceleration hardware driver to the context control component deployed by the hardware abstraction layer through the compatibility layer; and configure the hardware abstraction layer according to the context control information through the context control component, and obtain a configured hardware abstraction layer.

7. The apparatus according to claim 6, characterized in that, The hardware abstraction layer configuration module is also used for: The context control component obtains configuration information parsing data corresponding to the context control information; based on the configuration information parsing data, a configuration file corresponding to the hardware abstraction layer is constructed; and according to the configuration file, an image buffer array corresponding to the hardware abstraction layer is configured. Based on the configuration file and the image buffer array, an interaction context object corresponding to the hardware abstraction layer is constructed, and the configured hardware abstraction layer is obtained.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images