Video hardware accelerated decoding method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a video hardware accelerated decoding method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: receiving a video decoding processing request, determining a target application and a target chip corresponding to the video decoding processing request, determining a video acceleration interface associated with the target application and a video acceleration hardware drive corresponding to the video acceleration interface, and based on the video acceleration hardware drive, carrying out video decoding processing on the target chip. Determining a hardware abstraction layer corresponding to the target chip, obtaining a video coding and decoding interface deployed for the hardware abstraction layer, accessing the video coding and decoding interface, calling a video processing unit matched with the video coding and decoding interface, and performing video decoding on video data corresponding to the video decoding processing request to obtain video decoding data. By adopting the method, compatibility between the video processing unit and the video acceleration hardware drive of the target chip is realized, targeted decoding processing is realized through the video processing unit, and the video decoding efficiency and quality are further improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for hardware accelerated video decoding. Background Art

[0002] With the development of computer technology and the promotion and application of various video processing applications, in order to improve the processing efficiency of video processing applications, a third-party interface has emerged based on hardware deployment, and the third-party interface has been called to perform video decoding and encoding, thereby realizing the use of hardware acceleration to improve the efficiency of video decoding and encoding.

[0003] In traditional technology, a VAAPI (Video Acceleration API) framework is usually deployed based on hardware, and the VAAPI framework is used to provide interfaces for various video processing applications. These video processing applications can implement hardware-accelerated video decoding by calling the interfaces provided by the VAAPI framework.

[0004] However, the traditionally used VAAPI framework supports hardware called GPU (graphics processing unit). Since the GPU's own processing function focuses on graphics processing, when the GPU is used for video processing, targeted and effective processing cannot be achieved and more resources are required. Therefore, traditional video processing methods still have problems such as low processing efficiency and excessive resource consumption. Summary of the invention

[0005] Based on this, it is necessary to provide a video hardware accelerated decoding method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of decoding processing based on video data and reduce resource consumption in response to the above technical problems.

[0006] In a first aspect, the present application provides a video hardware accelerated decoding method, comprising: receiving a video decoding processing request, determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application, and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip, and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, performing video decoding on video data corresponding to the video decoding processing request, and obtaining video decoded data.

[0007] In one of the embodiments, the method further includes: configuring a compatibility layer for the target chip according to a context processing component deployed by the video acceleration hardware driver to obtain a configured compatibility layer; through the compatibility layer, feeding back context control information corresponding to the video acceleration hardware driver to a context control component deployed by a hardware abstraction layer; and through the context control component, configuring the hardware abstraction layer according to the context control information to obtain a configured hardware abstraction layer.

[0008] In one of the embodiments, the hardware abstraction layer is configured according to the context control information through the context control component to obtain a configured hardware abstraction layer, including: obtaining configuration information parsing data corresponding to the context control information through the context control component; constructing a configuration file corresponding to the hardware abstraction layer according to the configuration information parsing data; configuring an image buffer array corresponding to the hardware abstraction layer according to the configuration file; constructing an interactive context object corresponding to the hardware abstraction layer according to the configuration file and the image buffer array to obtain a configured hardware abstraction layer.

[0009] In one of the embodiments, the calling of the video processing unit matching the video codec interface to perform video decoding on the video data corresponding to the video decoding processing request to obtain video decoding data includes: calling the video processing unit matching the video codec interface to obtain the video data corresponding to the video decoding processing request, generating a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer, and determining a target decoding function corresponding to the video data; performing function decoding processing on the candidate rendering list according to the target decoding function to obtain surface decoding data; decoding and parsing the surface decoding data to obtain video decoding data.

[0010] In one of the embodiments, generating a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer includes: when the image buffer array in the hardware abstraction layer is in an idle state, setting the surface state of the image buffer array to a rendering state to obtain a rendering buffer array; storing the video data in the rendering buffer array to obtain a target rendering buffer array carrying the video data; storing the target rendering buffer array in a rendering list to generate a candidate rendering list.

[0011] In one of the embodiments, the method further includes: deploying a hardware abstraction layer status monitoring function for the hardware abstraction layer, and calling the hardware abstraction layer status monitoring function to perform status monitoring on the hardware abstraction layer to obtain a data flow status corresponding to the hardware abstraction layer; determining a status of an output buffer corresponding to an image buffer array in the hardware abstraction layer according to the data flow status; when it is determined based on the status of the output buffer that the output buffer is full, obtaining video decoding data corresponding to the image buffer array, and clearing the input buffer and output buffer corresponding to the image buffer array to switch the image buffer array to an idle state.

[0012] In the second aspect, the present application also provides a video hardware accelerated decoding device, including: a target application determination module, used to receive a video decoding processing request, determine the target application and target chip corresponding to the video decoding processing request; a video acceleration hardware driver determination module, used to determine the video acceleration interface associated with the target application, and the video acceleration hardware driver corresponding to the video acceleration interface; a video codec interface acquisition module, used to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, and obtain the video codec interface deployed for the hardware abstraction layer; a video decoding data acquisition module, used to access the video codec interface, call the video processing unit matching the video codec interface, perform video decoding on the video data corresponding to the video decoding processing request, and obtain video decoding data.

[0013] In the third aspect, the present application also provides a computer device, including a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program: receiving a video decoding processing request, determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application, and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip, and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, decoding the video data corresponding to the video decoding processing request, and obtaining video decoding data.

[0014] In a fourth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which implements the following steps when executed by a processor: receiving a video decoding processing request, determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application, and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip, and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, performing video decoding on the video data corresponding to the video decoding processing request, and obtaining video decoding data.

[0015] In a fifth aspect, the present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps: receiving a video decoding processing request, determining a target application and a target chip corresponding to the video decoding processing request; determining a video acceleration interface associated with the target application, and a video acceleration hardware driver corresponding to the video acceleration interface; based on the video acceleration hardware driver, determining a hardware abstraction layer corresponding to the target chip, and obtaining a video codec interface deployed for the hardware abstraction layer; accessing the video codec interface, calling a video processing unit matching the video codec interface, performing video decoding on the video data corresponding to the video decoding processing request, and obtaining video decoding data.

[0016] In the above-mentioned video hardware accelerated decoding method, device, computer equipment, computer-readable storage medium and computer program product, by receiving a video decoding processing request, determining the target application and target chip corresponding to the video decoding processing request, and determining the video acceleration interface associated with the target application, and the video acceleration hardware driver corresponding to the video acceleration interface, so as to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, thereby achieving compatibility between the video processing unit of the target chip and the video acceleration hardware driver, and solving the problem that the video processing unit cannot be called directly by calling the video acceleration hardware driver to perform video decoding processing. Furthermore, by accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit matching the video codec interface is called to implement video decoding processing of the video data corresponding to the video decoding processing request, and obtain video decoding data. Compared with the traditional method of using a graphics processor for video decoding, the video processing unit implements targeted decoding processing, further improving the efficiency and quality of video decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A diagram showing an application environment of a method for hardware accelerated decoding of a video in one embodiment;

[0019] Figure 2 A schematic diagram of a process flow of a method for hardware accelerated decoding of a video in one embodiment;

[0020] Figure 3 A schematic diagram of a flow chart of an upper layer application calling a video processing unit to perform video decoding processing in one embodiment;

[0021] Figure 4 A schematic diagram of a flow chart of a method for hardware accelerated video decoding in another embodiment;

[0022] Figure 5 A schematic diagram of a data transmission process between a video acceleration hardware driver and a hardware abstraction layer in one embodiment;

[0023] Figure 6 A schematic diagram of a process of obtaining a configured hardware abstraction layer in one embodiment;

[0024] Figure 7 A schematic diagram of a configuration process of a hardware abstraction layer in one embodiment;

[0025] Figure 8 A timing diagram of implementing hardware abstraction layer configuration based on a video acceleration interface, a video acceleration hardware driver, and a hardware abstraction layer in one embodiment;

[0026] Fig. 9 A schematic diagram of a process for obtaining video decoding data in one embodiment;

[0027] Fig.10 A timing diagram of implementing video decoding processing based on a video acceleration interface, a video acceleration hardware driver, and a hardware abstraction layer in one embodiment;

[0028] Fig.11 A schematic diagram of a flow chart of a method for hardware accelerated decoding of a video in yet another embodiment;

[0029] Fig.12 A schematic diagram of a process of calling a hardware abstraction layer status monitoring function to perform status monitoring on a hardware abstraction layer in one embodiment;

[0030] Fig.13 is a structural block diagram of a video hardware accelerated decoding device in one embodiment;

[0031] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] The video hardware accelerated decoding method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, portable wearable devices, and aircraft, etc. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, and projection devices. Portable wearable devices can be smart watches, smart bracelets, and head-mounted devices. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, or a server cluster composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and basic cloud computing services such as big data and artificial intelligence platforms. Among them, the terminal 102 and the server 104 can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present application. Among them, the terminal 102 and the server 104 can be used alone to execute the video hardware accelerated decoding method provided in the embodiment of the present application, and the terminal 102 and the server 104 can also cooperate to execute the video hardware accelerated decoding method provided in the embodiment of the present application. For example, taking the video hardware accelerated decoding method provided in the embodiment of the present application as an example of the terminal 102 and the server 104 cooperating to execute the video hardware accelerated decoding method provided in the embodiment of the present application, the server 104 receives the video decoding processing request fed back by the terminal 102, and determines the target application and the target chip corresponding to the video decoding processing request, so as to further determine the video acceleration interface associated with the target application, and the video acceleration hardware driver corresponding to the video acceleration interface. Further, the server 104 determines the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, and obtains the video codec interface deployed for the hardware abstraction layer, so as to access the video codec interface, call the video processing unit matching the video codec interface, and decode the video data corresponding to the video decoding processing request to obtain the video decoding data. After obtaining the video decoding data, the server 104 may feed back the obtained video decoding data to the terminal device 102 for display or storage.

[0034] In an exemplary embodiment, Figure 2 As shown, a video hardware accelerated decoding method is provided, and the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S202 to S208. Among them:

[0035] Step S202: receiving a video decoding processing request, and determining a target application and a target chip corresponding to the video decoding processing request.

[0036] Specifically, upper-layer applications deployed by terminal devices, such as video editing applications, video playback applications, browsers, and other applications, can trigger video decoding processing requests during actual applications based on the editing or playback operations of the video by the user. The terminal feeds back the detected video decoding processing request to the server. After receiving the video decoding processing request, the server parses the video decoding processing request to determine the target application and target chip corresponding to the video decoding processing request.

[0037] The target application can be understood as the upper-layer application deployed by the terminal device that triggers the video decoding processing request, such as video editing applications, video playback applications, and browser applications. The target chip can be different types of processor chips in different actual application scenarios according to different actual application requirements, integrating components such as graphics processing acceleration GPU, video decoding VPU, display control interface and video memory controller to improve computing and multimedia processing capabilities.

[0038] Step S204: determining a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface.

[0039] Specifically, after determining the target application corresponding to the video decoding processing request, such as a video editing application, a video playback application, or a browser, the server further determines the video acceleration interface associated with the target application, that is, the video acceleration interface (i.e., Video Acceleration API, abbreviated as VAAPI) that is accessible during the operation of the target application deployed on the terminal and is used to implement hardware acceleration, such as which interfaces in the video acceleration interface library.

[0040] After determining the video acceleration interface associated with the target application, the server further determines the video acceleration hardware driver corresponding to the video acceleration interface, for example, through the video acceleration interface to be accessed, determines the video acceleration hardware drivers accessible by these interfaces.

[0041] Step S206: determining a hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, and acquiring a video codec interface deployed for the hardware abstraction layer.

[0042] Specifically, the server can determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, such as a hardware abstraction layer pre-configured for processor chip A, so as to further obtain the video codec interface deployed for the hardware abstraction layer.

[0043] Among them, the hardware abstraction layer pre-configured for the target chip (such as processor chip A) can be understood as a conversion layer that realizes the access and call of the video acceleration interface to the video processing unit. It is deployed with a video codec interface that can be used to access the video processing unit. By accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit can be used downward to perform video decoding processing.

[0044] Step S208, accessing the video codec interface, calling the video processing unit matching the video codec interface, performing video decoding on the video data corresponding to the video decoding processing request, and obtaining video decoding data.

[0045] Specifically, the server can call the video processing unit matching the video codec interface by accessing the video codec interface, and obtain the video data corresponding to the video decoding processing request so as to decode the video data through the video processing unit to obtain the video decoding data.

[0046] Among them, by calling the video processing unit, a candidate rendering list can be generated based on the video data and the image buffer array deployed in the hardware abstraction layer, and by determining the target decoding function corresponding to the video data, the candidate rendering list is subjected to function decoding processing according to the target decoding function to obtain surface decoding data. Finally, by decoding and parsing the surface decoding data, video decoding data corresponding to the video data can be obtained.

[0047] In an exemplary embodiment, Figure 3 As shown, a schematic diagram of an upper layer application calling a video processing unit to perform video decoding processing is provided, referring to Figure 3 It can be seen that in the process of the upper-layer application calling the video processing unit to perform video decoding processing, first, the upper-layer application includes a video editing application, a video playback application, a browser, etc., which can trigger a video decoding processing request in the actual application process. After receiving the video decoding processing request, the server determines the target application corresponding to the video decoding processing request (for example, Figure 3 video editing applications in ), and target chips (such as Figure 3 chip in the target application) and determine the video acceleration interface associated with the target application (such as Figure 3 Some video acceleration interfaces in the VAAPI library in ), and video acceleration hardware drivers corresponding to the video acceleration interfaces (such as Figure 3VA Hardware Driver in the .

[0048] Secondly, based on the video acceleration hardware driver, determine the compatibility layer corresponding to the target chip (for example Figure 3 The X100_video_dir layer in the target chip is determined by the compatibility layer (for example, Figure 3 The VDEC layer in the hardware abstraction layer is deployed with a video codec interface, which is used to use the video processing unit to perform video decoding processing downward. After obtaining the video codec interface deployed for the hardware abstraction layer, the video codec interface can be accessed from the video processing unit layer (such as Figure 3 In the VPU driver layer, that is, the video processing unit driver layer, the video processing unit matching the corresponding video codec interface is called to decode the video data corresponding to the video decoding processing request to obtain video decoding data.

[0049] In the above-mentioned video hardware accelerated decoding method, by receiving a video decoding processing request, the target application and the target chip corresponding to the video decoding processing request are determined, and the video acceleration interface associated with the target application and the video acceleration hardware driver corresponding to the video acceleration interface are determined, so as to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, thereby achieving compatibility between the video processing unit of the target chip and the video acceleration hardware driver, and solving the problem that the video processing unit cannot be called directly by calling the video acceleration hardware driver to perform video decoding processing. Furthermore, by accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit matching the video codec interface is called to implement video decoding processing of the video data corresponding to the video decoding processing request, and obtain video decoding data. Compared with the traditional method of using a graphics processor for video decoding, the video processing unit implements targeted decoding processing, further improving the efficiency and quality of video decoding.

[0050] In an exemplary embodiment, Figure 4 As shown, a video hardware accelerated decoding method is provided, and the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S402 to S406. Among them:

[0051] Step S402: configure a compatibility layer for the target chip according to the context processing component deployed by the video acceleration hardware driver to obtain a configured compatibility layer.

[0052] Among them, before the server determines the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, it is also necessary to pre-configure the hardware abstraction layer and deploy a video codec interface for the hardware abstraction layer, so that when video decoding processing is required, the video processing unit can be called to perform video decoding processing by accessing the video codec interface deployed by the hardware abstraction layer.

[0053] Specifically, during the process of hardware abstraction layer, the server involves compatibility layer configuration, specifically, by calling the context processing component deployed by the video acceleration hardware driver to perform compatibility layer configuration processing, including configuring the compatibility layer and configuring the compatibility layer buffer.

[0054] The configuration of the compatibility layer can obtain the compatibility layer context, through which the context control information corresponding to the video acceleration hardware driver can be sent to the hardware abstraction layer. After the configuration of the compatibility layer buffer, the compatibility layer surface data can be stored in the compatibility layer buffer, so that when the hardware abstraction layer is configured, the compatible surface data can be sent to the context component of the hardware abstraction layer to implement the hardware abstraction layer configuration.

[0055] Step S404: Feedback the context control information corresponding to the video acceleration hardware driver to the context control component deployed by the hardware abstraction layer through the compatibility layer.

[0056] Specifically, after the server is configured to obtain the compatibility layer corresponding to the target chip, the server feeds back the context control information corresponding to the video acceleration hardware driver to the context control component deployed in the hardware abstraction layer through the compatibility layer. Specifically, the server sends the context control information corresponding to the video acceleration hardware driver to the context control component deployed in the hardware abstraction layer through the compatibility layer context.

[0057] Step S406: Perform hardware abstraction layer configuration according to the context control information through the context control component to obtain a configured hardware abstraction layer.

[0058] Specifically, the server feeds back the context control information corresponding to the video acceleration hardware driver to the context control component deployed by the hardware abstraction layer through the configured compatibility layer, and then calls the context control component deployed by the hardware abstraction layer to configure the abstract layer according to the context control information corresponding to the video acceleration hardware driver, thereby obtaining the configured hardware abstraction layer.

[0059] After the hardware abstraction layer is configured, a video codec interface for calling the video processing unit is further deployed for the hardware abstraction layer, so that the video processing unit is called to perform video decoding processing by accessing the video codec interface.

[0060] In an exemplary embodiment, Figure 5 As shown, a schematic diagram of the data transmission process between a video acceleration hardware driver and a hardware abstraction layer is provided. Figure 5 It can be seen that the video acceleration hardware driver is deployed with a context processing component, which can realize the configuration of the compatibility layer and the configuration of the compatibility layer buffer. Through the context processing component deployed by the video acceleration hardware driver, the compatibility layer can be configured to obtain the compatibility layer context. Through the compatibility layer context, the context control information corresponding to the video acceleration hardware driver can be sent to the context control component deployed by the hardware abstraction layer. And through the context processing component deployed by the video acceleration hardware driver, after the compatibility layer buffer is configured, the compatibility layer surface data can be stored in the compatibility layer buffer, so that when the hardware abstraction layer is configured, the compatible surface data can be sent to the context component deployed by the hardware abstraction layer. Among them, the context control component deployed by the hardware abstraction layer configures the hardware abstraction layer according to the context control information and the compatible layer surface data, and obtains the configured hardware abstraction layer.

[0061] In this embodiment, a compatibility layer is configured for a target chip according to a context processing component deployed by a video acceleration hardware driver to obtain a configured compatibility layer. Through the compatibility layer, context control information corresponding to the video acceleration hardware driver is fed back to a context control component deployed by a hardware abstraction layer. Through the context control component, a hardware abstraction layer is configured according to the context control information to obtain a configured hardware abstraction layer, thereby achieving compatibility between a video processing unit of the target chip and a video acceleration hardware driver, solving the problem that the video processing unit cannot be called directly to perform video decoding processing by calling the video acceleration hardware driver. After obtaining the configured hardware abstraction layer, video decoding processing can be implemented based on the hardware abstraction layer to improve video decoding efficiency and quality.

[0062] In an exemplary embodiment, Figure 6 As shown, the step of obtaining a configured hardware abstraction layer, that is, performing hardware abstraction layer configuration according to context control information through the context control component, and obtaining the configured hardware abstraction layer, specifically includes the following steps S602 to S608. Among them:

[0063] Step S602: acquiring configuration information parsing data corresponding to the context control information through the context control component.

[0064] Specifically, the server obtains configuration information parsing data corresponding to the context control information through the context control component deployed by the hardware abstraction layer, which may be config (i.e., a file for storing configuration information) parsing data provided by target applications such as ffmpeg (i.e., video editing software).

[0065] Step S604, parsing data according to the configuration information, and constructing a configuration file corresponding to the hardware abstraction layer.

[0066] Specifically, the server, through the context control component deployed by the hardware abstraction layer, parses data according to the configuration information (such as config parsing data), and constructs a configuration file corresponding to the hardware abstraction layer (such as object_config, which is understood as an object (ie, hardware abstraction layer) configuration file).

[0067] Step S606: According to the configuration file, an image buffer array corresponding to the hardware abstraction layer is configured.

[0068] Specifically, the server configures the image buffer array (such as surface array, understood as used for) corresponding to the hardware abstraction layer according to the configuration file (such as object_config) corresponding to the hardware abstraction layer through the context control component deployed by the hardware abstraction layer.

[0069] Step S608, constructing an interactive context object corresponding to the hardware abstraction layer according to the configuration file and the image buffer array, and obtaining the configured hardware abstraction layer.

[0070] Specifically, the server constructs an interactive context object (such as object_context) corresponding to the hardware abstraction layer according to the configuration file (such as object_config) and the image buffer array (such as the surface array) through the context control component deployed by the hardware abstraction layer. By constructing the configuration file, configuring the image buffer array, and constructing the interactive context object, the configuration of the hardware abstraction layer is completed, and the configured hardware abstraction layer is obtained.

[0071] In an exemplary embodiment, Figure 7 As shown, a hardware abstraction layer configuration process is provided. Figure 7 It can be seen that the configuration process of the hardware abstraction layer specifically involves 0-information initialization, 1-configuration file construction, 2-image cache array construction, 3-interaction context object construction, and 4-image buffer construction, among which:

[0072] P0, 0-Information initialization: Before the server obtains the configuration information parsing data corresponding to the context control information, it also needs to process the request data corresponding to the video decoding request (i.e. Figure 7 request_data in the context control information (i.e. Figure 7Specifically, the malloc function (i.e., dynamic memory allocation function) in the request data is used to allocate a stack in the memory of the central processing unit to complete the initialization of the context control information corresponding to the video acceleration hardware driver.

[0073] Furthermore, the server can obtain configuration information parsing data corresponding to the context control information through the context control component deployed by the hardware abstraction layer, which can be config (i.e., a file for storing configuration information) parsing data provided by target applications such as ffmpeg (i.e., video editing software).

[0074] P1.1-Configuration file construction: Through the context control component deployed by the hardware abstraction layer, the configuration file corresponding to the hardware abstraction layer is constructed according to the configuration information parsing data (ie, config parsing data). Figure 7 The configuration file object_config corresponding to the hardware abstraction layer can be used to further initialize the data stream corresponding to the context control information.

[0075] P2, 2-Image cache array construction: The server configures the image buffer array corresponding to the hardware abstraction layer according to the configuration file (i.e. Figure 7 ). Among them, reference Figure 7 It can be seen that for the image buffer array object_surface, after creating the image buffer array object_surface, the image buffer array object_surface is used to obtain the decoded data according to the context control information corresponding to the hardware abstraction layer.

[0076] Among them, refer to Figure 7 It can be seen that for the image buffer array object_surface, the current video data to be rendered and the image buffer array object_surface can be put into the candidate rendering list. After the rendering is completed, the render_surface (i.e., the image buffer array whose surface state is the rendering state) in the interactive context object (i.e., object_context) is cleared. Among them, the surface decoding data obtained by function decoding processing on the candidate rendering list is used to store in the context control information corresponding to the hardware abstraction layer, and can be copied to the image buffer array object_surface.

[0077] P3, 3- Interaction context object construction: The server constructs the interaction context object corresponding to the hardware abstraction layer (i.e. Figure 7). Figure 7 It can be seen that when the interactive context object object_context is created, the handle is obtained and the handle of the interactive context object object_context is saved in the context control information corresponding to the hardware abstraction layer. At the same time, the context object context being processed in the current video acceleration hardware driver is also stored in the context control information corresponding to the hardware abstraction layer.

[0078] Among them, refer to Figure 7 It can be seen that when creating the interactive context object object_context, the configuration information corresponding to the image cache is obtained from the context control information corresponding to the hardware abstraction layer, which is used to configure the default parameters of the image cache area. According to the configuration information corresponding to the image cache, the image buffer array object_surface can be further filled with array source data.

[0079] P4, 4-Image buffer construction: After constructing the interaction context object, further construct the image buffer (i.e. Figure 7 The image buffer object_buffer is used to store data. The image buffer object_buffer is only used to store initial data. The image buffer array object_surface is constructed before the image buffer object_buffer. However, during image rendering, the data in the image buffer object_buffer is transferred to the corresponding part of the image buffer array object_surface, which is used as the data input for data decoding. The decoded data is also put into the corresponding part of the image buffer array object_surface through the callback function. The image buffer array object_surface is used to implement object data management, and the image buffer object_buffer is used to temporarily store input data.

[0080] In an exemplary embodiment, Figure 8 As shown, a schematic diagram of implementing hardware abstraction layer configuration based on a video acceleration interface, a video acceleration hardware driver, and a hardware abstraction layer is provided. Figure 8 It can be seen that the process of hardware abstraction layer configuration includes:

[0081] 1) The target application sends the request data corresponding to the video decoding processing request to the video acceleration hardware driver through the video acceleration interface in the video acceleration interface framework. Specifically, the target application may be a video editing application, the video acceleration interface framework may be a VAAPI framework, the video acceleration hardware driver may be X100_driver, and the hardware abstraction layer may be a VDEC layer. 2) The video acceleration hardware driver allocates and initializes the request data corresponding to the video decoding processing request based on the hardware abstraction layer registration function table. 3) The video acceleration hardware driver obtains the configuration information parsing data corresponding to the context control information, and initializes the hardware abstraction layer data stream when the data type corresponding to the configuration information parsing data is the target type. 4) The video acceleration hardware driver constructs a configuration file corresponding to the hardware abstraction layer according to the configuration information parsing data. 5) The video acceleration hardware driver configures the image buffer array corresponding to the hardware abstraction layer according to the configuration file, and initializes the image buffer array according to the height and width information in the configuration file. 6) The video acceleration hardware driver constructs an interactive context object corresponding to the hardware abstraction layer according to the configuration file and the image buffer array. 7) Based on the hardware abstraction layer, the video acceleration hardware driver deploys the hardware abstraction layer status monitoring function, and monitors the data flow status of the hardware abstraction layer through the hardware abstraction layer status monitoring function. 8) After constructing the interaction context object, the video acceleration hardware driver constructs an image buffer for storing data and completes the configuration of the hardware abstraction layer.

[0082] In this embodiment, through the context control component, configuration information parsing data corresponding to the context control information is obtained, and a configuration file corresponding to the hardware abstraction layer is constructed according to the configuration information parsing data. According to the configuration file, an image buffer array corresponding to the hardware abstraction layer is configured; according to the configuration file, an interactive context object corresponding to the hardware abstraction layer is constructed to obtain a configured hardware abstraction layer, thereby achieving compatibility between the video processing unit and the video acceleration hardware driver of the target chip, solving the problem that the video processing unit cannot be called directly by calling the video acceleration hardware driver to perform video decoding processing. After obtaining the configured hardware abstraction layer, video decoding processing can be implemented based on the hardware abstraction layer, thereby improving video decoding efficiency and quality.

[0083] In an exemplary embodiment, Fig. 9 As shown, the step of obtaining the video decoding data, i.e. calling the video processing unit matching the video codec interface, performing video decoding on the video data corresponding to the video decoding processing request, and obtaining the video decoding data, specifically includes the following steps S902 to S906. Among them:

[0084] Step S902, calling the video processing unit matching the video codec interface, obtaining video data corresponding to the video decoding processing request, generating a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer, and determining a target decoding function corresponding to the video data.

[0085] Specifically, the server accesses the video codec interface deployed by the hardware abstraction layer, calls the video processing unit matching the video codec interface, obtains the video data corresponding to the video decoding processing request, and performs video decoding processing on the video data corresponding to the video decoding processing request to obtain video decoding data.

[0086] Wherein, for the obtained video data corresponding to the video decoding processing request, a candidate rendering list is generated based on the video data and the image buffer array in the hardware abstraction layer, so as to further perform decoding processing on the candidate rendering list. Wherein, after obtaining the video data corresponding to the video decoding processing request, a target decoding function corresponding to the video data is further determined, so as to perform decoding processing on the candidate rendering list through the target decoding function.

[0087] In an exemplary embodiment, generating a candidate rendering list based on video data and an image buffer array in a hardware abstraction layer includes:

[0088] When the image buffer array in the hardware abstraction layer is in an idle state, the surface state of the image buffer array is set to a rendering state to obtain a rendering buffer array; the video data is stored in the rendering buffer array to obtain a target rendering buffer array carrying the video data; the target rendering buffer array is stored in a rendering list to generate a candidate rendering list.

[0089] Specifically, when the image buffer array in the hardware abstraction layer is in an idle state, that is, when the image buffer array in the current hardware abstraction layer is in a usable state, the surface state of the image buffer array is set to a rendering state to obtain a rendering buffer array, and in the interaction context object, the currently rendered surface is marked as an image buffer array (i.e., a rendering buffer array) whose surface state is a rendering state.

[0090] The video data corresponding to the video decoding processing request may be first stored in the image buffer. When video decoding processing is required, the video data is copied from the image buffer and stored in the rendering buffer array to obtain a target rendering buffer array carrying the video data. Furthermore, a candidate rendering list is generated by obtaining a pre-built rendering list and storing the target rendering buffer array in the pre-built rendering list. The actually required video decoding data may be obtained by decoding the candidate rendering list.

[0091] Step S904 , performing function decoding processing on the candidate rendering list according to the target decoding function to obtain surface decoding data.

[0092] Specifically, the server determines the target decoding function corresponding to the video data and calls the target decoding function to perform function decoding processing on the candidate rendering list according to the target decoding function to obtain surface decoding data. When the surface decoding data is obtained, that is, after the function decoding processing is completed, the surface state of the target rendering buffer array in the candidate rendering list is set to an idle state, and the image buffer array (i.e., the rendering buffer array) whose surface state is the rendering state of the surface currently rendered in the interactive context object is cleared.

[0093] Step S906, decoding and parsing the surface decoded data to obtain video decoded data.

[0094] Specifically, the server can obtain video decoded data corresponding to the video data by further decoding and parsing the surface decoded data. When further decoding and parsing the surface decoded data, the surface decoded data can be set to start in a parsing state and the stop point can be set as the start node of the processing sequence to implement the decoding and parsing process.

[0095] Among them, for the decoding and parsing process, specifically when the surface decoding data is submitted for the first time, a processing node for the start of the sequence is created to trigger a stop callback event (i.e., complete the current decoding and parsing processing flow). At the same time, multiple image processing nodes need to be created to decode and parse the surface decoding data to obtain video decoding data.

[0096] In an exemplary embodiment, Fig.10 As shown, a schematic diagram of implementing video decoding processing based on a video acceleration interface, a video acceleration hardware driver and a hardware abstraction layer is provided. Fig.10 It can be seen that the video decoding process specifically includes:

[0097] P1. The first stage is the start stage, which indicates the stage of registering the current image buffer array surface to the context interaction context object. Among them: 1) The target application sends the video data corresponding to the video decoding processing request to the video acceleration hardware driver through the video acceleration interface. Among them, when sending video data to the video acceleration hardware driver, it is also necessary to send the image buffer array number and the interaction context object number to the video acceleration hardware driver, so that the video acceleration hardware driver can determine whether the corresponding image buffer array is in an idle state according to the image buffer array number. 2) When the video acceleration hardware driver determines that the image buffer array in the hardware abstraction layer is in an idle state, it sets the surface state of the image buffer array to the rendering state, obtains the rendering buffer array, and marks the currently rendered surface as the image buffer array (i.e., the rendering buffer array) whose surface state is in the rendering state in the interaction context object.

[0098] P2, the second stage is the rendering stage, which means the stage of copying the data in the image buffer buffer to the image buffer array surface. Among them: 3) The target application sends the interactive context object number and the image buffer number to the video acceleration hardware driver through the video acceleration interface. 4) The video acceleration hardware driver copies the video data from the image buffer corresponding to the image buffer number to the rendering image buffer array corresponding to the interactive context object corresponding to the interactive context object number, and obtains the target rendering buffer array carrying the video data.

[0099] P3, the third stage is the decoding stage, which means the stage of calling the decoding function in the hardware abstraction layer VDEC for decoding the current image buffer array surface. Among them: 5) The video acceleration hardware driver stores the target rendering buffer array into the pre-configured rendering list, obtains the candidate rendering list, and calls the target decoding data corresponding to the video data, performs function decoding and parsing on the candidate rendering list, and obtains the surface decoding data. 6) The video acceleration hardware driver sets the surface state of the target rendering buffer array in the candidate rendering list to the idle state, and clears the currently rendered rendering image buffer array in the interactive context object. 7) The video acceleration hardware driver sets the parsing state start point for the surface decoding data, and sets the stop point to the parsing mode of the processing sequence start node. 8) The video acceleration hardware driver creates a sequence start processing node for the hardware abstraction layer when the surface decoding data is submitted for the first time, which is used to trigger the stop callback event (that is, complete the current decoding and parsing processing flow), and creates multiple image processing nodes for decoding and parsing the surface decoding data to obtain video decoding data.

[0100] In this embodiment, by calling a video processing unit that matches the video codec interface, video data corresponding to the video decoding processing request is obtained, based on the video data and the image buffer array in the hardware abstraction layer, a candidate rendering list is generated, and a target decoding function corresponding to the video data is determined, and according to the target decoding function, function decoding processing is performed on the candidate rendering list to obtain surface decoding data, so as to decode and parse the surface decoding data to obtain video decoding data. Compared with the traditional method of using a graphics processor for video decoding, targeted decoding processing is implemented by the video processing unit to further improve the efficiency and quality of video decoding.

[0101] In an exemplary embodiment, Fig.11 As shown, a video hardware accelerated decoding method is provided, and the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps S1102 to S1106. Among them:

[0102] Step S1102, deploying a hardware abstraction layer status monitoring function for the hardware abstraction layer, and calling the hardware abstraction layer status monitoring function to perform status monitoring on the hardware abstraction layer, and obtaining a data flow status corresponding to the hardware abstraction layer.

[0103] Specifically, the server deploys a hardware abstraction layer status monitoring function (i.e., VDEC_Callback function) for the hardware abstraction layer, and monitors the status of the hardware abstraction layer by calling the hardware abstraction layer status monitoring function to obtain the data flow status corresponding to the hardware abstraction layer. The data flow status corresponding to the hardware abstraction layer specifically includes the state that the input buffer of the image buffer array is empty, the state that the output buffer of the image buffer array is filled, and the data flow stop state.

[0104] Among them, the state of the input buffer of the image buffer array being empty specifically refers to the fact that the image buffer array currently has no new input data to be input and processed, and the mapping of the image buffer array to the input address should be released, and the image buffer array should be removed to wait for the next remapping and input data. The state of the output buffer of the image buffer array being filled refers to the fact that the input data of the image buffer array has been decoded by the hardware abstraction layer VDEC, and the decoded data can be copied out, and the output area and input area of ​​the image buffer array can be cleared, so that the image buffer array is in the preparation stage (that is, the idle state that can be used), ready to receive the next segment of data for subsequent processing.

[0105] The data flow stop state specifically includes active stop state and passive stop state. Among them, the active stop state can be understood as when creating a data stream corresponding to the hardware abstraction layer, by specifying the type of creation only includes parsing and decoding, and specifying the stop point in the parsing state, which is usually set to the processing node that submits the first data. When the data is submitted, the stop callback function will be triggered, so that some stream attribute configurations and output content can be performed in the callback function. Among them, the passive stop state can be understood as a stop state that is triggered when a fault or problem occurs, such as a system error in a segment dump.

[0106] Step S1104: determining the state of the output buffer corresponding to the image buffer array in the hardware abstraction layer according to the data stream state.

[0107] Specifically, after obtaining the data flow status corresponding to the hardware abstraction layer, the server further determines the status of the output buffer corresponding to the image buffer array in the hardware abstraction layer based on the data flow status, such as determining that the output buffer corresponding to the image buffer array is empty or the output buffer corresponding to the image buffer array is full.

[0108] Step S1106, when it is determined based on the state of the output buffer that the output buffer is full, the video decoding data corresponding to the image buffer array is obtained, and the input buffer and output buffer corresponding to the image buffer array are cleared to switch the image buffer array to an idle state.

[0109] Specifically, when the server determines that the output buffer corresponding to the image buffer array is full based on the state of the output buffer, the server obtains the video decoding data corresponding to the image buffer array to feed back to the terminal for subsequent processing such as display and playback.

[0110] Among them, after obtaining the video decoding data corresponding to the image buffer array, the input buffer and output buffer corresponding to the image buffer array are further cleared to switch the image buffer array to an idle state, so that subsequent processing flows can call the image buffer array for data storage or data encoding and decoding processing, etc.

[0111] In an exemplary embodiment, Fig.12 As shown, a process of calling the hardware abstraction layer status monitoring function to monitor the status of the hardware abstraction layer is provided. Fig.12 It can be seen that when the hardware abstraction layer status monitoring function is called to monitor the status of the hardware abstraction layer, the data flow status corresponding to the hardware abstraction layer obtained may specifically include the state that the input buffer of the image buffer array is empty, the state that the output buffer of the image buffer array is filled, and the data flow stop state. Among them:

[0112] P1. The input buffer of the image buffer array is empty: Specifically, it means that the image buffer array currently has no new input data that needs to be input and processed. The image buffer array should be unmapped from the input address, and the image buffer array should be removed to wait for the next remapping and input data.

[0113] P2. The state in which the output buffer of the image buffer array is filled: This means that the input data of the image buffer array has been decoded by the hardware abstraction layer VDEC. The decoded data can be copied from the output buffer of the image buffer array, and the output area and input area of ​​the image buffer array are cleared, so that the image buffer array is in the preparation stage (i.e., an idle state that can be used), ready to receive the next segment of data for subsequent processing.

[0114] P3. Regarding the data flow stop state: specifically including active stop state and passive stop state. Among them, the active stop state can be understood as when creating a data flow corresponding to the hardware abstraction layer, by specifying the type of creation only includes parsing and decoding, and specifying the stop point in the parsing state, usually set to the processing node that submits the first data, then triggering the stop callback function when submitting the data, so that some stream attribute configuration and output content can be performed in the callback function. Among them, the passive stop state can be understood as the stop state that will be triggered when a fault or problem occurs, such as a system error in a segment dump.

[0115] Among them, by analyzing the data flow stop state, the current reason for the data flow stop is determined. For example, when it meets the stop point specified in the parsing state, that is, when it is caused by the configuration information, the operation of the reconfiguration function is triggered. At the same time, some stream attribute configurations and output content configurations are further performed in the callback function.

[0116] In the above-mentioned video hardware accelerated decoding method, a hardware abstraction layer status monitoring function is deployed for the hardware abstraction layer, and the hardware abstraction layer status monitoring function is called to monitor the status of the hardware abstraction layer, obtain the data flow status corresponding to the hardware abstraction layer, and when it is determined according to the data flow status that the output buffer corresponding to the image buffer array in the hardware abstraction layer is full, obtain the video decoding data corresponding to the image buffer array, clear the input buffer and output buffer corresponding to the image buffer array, and switch the image buffer array to an idle state, so that the state of the image buffer array can be modified and adjusted according to the data flow status during the actual processing process, so as to facilitate subsequent calls and video processing, reduce resource waste, and improve video processing efficiency.

[0117] In an exemplary embodiment, an overall process of a video hardware accelerated decoding method is provided, including the following processing steps:

[0118] 1) Receive a video decoding processing request, determine the target application and target chip corresponding to the video decoding processing request. 2) Determine the video acceleration interface associated with the target application, and the video acceleration hardware driver corresponding to the video acceleration interface. 3) According to the context processing component deployed by the video acceleration hardware driver, configure the compatibility layer of the target chip to obtain the configured compatibility layer. 4) Feedback the context control information corresponding to the video acceleration hardware driver to the context control component deployed by the hardware abstraction layer through the compatibility layer. 5) Obtain the configuration information parsing data corresponding to the context control information through the context control component. 6) Construct a configuration file corresponding to the hardware abstraction layer according to the configuration information parsing data. 7) The configuration file is configured to obtain an image buffer array corresponding to the hardware abstraction layer. 8) The configuration file and the image buffer array are used to construct an interactive context object corresponding to the hardware abstraction layer, and the configured hardware abstraction layer is obtained. 9) Based on the video acceleration hardware driver, determine the hardware abstraction layer corresponding to the target chip, and obtain the video codec interface deployed for the hardware abstraction layer. 10) Access the video codec interface to obtain the video data corresponding to the video decoding processing request using the video processing unit matching the video codec interface. 11) Deploy the hardware abstraction layer status monitoring function for the hardware abstraction layer. 12) Call the hardware abstraction layer status monitoring function to monitor the hardware abstraction layer and obtain the data flow status corresponding to the hardware abstraction layer. 13) According to the data flow status, determine the status of the output buffer corresponding to the image buffer array in the hardware abstraction layer. 14) When it is determined based on the status of the output buffer that the output buffer is full, obtain the video decoding data corresponding to the image buffer array, clear the input buffer and output buffer corresponding to the image buffer array, and switch the image buffer array to the idle state. 15) When the image buffer array in the hardware abstraction layer is in the idle state, set the surface state of the image buffer array to the rendering state to obtain the rendering buffer array. 16) Store the video data in the rendering buffer array, obtain the target rendering buffer array carrying the video data, store the target rendering buffer array in the rendering list, and generate a candidate rendering list. 17) Determine the target decoding function corresponding to the video data, perform function decoding processing on the candidate rendering list according to the target decoding function, and obtain surface decoding data. 18) Decode and parse the surface decoding data to obtain video decoding data.

[0119] In the above-mentioned video hardware accelerated decoding method, compatibility between the video processing unit of the target chip and the video acceleration hardware driver is achieved, which solves the problem that the video processing unit cannot be called directly to perform video decoding processing by calling the video acceleration hardware driver, and by accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit matching the video codec interface is called to implement video decoding processing of the video data corresponding to the video decoding processing request, and obtain video decoding data. Compared with the traditional method of using a graphics processor for video decoding, the targeted decoding processing implemented by the video processing unit further improves the video decoding efficiency and quality.

[0120] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiment of the present application also provides a video hardware accelerated decoding device for implementing the video hardware accelerated decoding method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more video hardware accelerated decoding device embodiments provided below can refer to the limitations of the video hardware accelerated decoding method above, and will not be repeated here.

[0122] In an exemplary embodiment, Fig.13 As shown, a video hardware accelerated decoding device is provided, including: a target application determination module 1302, a video acceleration hardware driver determination module 1304, a video codec interface acquisition module 1306 and a video decoding data acquisition module 1308, wherein:

[0123] The target application determination module 1302 is used to receive a video decoding processing request, determine the target application corresponding to the video decoding processing request, and the target chip; the video acceleration hardware driver determination module 1304 is used to determine the video acceleration interface associated with the target application, and the video acceleration hardware driver corresponding to the video acceleration interface; the video codec interface acquisition module 1306 is used to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, and obtain the video codec interface deployed for the hardware abstraction layer; the video decoding data acquisition module 1308 is used to access the video codec interface, call the video processing unit matching the video codec interface, decode the video data corresponding to the video decoding processing request, and obtain video decoding data.

[0124] In the above-mentioned video hardware accelerated decoding device, by receiving a video decoding processing request, the target application and the target chip corresponding to the video decoding processing request are determined, and the video acceleration interface associated with the target application and the video acceleration hardware driver corresponding to the video acceleration interface are determined, so as to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, thereby achieving compatibility between the video processing unit of the target chip and the video acceleration hardware driver, and solving the problem that the video processing unit cannot be called directly by calling the video acceleration hardware driver to perform video decoding processing. Furthermore, by accessing the video codec interface deployed for the hardware abstraction layer, the video processing unit matching the video codec interface is called to implement video decoding processing of the video data corresponding to the video decoding processing request, and obtain video decoding data. Compared with the traditional method of using a graphics processor for video decoding, the video processing unit implements targeted decoding processing, further improving the efficiency and quality of video decoding.

[0125] In an exemplary embodiment, a video hardware acceleration decoding device is provided, which also includes a hardware abstraction layer configuration module, which is used to: configure a compatibility layer for a target chip according to a context processing component deployed by a video acceleration hardware driver to obtain a configured compatibility layer; feed back context control information corresponding to the video acceleration hardware driver to a context control component deployed by the hardware abstraction layer through the compatibility layer; and configure the hardware abstraction layer according to the context control information through the context control component to obtain a configured hardware abstraction layer.

[0126] In an exemplary embodiment, the hardware abstraction layer configuration module is also used to: obtain configuration information parsing data corresponding to the context control information through the context control component; construct a configuration file corresponding to the hardware abstraction layer according to the configuration information parsing data; configure an image buffer array corresponding to the hardware abstraction layer according to the configuration file; construct an interactive context object corresponding to the hardware abstraction layer according to the configuration file and the image buffer array, and obtain a configured hardware abstraction layer.

[0127] In an exemplary embodiment, the video decoding data acquisition module is also used to: call a video processing unit that matches the video codec interface to obtain video data corresponding to the video decoding processing request, generate a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer, and determine the target decoding function corresponding to the video data; perform function decoding processing on the candidate rendering list according to the target decoding function to obtain surface decoding data; and decode and parse the surface decoding data to obtain video decoding data.

[0128] In an exemplary embodiment, the video decoding data acquisition module is also used to: when the image buffer array in the hardware abstraction layer is in an idle state, set the surface state of the image buffer array to a rendering state to obtain a rendering buffer array; store the video data in the rendering buffer array to obtain a target rendering buffer array carrying the video data; store the target rendering buffer array in a rendering list to generate a candidate rendering list.

[0129] In an exemplary embodiment, a video hardware accelerated decoding device is provided, which also includes a status monitoring module, which is used to: deploy a hardware abstraction layer status monitoring function for a hardware abstraction layer, and call the hardware abstraction layer status monitoring function to perform status monitoring on the hardware abstraction layer to obtain a data flow status corresponding to the hardware abstraction layer; determine the status of an output buffer corresponding to an image buffer array in the hardware abstraction layer according to the data flow status; when it is determined based on the status of the output buffer that the output buffer is full, obtain video decoding data corresponding to the image buffer array, and clear the input buffer and output buffer corresponding to the image buffer array to switch the image buffer array to an idle state.

[0130] Each module in the above-mentioned video hardware accelerated decoding device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0131] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.14As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as target applications, target chips, video acceleration interfaces, hardware abstraction layers, video codec interfaces, video processing units, video data, and video decoding data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video hardware accelerated decoding method is implemented.

[0132] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0133] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0135] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0137] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0139] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video hardware accelerated decoding method, characterized in that: The method comprises: receiving a video decoding processing request, and determining a target application and a target chip corresponding to the video decoding processing request; Determining a video acceleration interface associated with the target application and a video acceleration hardware driver corresponding to the video acceleration interface; Based on the video acceleration hardware driver, determine the hardware abstraction layer corresponding to the target chip, and obtain the video codec interface deployed for the hardware abstraction layer; The video codec interface is accessed, a video processing unit matching the video codec interface is called, and video decoding is performed on the video data corresponding to the video decoding processing request to obtain video decoding data.

2. The method according to claim 1, characterized in that: The method further comprises: According to the context processing component deployed by the video acceleration hardware driver, a compatibility layer is configured for the target chip to obtain a configured compatibility layer; Feedback the context control information corresponding to the video acceleration hardware driver to the context control component deployed in the hardware abstraction layer through the compatibility layer; The hardware abstraction layer is configured through the context control component according to the context control information to obtain a configured hardware abstraction layer.

3. The method according to claim 2, characterized in that The step of configuring the hardware abstraction layer according to the context control information through the context control component to obtain the configured hardware abstraction layer includes: Acquiring, by means of the context control component, configuration information parsing data corresponding to the context control information; Parsing data according to the configuration information, and constructing a configuration file corresponding to the hardware abstraction layer; According to the configuration file, an image buffer array corresponding to the hardware abstraction layer is configured; According to the configuration file and the image buffer array, an interactive context object corresponding to the hardware abstraction layer is constructed to obtain a configured hardware abstraction layer.

4. The method according to any one of claims 1 to 3, characterized in that: The calling of the video processing unit matching the video codec interface to perform video decoding on the video data corresponding to the video decoding processing request to obtain video decoding data includes: Calling a video processing unit matching the video codec interface, obtaining video data corresponding to the video decoding processing request, generating a candidate rendering list based on the video data and an image buffer array in the hardware abstraction layer, and determining a target decoding function corresponding to the video data; According to the target decoding function, performing function decoding processing on the candidate rendering list to obtain surface decoding data; The surface decoded data is decoded and parsed to obtain video decoded data.

5. The method according to claim 4, characterized in that The generating a candidate rendering list based on the video data and the image buffer array in the hardware abstraction layer comprises: When the image buffer array in the hardware abstraction layer is in an idle state, setting the surface state of the image buffer array to a rendering state to obtain a rendering buffer array; Storing the video data in the rendering buffer array to obtain a target rendering buffer array carrying the video data; The target rendering buffer array is stored in a rendering list to generate a candidate rendering list.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: For the hardware abstraction layer, deploy a hardware abstraction layer state monitoring function, call the hardware abstraction layer state monitoring function, perform state monitoring on the hardware abstraction layer, and obtain a data flow state corresponding to the hardware abstraction layer; Determine the state of the output buffer corresponding to the image buffer array in the hardware abstraction layer according to the data stream state; When it is determined based on the state of the output buffer that the output buffer is full, the video decoding data corresponding to the image buffer array is acquired, and the input buffer and output buffer corresponding to the image buffer array are cleared to switch the image buffer array to an idle state.

7. A video hardware accelerated decoding device, characterized in that: The device comprises: A target application determination module, configured to receive a video decoding processing request and determine a target application and a target chip corresponding to the video decoding processing request; A video acceleration hardware driver determination module, used to determine a video acceleration interface associated with the target application, and a video acceleration hardware driver corresponding to the video acceleration interface; A video codec interface acquisition module, used to determine the hardware abstraction layer corresponding to the target chip based on the video acceleration hardware driver, and obtain the video codec interface deployed for the hardware abstraction layer; The video decoding data acquisition module is used to access the video codec interface, call the video processing unit matching the video codec interface, perform video decoding on the video data corresponding to the video decoding processing request, and obtain video decoding data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images