Video encoding and decoding method, device and system, electronic equipment, medium and product

Abstract

The invention provides a video coding and decoding method, device and system, electronic equipment, a medium and a product, and the method comprises the steps: calling a hardware coding and decoding interface corresponding to a target video format of a target video frame in response to a coding and decoding request triggered for the target video frame; and under the condition that the coding and decoding hardware corresponding to the hardware coding and decoding interface is in an unavailable state, generating a coding and decoding result of the target video frame through a software simulation program corresponding to the hardware coding and decoding interface. The hardware coding and decoding interface in the mode not only supports a hardware implementation mode, but also can perform processing based on a software simulation program under the condition that hardware is unavailable, so that the use flexibility of the interface is improved, and application and popularization are facilitated.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a video encoding and decoding method, apparatus, system, electronic device, medium, and product. Background Technology

[0002] Video encoding and decoding is a technical method for compressing and decompressing video data. Its core mission is to significantly reduce the storage volume and transmission bandwidth requirements of video data while ensuring visual quality.

[0003] This technology involves two inverse processes: the encoder converts the raw video into a compressed bitstream, and the decoder restores the bitstream to playable video footage. From the early motion-compensated MPEG-2 standard to today's ultra-high-definition encoding and decoding standards supporting 4K / 8K, video compression technology has undergone three key development stages: the first stage was based on discrete cosine transform; the second stage introduced inter-frame prediction and entropy coding; and the third stage integrates intelligent segmentation and prediction technologies, improving compression efficiency by more than twenty times.

[0004] As video resolution increases to 4K / 8K levels, pure software encoding and decoding methods face enormous computational pressure. To improve video encoding and decoding efficiency, hardware-based methods can be used, leveraging hardware performance. However, the hardware encoding / decoding interfaces offered by these methods have limited functionality, supporting only hardware implementations. This results in poor interface flexibility and hinders widespread application. Summary of the Invention

[0005] This disclosure provides a video encoding / decoding method, apparatus, system, electronic device, medium, and product to improve interface flexibility.

[0006] In a first aspect, this disclosure provides a video encoding / decoding method, the method comprising:

[0007] In response to a codec request triggered for a target video frame, the hardware codec interface corresponding to the target video format of the target video frame is invoked;

[0008] When the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is unavailable, the encoding / decoding result of the target video frame is generated by the software simulation program corresponding to the hardware encoding / decoding interface.

[0009] In one optional implementation, the invocation of the hardware codec interface corresponding to the target video format of the target video frame includes:

[0010] According to the interface specification of the hardware codec interface, determine the interface parameter set corresponding to the target video frame, and send an interface call instruction containing the interface parameter set to the hardware codec interface;

[0011] The interface specification of the hardware codec interface is used to define the types of parameters in the interface parameter set according to the format specification of the target video format.

[0012] In one optional implementation, the generation of the encoding / decoding result of the target video frame through the software simulation program corresponding to the hardware encoding / decoding interface includes:

[0013] During the encoding process, the interface parameter set is subjected to a first conversion process by the software simulation program to obtain a first conversion result; the encoding result of the target video frame is obtained based on the first conversion result, wherein the encoding result is represented by a binary bit stream;

[0014] During the decoding process, the interface parameter set is subjected to a second conversion process by the software simulation program to obtain a second conversion result; the binary bit stream is processed according to the second conversion result to obtain the decoding result of the target video frame.

[0015] In one optional implementation, the interface parameter set includes: an image parameter set, a strip parameter set, and an extensible parameter set;

[0016] The scalable parameter set includes version control parameters, and the parameter values ​​of the version control parameters correspond to the version update results of the target video format.

[0017] In one optional implementation, before generating the encoding / decoding result of the target video frame through the software simulation program corresponding to the hardware encoding / decoding interface when the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is unavailable, the method further includes:

[0018] The status of the codec hardware corresponding to the hardware codec interface is detected by the interface driver corresponding to the hardware codec interface.

[0019] The interface driver includes the software simulation program and the hardware driver; the hardware driver is used to drive the encoding / decoding hardware to perform encoding / decoding processing when the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is in an available state.

[0020] In one optional implementation, the encoding / decoding result generated by the software simulation program is stored in a first storage space. After generating the encoding / decoding result of the target video frame through the software simulation program corresponding to the hardware encoding / decoding interface, the method further includes:

[0021] The encoding and decoding results in the first storage space are transmitted to the second storage space so as to obtain the encoding and decoding results from the second storage space through the hardware encoding and decoding interface; wherein, the second storage space corresponds to the hardware encoding and decoding interface.

[0022] In one alternative implementation, the software simulation program runs on a first device, and the hardware driver runs on a second device corresponding to the codec hardware.

[0023] Furthermore, the first storage space includes: device memory space corresponding to the first device, and the second storage space includes: device memory space corresponding to the second device.

[0024] In one optional implementation, after generating the encoding and decoding result of the target video frame through the software simulation program corresponding to the hardware encoding and decoding interface, the method further includes:

[0025] The encoding / decoding result is matched with a preset standard result;

[0026] If the matching fails, adjust the interface specification of the hardware codec interface and / or the program execution logic of the software simulation program.

[0027] In one optional implementation, after generating the encoding and decoding result of the target video frame through the software simulation program corresponding to the hardware encoding and decoding interface, the method further includes:

[0028] The runtime status data corresponding to the software simulation program is stored in a shared storage area so that the hardware driver loads the runtime status data during the initialization process.

[0029] In one alternative implementation, the encoding / decoding request is a hardware acceleration request, the hardware encoding / decoding interface is a hardware acceleration interface, and the target video format corresponds to a video standard format whose release time is later than a preset time threshold.

[0030] Secondly, this disclosure provides a video encoding / decoding apparatus, the apparatus comprising:

[0031] The response module is adapted to respond to a codec request triggered for a target video frame by calling a hardware codec interface corresponding to the target video format of the target video frame.

[0032] The generation module is adapted to generate the encoding and decoding result of the target video frame through a software simulation program corresponding to the hardware encoding and decoding interface when the encoding and decoding hardware corresponding to the hardware encoding and decoding interface is unavailable.

[0033] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the methods described above.

[0034] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-described method when executed by a processor / processor core.

[0035] Fifthly, this disclosure provides a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, the processor in the electronic device executes the above-described method.

[0036] Sixthly, this disclosure provides a video encoding and decoding system, the system comprising: the video encoding and decoding apparatus described above.

[0037] In the embodiments provided in this disclosure, upon receiving an encoding / decoding request triggered for a target video frame, a hardware encoding / decoding interface corresponding to the target video format of the target video frame is invoked. Furthermore, if the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is unavailable, the encoding / decoding result of the target video frame is generated through a software simulation program corresponding to the hardware encoding / decoding interface. This approach not only supports hardware implementation but also enables processing based on a software simulation program when hardware is unavailable, thereby improving the flexibility of the interface and facilitating its application and promotion.

[0038] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0039] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0040] Figure 1 A flowchart illustrating a video encoding / decoding method according to an embodiment of this disclosure is shown.

[0041] Figure 2 A schematic diagram illustrating the development of a video codec hardware acceleration interface in related technologies is shown.

[0042] Figure 3 A schematic diagram of the architecture of a video codec system in one example of this disclosure is shown;

[0043] Figure 4 A block diagram of a video encoding / decoding apparatus provided in an embodiment of this disclosure;

[0044] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0045] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0046] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0047] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0048] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0049] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0050] Figure 1 A flowchart illustrating a video encoding / decoding method provided in an embodiment of this disclosure is shown. Figure 1 As shown, the method includes:

[0051] Step S110: In response to the encoding / decoding request triggered for the target video frame, call the hardware encoding / decoding interface corresponding to the target video format of the target video frame.

[0052] The target video frame typically refers to an independent video frame unit to be processed, including but not limited to various frame types such as I-frames, P-frames, and B-frames, which may have various different resolution specifications. Encoding / decoding requests are usually operation instructions initiated by the video processing system, triggered by various application scenarios such as user playback commands, video transcoding tasks, and real-time communication data streams.

[0053] Hardware codec interfaces are programming interfaces that conform to industry standards (such as VAAPI and Vulkan Video) or vendor specifications, used to abstract away underlying hardware differences. Through these interfaces, the corresponding codec hardware can be invoked, thereby implementing codec functionality via hardware.

[0054] The hardware codec interface corresponds to the specified video format. Therefore, in this step, it is necessary to determine the target video format of the target video frame, and then select the hardware codec interface corresponding to the target video format from a number of pre-configured hardware codec interfaces.

[0055] Step S120: When the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is unavailable, the encoding / decoding result of the target video frame is generated by the software simulation program corresponding to the hardware encoding / decoding interface.

[0056] Each hardware codec interface has corresponding codec hardware to perform codec processing on the video format corresponding to that hardware codec interface. In this step, it is necessary to detect whether the codec hardware corresponding to the hardware codec interface is available, and if the codec hardware is unavailable, the codec result of the target video frame is generated by the software simulation program corresponding to the hardware codec interface.

[0057] The unavailability states include various situations such as physical layer unavailability (hardware not installed or damaged) and logical layer unavailability (driver abnormality, resource conflict, temperature protection), which can be detected through various methods such as abnormal hardware self-test return values ​​(e.g., register read failure), driver initialization timeout, and resource monitoring. This application does not limit the specific detection method. A software simulation program refers to an encoding / decoding program implemented purely in software with hardware interface compatibility, which typically meets requirements such as functional integrity, interface consistency, and output compatibility. Functional integrity means supporting all encoding tools defined by the corresponding video standard (e.g., AVS3's 65 prediction modes); interface consistency means accepting completely identical input parameter structures; and output compatibility means generating bit-accurate or error-controllable results. For example, a software simulation program includes software modules that fully implement video standard algorithms, with functional interfaces equivalent to hardware acceleration, and can be implemented based on standard reference software or open-source codecs, etc.

[0058] By using a software simulation program, encoding and decoding processing can be performed in place of the hardware when the hardware is unavailable. This allows the hardware encoding / decoding interface to be compatible with both hardware and software processing modes, thereby improving its versatility and flexibility. Therefore, in the embodiments provided in this disclosure, upon receiving an encoding / decoding request triggered for a target video frame, the hardware encoding / decoding interface corresponding to the target video format of the target video frame is invoked. Furthermore, if the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is unavailable, the encoding / decoding result of the target video frame is generated by the software simulation program corresponding to the hardware encoding / decoding interface. This approach not only supports hardware implementation but also allows processing based on a software simulation program when hardware is unavailable, thus improving the interface's flexibility and facilitating its application and promotion.

[0059] Furthermore, those skilled in the art can make various modifications and variations to the embodiments disclosed herein:

[0060] In developing this invention, the inventors discovered that traditional hardware codec interfaces suffer from deep coupling between format specifications and hardware implementation. This means that new video standards must wait for the corresponding hardware iterations to be completed before development can begin. Furthermore, hardware codec interfaces with deep hardware coupling exhibit poor cross-platform compatibility; for example, parameter sets from different manufacturers are incompatible, and the hardware's fixed encoding toolkit combination can limit algorithm optimization. In short, traditional hardware-based codec interfaces, due to their deep coupling with the hardware, require many parameters to be defined by the hardware, resulting in poor interface flexibility and making it impossible to develop the hardware codec interface before hardware iterations are complete. To address these issues, this application decouples the interface specification from the hardware, avoiding excessive reliance on hardware during the development process. This improves the universality and flexibility of the hardware codec interface and enables rapid support for new video standards, allowing for the priority completion of hardware codec interface development before the hardware is ready. Accordingly, in this application, the hardware codec interface corresponding to the target video format of the target video frame can be invoked in the following way: according to the interface specification of the hardware codec interface, an interface parameter set corresponding to the target video frame is generated, and an interface call instruction containing the interface parameter set is sent to the hardware codec interface; wherein, the interface specification of the hardware codec interface is used to define the parameter types in the interface parameter set according to the format specification of the target video format.

[0061] The hardware codec interface specification is defined based on the target video format's format specification to limit the types and number of parameters included in the interface parameter set. In specific implementation, the hardware codec interface specification can be determined as follows: standard syntax elements in the video standard format corresponding to the hardware codec interface (such as AVS3's QTBT depth) are mapped to hardware-independent parameters, thereby defining the parameter types of hardware-independent general parameters (such as motion vector precision of 1 / 8 pixel). Furthermore, to facilitate future expansion, optional private parameter fields can be retained to adapt to different hardware manufacturers.

[0062] In specific implementation, the interface parameter set corresponding to the target video frame can be determined in the following way: extract multiple syntax elements defined by the standard according to the interface specification of the hardware codec interface, such as quantization matrix, intra-frame prediction, motion vector, etc. Then, according to the interface specification, convert the extracted multiple syntax elements into an interface parameter set. Taking the AVS standard format as an example, the parameters in the interface parameter set can include at least one of the following: (1) Sequence-level parameters: level, chroma format, bit depth, etc., used to realize the global settings of the video stream; (2) Image-level parameters: quantization parameters (QP), frame type, etc., used to realize frame-level control information; (3) Slice-level parameters: motion vector prediction mode, transform block size, loop filter strength, etc., used to realize local coding control; (4) Extended parameters: used to realize customized processing of different manufacturers, or to adapt to the updated video standard format. Accordingly, in the encoding process, the target video frame is encoded based on the above interface parameter set to obtain a binary bit stream; in the decoding process, it is necessary to parse the encoded binary bit stream and restore the binary bit stream to the original target video frame according to the interface parameter set.

[0063] Optionally, considering that the software simulation scheme may not be able to directly reuse the hardware interface parameter structure, in order to ensure the consistency of the output results, it is necessary to further establish a standardized conversion mechanism from parameter set to bit stream to ensure the consistency of the output. Accordingly, when generating the encoding and decoding results of the target video frame through the software simulation program corresponding to the hardware encoding and decoding interface, it can be achieved in the following way: the interface parameter set is converted by the software simulation program, and the encoding and decoding are performed according to the conversion result. Specifically, in the encoding process, the interface parameter set is converted by the software simulation program to obtain the first conversion result; the encoding result of the target video frame is obtained according to the first conversion result; wherein, the encoding result is represented by a binary bit stream. The first conversion process is used to convert the interface parameters into an intermediate format that can be directly used by the encoder during the encoding process, which may involve at least one of the following processing methods: (1) parameter serialization: converting structured parameters into a linear byte stream; (2) format adaptation: adjusting the parameter format to match the specific encoder requirements. Accordingly, the encoding result of the target video frame can be obtained according to the first conversion result. Similarly, during the decoding process, the interface parameter set is subjected to a second transformation process through a software simulation program to obtain the second transformation result; the binary bitstream is then processed based on the second transformation result to obtain the decoding result of the target video frame. The second transformation process is used to reconstruct the interface parameters into the complete context required for decoding during the decoding process, and may involve at least one of the following processing methods: (1) parameter parsing: reconstructing structured parameters from binary data; (2) semantic recovery: supplementing the implicit parameter semantics of the hardware interface; (3) relation reconstruction: restoring the dependencies and constraints between parameters, etc. Accordingly, the decoding result can be obtained based on the second transformation result.

[0064] Therefore, in the parameter conversion stage, interface parameters can be mapped to standard syntax elements through semantic parsing. Then, using a bitstream organization method, the syntax elements are arranged in the order specified by the standard, and control information such as start codes and length prefixes are added. Additionally, checksums are calculated and inserted to obtain a binary bitstream corresponding to the interface parameter set, thus achieving encoding processing. In the above method, considering the potential differences between hardware and software encoding methods, the interface parameter set is converted based on the software encoding method. Encoding is then performed based on the converted result, improving the efficiency and accuracy of software encoding.

[0065] In one alternative implementation, considering that traditional parameter set designs are fixed and require complete reconstruction of the parameter structure after video standard version upgrades, thus failing to flexibly adapt to the dynamic evolution of the standard, the interface parameter set in this application includes multiple types of parameter sets to facilitate flexible adaptation. Specifically, the aforementioned interface parameter set may include: an image parameter set corresponding to frame-level control parameters, a stripe parameter set corresponding to slice-level coding tool parameters, and an extensible parameter set. The extensible parameter set may include version control parameters, and the parameter values ​​of the version control parameters correspond to the version update results of the target video format.

[0066] The image parameter set typically contains global control parameters at the video frame level, used to describe the coding characteristics of a single frame. Typical parameters may include frame type (I / P / B frame), quantization parameter baseline, resolution information, and timestamps. The slice parameter set defines coding control parameters for local regions within a video frame, managing slice-level processing strategies. Key parameters include motion vector prediction mode, transform block size range, and loop filter strength. The scalable parameter set provides a flexible parameter space for standard evolution and vendor extensions. It can contain version control parameters and reserved fields, exhibiting forward compatibility (new devices can recognize older version parameters) and backward scalability (reserving parameter bits for future use and allowing proprietary parameter extensions for vendor differentiation). Version control parameters identify the version status of the video codec standard, and their values ​​map key nodes in the standard's evolution. Examples include major version numbers (major feature updates), minor version numbers (toolset extensions), and revision numbers (bug fixes). In practical implementation, this can be achieved based on a parameter set interaction mechanism. For example, an image parameter set-dominant principle can be adopted, requiring the strip parameter set to reference a valid image parameter set, while the extended parameter set can selectively associate with either the image parameter set or the strip parameter set. By setting multiple types of parameter sets and configuring extensible parameter sets, the versatility of the hardware codec interface can be improved, enabling it to flexibly adapt to video standards that are in the process of iteration.

[0067] In one optional implementation, when the codec hardware corresponding to the hardware codec interface is unavailable, before generating the codec result of the target video frame through the software simulation program corresponding to the hardware codec interface, the state of the codec hardware corresponding to the hardware codec interface is detected by the interface driver program corresponding to the hardware codec interface. The interface driver program is a control module between the video codec application and the underlying hardware, and its core functions include hardware state management, resource allocation, and instruction translation. In this application, it refers to a driver architecture that integrates hardware drivers and software simulation. The interface driver program further includes a software simulation program and a hardware driver program. The hardware driver program is used to drive the codec hardware to perform codec processing when the codec hardware corresponding to the hardware codec interface is available. The hardware driver program is a software component specifically designed to control the physical codec hardware, responsible for converting standard interface calls into hardware-executable operations. Typical functions include register programming, DMA transfer control, and interrupt handling. The software simulation program is a pure software implementation that provides equivalent functionality when the hardware is unavailable. The unavailable state includes various states in which the encoding and decoding hardware cannot respond to requests normally, including but not limited to: physical layer abnormalities, such as hardware not being powered on, damaged, or not installed; and logical layer failures, such as driver loading failure or register access errors.

[0068] In one alternative implementation, considering that the hardware codec interface is configured for hardware, the storage space configured for this interface may differ from the storage space corresponding to the software simulation method. Therefore, to ensure compatibility between hardware and software codec methods, the codec results can be transmitted across storage spaces. Accordingly, the codec results generated by the software simulation program are stored in the first storage space. After the software simulation program corresponding to the hardware codec interface generates the codec results for the target video frame, the codec results in the first storage space are further transmitted to the second storage space to retrieve the codec results from the second storage space via the hardware codec interface; wherein, the second storage space corresponds to the hardware codec interface.

[0069] The first storage space refers to the system memory area corresponding to the runtime of the software simulation program. It can adopt a general memory management mechanism, and its access permissions are controlled by the operating system. The storage layout meets the software processing requirements (such as using a row storage format). The second storage space is a storage area specifically designed for hardware encoding and decoding interfaces. Its physical addresses can be contiguous or meet hardware alignment requirements (such as 256-byte boundaries), and it supports direct DMA access and zero-copy transfer. Specifically, it may take various forms such as video memory, shared memory, or dedicated cache.

[0070] The process of transferring the encoding / decoding results from the first storage space to the second storage space is essentially a transfer of the encoding / decoding results between different memory domains. This involves the following processing: format conversion (e.g., converting the first format corresponding to software encoding / decoding to the second format corresponding to hardware encoding / decoding), data reassembly (e.g., reassembling row storage into block storage), and adding metadata (e.g., adding timestamps, frame number synchronization, etc.). In practice, the first storage space can be dynamically allocated, enabling on-demand allocation and supporting non-contiguous physical addresses. Therefore, it offers greater access flexibility, enabling arbitrary byte access and low latency, making it particularly suitable for CPU caching. The second storage space can be configured based on hardware constraints, and a pre-allocated memory pool can be used to reduce real-time allocation overhead. Furthermore, batch transmission (aggregating small data packets) can improve transmission efficiency.

[0071] In one alternative implementation, the software emulation program runs on a first device, while the encoding / decoding hardware resides in a second device. Furthermore, the first storage space includes a device memory space corresponding to the first device, and the second storage space includes a device memory space corresponding to the second device. Specifically, the first device can be a CPU device, used as a general-purpose computing device to execute the software emulation program; its memory space can support virtual memory management mechanisms, and its access granularity can be accurate to the byte level. The second device can be a GPU device, used as a dedicated acceleration device, on which encoding / decoding hardware is installed. Device memory space refers to the physical storage area bound to a specific computing device.

[0072] When the first device is a CPU, it may include an application layer, an interface layer, and a driver layer. Specifically, the application layer is used to obtain encoding / decoding requests triggered for a target video frame and send these requests to the interface layer for processing. Correspondingly, the interface layer is configured with a hardware encoding / decoding interface. By calling this interface, the interface driver in the driver layer is triggered to perform subsequent processing. The interface driver in the driver layer may include the aforementioned software simulation program and hardware driver, with the execution of either the software simulation program or the hardware driver determined based on the detected state of the encoding / decoding hardware. The encoding / decoding hardware is located in the second device. Therefore, the video encoding / decoding method in this disclosure is typically executed by the first device, and the encoding / decoding hardware in the second device needs to be called during the operation of the hardware driver. Alternatively, the video encoding / decoding in this disclosure may also be executed by a third device different from the first and second devices. For example, a front-end server (i.e., the third device) can be set up to uniformly process the encoding / decoding requests sent by the client, and the front-end server can call the hardware encoding / decoding interface in the first device for processing. This disclosure does not limit the specific implementation details.

[0073] In one optional implementation, to ensure the accuracy of the pre-configured hardware codec interface and avoid encoding / decoding errors caused by incorrect interface configuration, after generating the encoding / decoding result of the target video frame through the software simulation program corresponding to the hardware codec interface, the encoding / decoding result is further matched with a preset standard result. If the match fails, the interface specification of the hardware codec interface and / or the program execution logic of the software simulation program are adjusted. The preset standard result refers to a reference output conforming to the video codec standard specification, and its generation sources may include: verification bitstream decoding results released by standards organizations, certified software reference model outputs, and consistency data from cross-validation of hardware from multiple vendors. A mismatch means that there is a difference between the encoding / decoding result and the standard result that exceeds the allowable range. Specifically, a mismatch can be characterized by pixel-level differences, syntax element errors (inconsistent frame type, motion vectors, etc.), and timing deviations (decoding delay exceeding standard requirements). Accordingly, the purpose of adjusting the hardware codec interface specification is to dynamically modify the interface parameters, including redefining parameter value ranges (such as adjusting the maximum QP value), adding or deleting parameter items, and modifying the constraints between parameters. The purpose of adjusting the execution logic of the software simulation program is to modify its processing flow, specifically including the following modifications: algorithm implementation replacement (such as replacing standard DCT with fast DCT), flow control optimization (skipping non-critical steps), and precision compensation (floating-point to fixed-point error correction). Therefore, by adjusting the hardware codec interface specification, the types and number of parameters contained in the hardware codec interface can be adjusted; by adjusting the execution logic of the software simulation program, the encoding mode and encoding algorithm can be changed.

[0074] In summary, mismatches can be pre-classified into several types: the first type, determined by the average PSNR of the frame, corresponding to pixel domain mismatch; the second type, determined by the syntax element compliance rate, corresponding to bitstream domain mismatch; and the third type, determined by decoding delay fluctuations, corresponding to temporal domain mismatch. Furthermore, the mismatch level can be determined based on the type and number of mismatches. Correspondingly, the adjustment methods can also be divided into several different levels of adjustment based on the type or level of mismatch. For example, the adjustment levels can include: (1) First-level adjustment: in case of severe mismatch, frame-level adjustment is performed by modifying global parameters; or, adjustments can be made for the coding mode; (2) Second-level adjustment: in case of anomalies in some areas, strip-level adjustment is performed by optimizing local area parameters; (3) Third-level adjustment: in case of mismatch in a few areas, block-level adjustment is achieved by correcting individual coding units.

[0075] In one alternative implementation, considering the ability to switch between hardware and software driver modes, a shared storage area can be used to ensure the normal operation of the hardware driver after switching, allowing both modes to share state data. Accordingly, after the software simulation program corresponding to the hardware codec interface generates the encoding and decoding results of the target video frame, the runtime state data corresponding to the software simulation program is further stored in the shared storage area, so that the hardware driver loads the runtime state data during initialization.

[0076] The operational status data can include: dynamic context information that needs to be persistently saved during video encoding and decoding to ensure the continuity of the processing flow and the recoverability of the state. For example, it can include at least one of the following information to characterize the internal state of the decoder: reference frame management status, used to record the storage location, display order, and timestamp information of decoded reference frames in the decoded image buffer to ensure the correctness of inter-frame prediction; slice processing progress, used to track the decoding completion status of each slice in the current frame and support synchronous control of parallel decoding. It can also include information to characterize dynamic control parameters: for example, bitrate control status, which may include real-time adjustment parameters such as virtual buffer fullness, quantization parameter (QP) adjustment trend, and target bitrate allocation; motion prediction status, such as storing prediction information such as motion vector information of the current frame and reference frame index. The shared storage area is a physical memory area used to achieve cross-processing unit state synchronization. State synchronization enables flexible switching between hardware-driven and software-simulated approaches. In one optional implementation, to facilitate hardware acceleration for new video standards, the encoding / decoding request can be a hardware acceleration request, and correspondingly, the hardware encoding / decoding interface can be a hardware acceleration interface. Furthermore, the target video format corresponds to a video standard format whose release date is later than a preset time threshold. Hardware acceleration requests typically require video operation instructions that call dedicated codec hardware and may have the following characteristics: carrying a hardware acceleration flag, including a device selection identifier, and specifying minimum performance requirements. Correspondingly, the hardware acceleration interface is a standardized hardware access layer provided by the operating system. The preset time threshold is a technical dividing point used to determine the age of a video standard, and can be in various forms such as an absolute threshold (e.g., current date - standard release date) or a relative threshold (e.g., new standard date interval × 1.5). In practice, this threshold can be flexibly set according to factors such as the iteration cycle of the new standard to facilitate the rapid configuration of a dedicated hardware acceleration interface for newly added video standards.

[0077] To facilitate understanding, an example is provided below to illustrate the specific implementation details of the video encoding and decoding method provided in this application. This example aims to provide a software-bootstrapping-based video encoding and decoding hardware acceleration interface design scheme, involving technical fields such as video encoding and decoding hardware acceleration, software interface design, and system integration. In this example, the hardware encoding and decoding interface is a hardware acceleration interface (also called an acceleration sub-interface), and the encoding and decoding request is a hardware acceleration request. Video hardware acceleration refers to using hardware modules (such as GPUs) to replace software in implementing video encoding and decoding, thereby improving video encoding and decoding efficiency, reducing CPU load, and saving power. On general-purpose computer operating systems (such as Windows or Linux), video hardware acceleration requires the implementation of corresponding drivers based on certain software interfaces. These drivers then drive the relevant hardware modules to complete the video encoding and decoding tasks.

[0078] In related technologies, video codec acceleration interfaces implemented based on video codec acceleration interface frameworks can only support a limited number of video standards or formats. For example, whether the video codec acceleration interface is based on LINUX VAAPI or Vulkan Video Extension, it only supports a limited number of video standards or formats: for instance, Vulkan Video only supported H.264 and H.265 encoding and decoding before 2023. Both LINUX VAAPI and Vulkan Video Extension belong to video codec acceleration interface frameworks. Consequently, when new hardware needs to support new video standard formats, such as AVS / AVS2, through existing video codec acceleration interface solutions, it is usually necessary to wait for the relevant hardware manufacturers to produce hardware that supports AVS / AVS2 encoding and decoding before adding AVS / AVS2 standard-related sub-interface definitions to the VAAPI or VulkanVideo video codec acceleration interface framework, then developing the corresponding driver, and finally performing integration testing. The entire development cycle is relatively long.

[0079] Figure 2 A schematic diagram illustrating the development of a video codec hardware acceleration interface in related technologies is shown. For example... Figure 2As shown, the video encoding and decoding system includes an application layer, an interface layer, a driver layer, and a hardware layer. The application layer can include various multimedia frameworks, such as: (1) FFMpeg: an open-source multimedia framework providing cross-platform audio and video encoding and decoding capabilities; (2) MPV: a lightweight media player developed based on MPlayer; and (3) Chrome: a globally widely used browser with built-in WebRTC and WebM support. The interface layer is used to implement the video encoding and decoding hardware acceleration interface, specifically including multiple acceleration sub-interfaces corresponding to various different video standards. For example, Figure 2 The diagram illustrates the H.264 acceleration sub-interface corresponding to the H.264 video standard, the H.265 acceleration sub-interface corresponding to the H.265 video standard, the AVS acceleration sub-interface corresponding to the AVS video standard, and the AVS2 acceleration sub-interface corresponding to the AVS2 video standard. Correspondingly, the driver layer includes multiple hardware drivers corresponding to the various acceleration sub-interfaces, such as... Figure 2 The diagram shows the H.264 hardware driver corresponding to the H.264 acceleration sub-interface, the H.265 hardware driver corresponding to the H.265 acceleration sub-interface, the AVS hardware driver corresponding to the AVS acceleration sub-interface, and the AVS2 hardware driver corresponding to the AVS2 acceleration sub-interface. Additionally, the hardware layer includes multiple hardware modules corresponding to these hardware drivers, such as... Figure 2 The diagram shows the H264 codec hardware module corresponding to the H264 hardware driver, the H265 codec hardware module corresponding to the H265 hardware driver, the AVS codec hardware module corresponding to the AVS hardware driver, and the AVS2 codec hardware module corresponding to the AVS2 hardware driver.

[0080] Therefore, in existing hardware acceleration implementations, the acceleration sub-interface corresponding to a certain video standard needs to rely on the corresponding codec hardware. Consequently, this acceleration sub-interface needs to be tightly coupled with the codec hardware; for example, the parameters in the acceleration sub-interface need to be associated with physical components such as registers and clocks in the hardware. Therefore, it is necessary to first design the codec acceleration hardware corresponding to the specified video standard format, then add the acceleration sub-interface corresponding to that specified video standard format at the interface layer, and finally add the driver implementation corresponding to that acceleration sub-interface at the driver layer. For example, Figure 2 The AVS acceleration sub-interface and AVS2 acceleration sub-interface within the dashed box can only be implemented after the corresponding codec hardware is developed. This demonstrates that the implementation process of traditional acceleration sub-interfaces lags behind the hardware development process, resulting in low timeliness and an inability to quickly configure corresponding acceleration sub-interfaces for new video standards, thus reducing interface development efficiency.

[0081] Furthermore, interface frameworks such as VAAPI or Vulkan Video are hardware acceleration interface frameworks commonly used by multiple hardware manufacturers. When a manufacturer needs to add an acceleration sub-interface based on this framework to support a new video codec standard, the manufacturer needs to verify and demonstrate the rationality and universality of the newly added acceleration sub-interface so that it can be accepted and reused by other manufacturers. To achieve this, the manufacturer may need to implement the hardware driver corresponding to the new acceleration sub-interface and make the hardware driver open source for review by other manufacturers. However, since the hardware driver involves some hardware product-related content, open source hardware drivers may lead to the leakage of intellectual property information of the hardware product. For the above reasons, it is generally not convenient to make hardware drivers open source, thus further increasing the difficulty of promoting acceleration sub-interfaces.

[0082] It is evident that traditional hardware-driven acceleration sub-interfaces suffer from numerous problems, such as long development cycles and difficulty in rapid deployment.

[0083] To address the aforementioned issues and avoid hardware-platform-based verification during the collaborative development of hardware acceleration sub-interfaces, this example proposes a software-bootstrapping-based hardware codec acceleration sub-interface design and implementation scheme. In this example, the driver does not completely rely on external hardware to complete the encoding and decoding tasks. Instead, it integrates specific standard video codecs within the driver itself through software, providing a hardware-independent video acceleration interface design verification scheme that facilitates the testing and verification of the interface itself.

[0084] Figure 3 A schematic diagram of the video encoding / decoding system in this example is shown. Figure 3 As shown, this example proposes a hybrid hardware and software driver implementation scheme. In this scheme, some drivers need to drive external hardware to complete encoding and decoding tasks. For example, for H.264 / H.265 encoding and decoding tasks, it is necessary to use the H.264 / H.265 hardware driver to drive the H.264 / H.265 encoding and decoding hardware module for encoding and decoding processing. Other drivers directly complete the encoding and decoding tasks through software implementation. For example, for AVS / AVS2 encoding and decoding tasks, it directly uses the AVS / AVS2 software driver to complete the encoding and decoding tasks without calling the hardware module.

[0085] Therefore, in this example, the acceleration sub-interface corresponding to the specified video standard (such as the acceleration sub-interface corresponding to the newly added video standard) can be directly implemented through software driver. This makes it easier to define the acceleration sub-interface in advance before the hardware development is completed, and avoids the information leakage problem caused by the open source of the hardware driver.

[0086] The hardware codec acceleration interface design scheme based on software bootstrapping provided in this example mainly includes the following process: First, add an acceleration sub-interface for a specific video standard in the interface layer, and then add the codec software for that specific video standard in the driver layer.

[0087] Specifically, the design scheme of this hardware codec acceleration interface mainly includes the following steps:

[0088] Step 1: Configure the hardware acceleration interface corresponding to the specified video standard in the video codec hardware acceleration interface layer.

[0089] This involves adding the interface definition information (i.e., interface specification) of the hardware acceleration interface corresponding to the specified video standard to the video codec hardware acceleration interface layer. The interface definition information is typically determined based on the syntax elements and hardware abstraction results of the standard document; for example, it usually only includes hardware-independent information such as necessary algorithm parameters and scalable parameters. During interface configuration, hardware-related parameters and vendor-customized parameters should be avoided, and the number of standard-related parameters and fields should be increased as much as possible.

[0090] In practical implementation, key functions, such as computationally intensive functions or functions used for encoding and decoding, can be identified from the reference software corresponding to the specified video standard. During subsequent hardware development, these key functions can be replaced with hardware-based solutions to achieve hardware acceleration. Correspondingly, key syntax elements from the video standard, such as intra-frame prediction, motion vectors, and quantization matrices, are extracted based on these key functions. The interface definition information is then configured according to these key syntax elements. The interface layer converts parameters passed from the application into a format recognizable by the driver layer. This conversion process typically strictly adheres to standard specifications and does not introduce any hardware-specific parameters. It can usually be converted into sequence parameters (such as video size) and image parameters (such as frame type).

[0091] Step 2: Configure the software codec driver corresponding to the specified video standard in the driver layer.

[0092] Software encoding and decoding can be implemented in various ways, such as reference codecs corresponding to video standards or open-source codecs. This example does not limit the specific details.

[0093] Step 3: Verify the functionality of the configured hardware acceleration interface based on the software codec driver.

[0094] Among them, based on the software codec driver, the video codec function of the hardware acceleration interface corresponding to the newly added standard was tested and verified to prove that the codec application implementation of the corresponding standard can be completed based on the sub-interface.

[0095] Step 4: Once hardware capable of supporting the specified video standard exists, develop the corresponding hardware driver based on the hardware acceleration interface.

[0096] Once the hardware supports acceleration for a new standard, a corresponding hardware driver is developed based on the new hardware acceleration sub-interface (i.e., hardware acceleration interface) to drive the hardware to complete the encoding and decoding tasks, thereby achieving true hardware acceleration for encoding and decoding.

[0097] In summary, this example can solve at least the following technical problems: (1) When the video codec hardware is not implemented, a hardware acceleration sub-interface can be developed in advance, and the interface itself can be tested and verified based on the software codec driver solution, thereby accelerating the development and promotion of the interface. (2) Without disclosing the hardware driver, only disclosing the software codec driver can guide other hardware manufacturers to connect to the hardware acceleration sub-interface, thereby achieving the confidentiality of hardware intellectual property rights. (3) For some low-performance and low-usage video standards, such as MPEG2 / RMVB / H263, the software codec driver can also directly replace the hardware design to achieve the delivery of the final product, thereby simplifying the complexity of hardware design.

[0098] In summary, this example achieves at least the following beneficial effects: defining and verifying the feasibility and usability of hardware acceleration interfaces through software bootstrapping. It decouples the definition of the hardware acceleration interface from the specific hardware implementation, thereby eliminating the hardware acceleration interface's dependence on hardware and allowing its integration to proceed synchronously with hardware development, thus accelerating the overall system development process. Software bootstrapping refers to a technical method that utilizes the complete implementation of the video standard reference software, generates hardware acceleration interface specifications through structured modification, and completes self-verification. Essentially, it establishes a closed-loop verification system from the software reference implementation to the hardware interface design, realizing an innovative approach of "defining hardware interfaces with software."

[0099] Figure 4 This is a block diagram of a video encoding / decoding apparatus provided in an embodiment of this disclosure. (Refer to...) Figure 4 This disclosure provides a video encoding / decoding apparatus, which includes:

[0100] The response module 41 is adapted to call the hardware codec interface corresponding to the target video format of the target video frame in response to a codec request triggered for the target video frame.

[0101] The generation module 42 is adapted to generate the encoding and decoding result of the target video frame through the software simulation program corresponding to the hardware encoding and decoding interface when the encoding and decoding hardware corresponding to the hardware encoding and decoding interface is unavailable.

[0102] In one alternative implementation, the response module is specifically adapted to:

[0103] According to the interface specification of the hardware codec interface, determine the interface parameter set corresponding to the target video frame, and send an interface call instruction containing the interface parameter set to the hardware codec interface;

[0104] The interface specification of the hardware codec interface is used to define the types of parameters in the interface parameter set according to the format specification of the target video format.

[0105] In one alternative implementation, the generation module is specifically adapted to:

[0106] During the encoding process, the interface parameter set is subjected to a first conversion process by the software simulation program to obtain a first conversion result; the encoding result of the target video frame is obtained based on the first conversion result; wherein, the encoding result is represented by a binary bit stream;

[0107] During the decoding process, the interface parameter set is subjected to a second conversion process by the software simulation program to obtain a second conversion result; the binary bit stream is processed according to the second conversion result to obtain the decoding result of the target video frame.

[0108] In one optional implementation, the interface parameter set includes: an image parameter set, a strip parameter set, and an extensible parameter set;

[0109] The scalable parameter set includes version control parameters, and the parameter values ​​of the version control parameters correspond to the version update results of the target video format.

[0110] In one alternative implementation, the generation module is further adapted to:

[0111] The status of the codec hardware corresponding to the hardware codec interface is detected by the interface driver corresponding to the hardware codec interface.

[0112] The interface driver includes the software simulation program and the hardware driver; the hardware driver is used to drive the encoding / decoding hardware to perform encoding / decoding processing when the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is in an available state.

[0113] In one optional implementation, the encoding / decoding result generated by the software simulation program is stored in a first storage space, and the generation module is further adapted to:

[0114] The encoding and decoding results in the first storage space are transmitted to the second storage space so as to obtain the encoding and decoding results from the second storage space through the hardware encoding and decoding interface; wherein, the second storage space corresponds to the hardware encoding and decoding interface.

[0115] In one alternative implementation, the software simulation program runs on a first device, and the hardware driver runs on a second device corresponding to the codec hardware.

[0116] Furthermore, the first storage space includes: device memory space corresponding to the first device, and the second storage space includes: device memory space corresponding to the second device.

[0117] In one alternative implementation, the generation module is further adapted to:

[0118] The encoding / decoding result is matched with a preset standard result;

[0119] If the matching fails, adjust the interface specification of the hardware codec interface and / or the program execution logic of the software simulation program.

[0120] In one alternative implementation, the generation module is further adapted to:

[0121] The runtime status data corresponding to the software simulation program is stored in a shared storage area so that the hardware driver loads the runtime status data during the initialization process.

[0122] In one alternative implementation, the encoding / decoding request is a hardware acceleration request, the hardware encoding / decoding interface is a hardware acceleration interface, and the target video format corresponds to a video standard format whose release time is later than a preset time threshold.

[0123] In addition, this disclosure also provides a video encoding and decoding system, the system including Figure 4 The video encoding / decoding device shown. Additionally, this system can... Figure 2 or Figure 3 The architecture implementation is shown.

[0124] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0125] Reference Figure 5This disclosure provides an electronic device, which includes: at least one processor 401; at least one memory 402; and one or more I / O interfaces 403 connected between the processor 401 and the memory 402; wherein the memory 402 stores one or more computer programs that can be executed by the at least one processor 401, and the one or more computer programs are executed by the at least one processor 401 to enable the at least one processor 401 to perform the above-described method.

[0126] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program implements the above-described method when executed by a processor / processor core. The computer-readable storage medium may be volatile or non-volatile.

[0127] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0128] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0129] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0130] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0131] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0132] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0133] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0137] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A video encoding and decoding method, characterized in that, The method includes: In response to a codec request triggered for a target video frame, the hardware codec interface corresponding to the target video format of the target video frame is invoked; When the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is unavailable, the encoding / decoding result of the target video frame is generated by the software simulation program corresponding to the hardware encoding / decoding interface.

2. The method according to claim 1, characterized in that, The invocation of the hardware codec interface corresponding to the target video format of the target video frame includes: According to the interface specification of the hardware codec interface, determine the interface parameter set corresponding to the target video frame, and send an interface call instruction containing the interface parameter set to the hardware codec interface; The interface specification of the hardware codec interface is used to define the types of parameters in the interface parameter set according to the format specification of the target video format.

3. The method according to claim 2, characterized in that, The generation of the encoding and decoding result of the target video frame through the software simulation program corresponding to the hardware encoding and decoding interface includes: During the encoding process, the interface parameter set is subjected to a first conversion process by the software simulation program to obtain a first conversion result; the encoding result of the target video frame is obtained based on the first conversion result; wherein, the encoding result is represented by a binary bit stream; During the decoding process, the interface parameter set is subjected to a second conversion process by the software simulation program to obtain a second conversion result; the binary bit stream is processed according to the second conversion result to obtain the decoding result of the target video frame.

4. The method according to claim 3, characterized in that, The interface parameter set includes: image parameter set, strip parameter set, and scalable parameter set; The scalable parameter set includes version control parameters, and the parameter values ​​of the version control parameters correspond to the version update results of the target video format.

5. The method according to any one of claims 1-4, characterized in that, Before generating the encoding / decoding result of the target video frame through the software simulation program corresponding to the hardware encoding / decoding interface when the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is unavailable, the method further includes: The status of the codec hardware corresponding to the hardware codec interface is detected by the interface driver corresponding to the hardware codec interface. The interface driver includes a hardware driver; the hardware driver is used to drive the encoding / decoding hardware to perform encoding / decoding processing when the encoding / decoding hardware corresponding to the hardware encoding / decoding interface is in an available state.

6. The method according to claim 5, characterized in that, The encoding / decoding result generated by the software simulation program is stored in the first storage space. After the software simulation program corresponding to the hardware encoding / decoding interface generates the encoding / decoding result of the target video frame, the method further includes: The encoding / decoding result in the first storage space is transmitted to the second storage space to obtain the encoding / decoding result from the second storage space through the hardware encoding / decoding interface; wherein the second storage space corresponds to the hardware encoding / decoding interface; wherein the software simulation program runs in the first device, and the encoding / decoding hardware is located in the second device; Furthermore, the first storage space includes: device memory space corresponding to the first device, and the second storage space includes: device memory space corresponding to the second device.

7. The method according to claim 5, characterized in that, After generating the encoding and decoding result of the target video frame through the software simulation program corresponding to the hardware encoding and decoding interface, the method further includes: The runtime status data corresponding to the software simulation program is stored in a shared storage area so that the hardware driver loads the runtime status data during the initialization process.

8. The method according to any one of claims 1-4, characterized in that, After generating the encoding and decoding result of the target video frame through the software simulation program corresponding to the hardware encoding and decoding interface, the method further includes: The encoding / decoding result is matched with a preset standard result; If the matching fails, adjust the interface specification of the hardware codec interface and / or the program execution logic of the software simulation program.

9. The method according to any one of claims 1-4, characterized in that, The encoding / decoding request is a hardware acceleration request, and the hardware encoding / decoding interface is a hardware acceleration interface; furthermore, the target video format corresponds to a video standard format whose release time is later than a preset time threshold.

10. A video encoding / decoding apparatus, characterized in that, The device includes: The response module is adapted to respond to a codec request triggered for a target video frame by calling a hardware codec interface corresponding to the target video format of the target video frame. The generation module is adapted to generate the encoding and decoding result of the target video frame through a software simulation program corresponding to the hardware encoding and decoding interface when the encoding and decoding hardware corresponding to the hardware encoding and decoding interface is unavailable.

11. A video encoding and decoding system, characterized in that, The system includes the video encoding / decoding apparatus of claim 10.

12. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-9.

14. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, characterized in that, When the computer-readable code is run in an electronic device, the processor in the electronic device performs the method according to any one of claims 1 to 9.

Images