Video encoding and decoding system and device, driving method, storage medium and electronic equipment

Abstract

The present disclosure provides a video codec system, driver method, device, storage medium, and electronic device. The video codec system comprises a user-mode portion, including an application interface layer, a protocol parsing layer, and a kernel adaptation layer; and a kernel-mode portion, including a kernel interface layer and a hardware driver layer. Through a unified codec framework and driver interface, the present disclosure enables adaptation to different protocol formats, kernel driver versions, and hardware versions, reducing development costs and improving system scalability and maintainability.

Description

Technical Field

[0001] The present disclosure relates to the field of video coding and decoding technology, and more particularly to a video coding and decoding system and device, a driving method, a storage medium, and an electronic device. Background Art

[0002] With the rapid development of video technology, video codecs are playing an increasingly important role in various application scenarios. However, video data in different application scenarios often uses different protocol formats (such as H264, H265, VP9, ​​AV1, etc.). Furthermore, device hardware platforms and operating system kernel driver versions vary widely. This requires video codec systems to be highly flexible and compatible to adapt to changing market demands and technical standards. Summary of the Invention

[0003] The present disclosure provides a video codec system and device, a driving method, a storage medium, and an electronic device. Through a unified codec framework and driver interface, it can adapt to different protocol formats, different kernel driver versions, and different hardware versions, reduce development costs, and improve the scalability and maintainability of the system.

[0004] In the first aspect, the embodiment of the present disclosure provides a video codec system. The video codec system includes: a user state part, including an application interface layer, a protocol parsing layer and a kernel adaptation layer; and a kernel state part, including a kernel interface layer and a hardware driver layer, wherein the application interface layer is configured to receive commands, parameters and / or codec data of an application program, and send them to the protocol parsing layer, wherein the protocol parsing layer is configured to create a user state context, and if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the commands, parameters and / or parsed data to the kernel adaptation layer, wherein the kernel adaptation layer is configured to select an adapted kernel driver version, open the corresponding character device file to obtain a file descriptor, and generate the corresponding version of the kernel command and parameters, if the parsed data exists, the kernel input data and / or data description structure of the corresponding version is generated, and the kernel command, the parameters, the kernel input data and / or the data description structure are sent to the kernel interface layer through the file descriptor, wherein the kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameters, the kernel input data and / or the data description structure, and send them to the hardware driver layer, wherein the hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and process the hardware task using the corresponding IP hardware.

[0005] In an implementation of the first aspect, the hardware driver layer is configured to return the processed output data to the kernel interface layer; the kernel interface layer is configured to integrate the output data into the output part in the data description structure and return it to the kernel adaptation layer; the kernel adaptation layer is configured to parse and process the output part in the data description structure to obtain processed data, and return it to the application interface layer; and the application interface layer is configured to return the processed data to the application.

[0006] In an implementation of the first aspect, the application interface layer includes: a channel application interface, a data sending interface, a data receiving interface, a control command interface, and a channel closing interface.

[0007] In an implementation of the first aspect, the protocol parsing layer includes: a command control module, a protocol parsing module, a video framing module, a syntax parsing module, a syntax error tolerance module, a bit rate control module, and a reference frame management module.

[0008] In an implementation of the first aspect, the kernel adaptation layer includes: a command control module, a drive data generation module, a kernel communication module, and a memory management module.

[0009] In an implementation of the first aspect, the kernel interface layer includes: an application channel interface, an input data interface, an output data interface, a command control interface, and a channel closing interface.

[0010] In an implementation of the first aspect, the hardware driver layer includes: a channel management module, a register generation module, a data processing module, a task queue management module, a task status management module, a hardware configuration module, and a hardware interrupt response module.

[0011] In an implementation of the first aspect, the data description structure includes: a file descriptor corresponding to the memory space, an input data offset and length, the number of register write segments, the offset and length of each write segment, the number of register read segments, and the offset and length of each read segment.

[0012] In an implementation of the first aspect, the kernel adaptation layer is further configured to: apply for memory space through the system's memory requester interface, and obtain the file descriptor corresponding to the memory space; memory-map the file descriptor corresponding to the memory space to obtain a user-state virtual address; and write the data to be transmitted to the kernel into the memory space through the user-state virtual address, and record the offset and data length of the corresponding data in the memory space in the data description structure; the kernel interface layer is further configured to: obtain the data description structure, memory-map the file descriptor corresponding to the memory space to obtain a kernel-state virtual address; and obtain the user-state transmitted data through the kernel-state virtual address based on the offset and data length in the data description structure.

[0013] In an implementation of the first aspect, the video codec system also includes the application, which is configured to: send codec commands and parameters through the application channel application interface of the application interface layer to obtain a codec channel number; send the codec channel number and input data through the application data sending interface of the application interface layer; send the codec channel number through the application data receiving interface and wait for output data; and after all input data processing is completed, send the codec channel number through the application interface layer's channel closing interface to close the corresponding channel.

[0014] In an implementation of the first aspect, the application channel interface of the application interface layer is configured to apply for a user-state data space for storing user-state context information of the codec channel according to the command and parameters after receiving the command and parameters of the application program, and send the command and parameters to the command control module in the protocol parsing layer; the command control module in the protocol parsing layer is configured to send the command and parameters to the command control module in the kernel adaptation layer; the command control module in the kernel adaptation layer is configured to select an adapted kernel driver version according to the command and parameters, open the corresponding character device file to obtain the file descriptor, generates a corresponding version of kernel commands and parameters, and sends the kernel commands and parameters to the channel application interface in the kernel interface layer through the file descriptor; the channel application interface in the kernel interface layer is configured to apply for a kernel-state data space for storing kernel-state context information of the codec channel according to the received kernel commands and parameters, and register it under the file descriptor, and send the kernel commands and parameters to the channel management module in the hardware driver layer; the channel management module in the hardware driver layer is configured to apply for the codec channel number according to the kernel commands and parameters, and return the codec channel number layer by layer.

[0015] In an implementation of the first aspect, the user-state context information is used to store codec channel data in the application interface layer, the protocol parsing layer, and the kernel adaptation layer, including: protocol information, codec channel number, kernel driver version, character device file, file descriptor, and calculation cache data.

[0016] In an implementation of the first aspect, the kernel state context information is used to store codec channel data in the kernel interface layer and the hardware driver layer, including: protocol information, codec channel number, hardware driver information, IP hardware information, output queue, and calculation cache data.

[0017] In an implementation of the first aspect, the data sending interface of the application interface layer is configured to receive the codec channel number and input data of the application, and forward it to the protocol parsing module in the protocol parsing layer; the protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, select the adapted protocol parsing module according to the protocol information in the user state context to parse the input data, and send the parsed data to the driver data generation module in the kernel adaptation layer; the driver data generation module in the kernel adaptation layer is configured to generate the kernel input data and the data description structure according to the parsed data, and send the parsed data to the driver data generation module in the kernel adaptation layer through the file in the user state context. The descriptor sends the kernel input data and the data description structure to the input data interface in the kernel interface layer; the input data interface in the kernel interface layer is configured to send the kernel input data and the data description structure to the data processing module in the hardware driver layer according to the hardware driver information in the kernel state context under the file descriptor; the data processing module in the hardware driver layer is configured to process the kernel input data and the data description structure, the register generation module is configured to obtain input register data according to the processed data and generate a hardware task, and the task status management module is configured to mark the hardware task as a waiting state and send it to the corresponding hardware driver task queue.

[0018] In an implementation manner of the first aspect, the information in the hardware task includes: register segmentation information, control register data, input register data, output register data, and a status flag.

[0019] In an implementation of the first aspect, the hardware-driven task queue includes: a first-in-first-out based serial single queue and a priority-based serial multiple queue.

[0020] In an implementation of the first aspect, the hardware driver is configured to select a task from the hardware driver task queue as a target task, the task status management module is configured to mark the target task as an execution state, initialize a timer with a preset time, and use the hardware configuration module to configure the input register data in the target task for the corresponding IP hardware; if the hardware interrupt response module in the hardware driver layer captures a hardware interrupt, the timer is turned off, and the output register of the hardware is read according to the data description structure in the target task. The task status management module marks the target task as an interrupt completion state, and resets the corresponding IP hardware if it is an error interrupt. The task queue management module sends the target task to the output queue of the corresponding kernel state context; if the hardware interrupt response module in the hardware driver layer captures a timer timeout, the task status management module marks the target task as a timeout state, turns off the timer, resets the corresponding IP hardware, and the task queue management module sends the target task to the output queue of the corresponding kernel state context.

[0021] In an implementation of the first aspect, the output queue of the kernel mode context is a serialized single queue based on a first-in-first-out principle.

[0022] In an implementation of the first aspect, the receiving data interface of the application interface layer is configured to receive the codec channel number of the application and forward it to the protocol parsing module in the protocol parsing layer; the protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, and send the protocol information in the context to the driver data generation module in the kernel adaptation layer; the driver data generation module in the kernel adaptation layer is configured to generate the kernel command and parameters according to the protocol information, and send the kernel command and parameters to the output data interface in the kernel interface layer through the file descriptor in the user state context; the output data interface in the kernel interface layer is configured to select one as the target task from the output queue in the kernel state context under the file description according to the kernel command and parameters, set the return value to correct if the target task is in an interrupt completion state, otherwise set the return value to error, return the output data and the return value in the target task layer by layer, and the task queue management module releases the data space of the target task.

[0023] In a second aspect, the embodiment of the present disclosure provides a video codec driving method. The video codec driving method includes: the application interface layer receives the application's commands, parameters and / or codec data, and sends them to the protocol parsing layer; the protocol parsing layer creates a user state context, and if the codec data exists, selects an adapted protocol parsing module to parse the codec data to generate parsed data, and sends the commands, parameters and / or parsed data to the kernel adaptation layer; the kernel adaptation layer selects an adapted kernel driver version, opens the corresponding character device file to obtain a file descriptor, generates a kernel command and parameter of a corresponding version, and if the parsed data exists, generates a kernel driver of a corresponding version. The kernel input data and / or data description structure are received by the kernel interface layer, and the kernel command, the parameters, the kernel input data and / or the data description structure are sent to the kernel interface layer through the file descriptor; the kernel interface layer creates a kernel state context, receives the kernel command, the parameters, the kernel input data and / or the data description structure, and sends them to the hardware driver layer; and the hardware driver layer selects an adapted hardware driver, generates a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and processes the hardware task using the corresponding IP hardware.

[0024] In a third aspect, the embodiment of the present disclosure provides a video codec device. The video codec device includes: an application interface layer, configured to receive commands, parameters and / or codec data of an application program and send them to a protocol parsing layer; a protocol parsing layer, configured to create a user state context, and if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the commands, parameters and / or parsed data to a kernel adaptation layer; a kernel adaptation layer, configured to select an adapted kernel driver version, open the corresponding character device file to obtain a file descriptor, generate kernel commands and parameters of the corresponding version, and generate the corresponding version if the parsed data exists. The kernel input data and / or data description structure is received by the kernel interface layer, and the kernel command, the parameters, the kernel input data and / or the data description structure are sent to the kernel interface layer through the file descriptor; the kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameters, the kernel input data and / or the data description structure, and send them to the hardware driver layer; and the hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and process the hardware task using the corresponding IP hardware.

[0025] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the video codec driving method provided by the embodiment of the present disclosure is implemented.

[0026] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory configured to store an executable program; and a processor configured to call the program so that the electronic device executes the video codec driving method provided according to an embodiment of the present disclosure.

[0027] The video codec system provided according to the embodiment of the present disclosure includes an application interface layer, a protocol parsing layer, a kernel adaptation layer, a kernel interface layer and a hardware driver layer. The application interface layer provides a simple application interface for upper-layer applications to call; the protocol parsing layer is responsible for parsing video streams in different protocol formats and converting them into a unified internal format; the kernel adaptation layer is responsible for interacting with different hardware platforms and operating system kernel drivers to realize the input and output of data between user mode and kernel mode; the hardware driver layer adapts to different IP hardware and uses IP hardware to realize hardware acceleration of video codec. In this way, all hardware drivers of different chip versions, different kernel versions and different hardware platforms can be compatible under one framework, which maximizes the reusability and portability of the driver. The simple application interface does not need to consider the differences between various underlying platforms, greatly shortens the development time of the application program, and speeds up the update and iteration speed of the product.

[0028] In addition, in the embodiments of the present disclosure, the protocol parsing layer integrates a variety of protocol parsing modules to achieve comprehensive support for different video protocol formats, such as H264, H265, VP9, ​​AV1, etc. These parsing modules can automatically identify and parse a variety of mainstream video transmission protocols and convert them into a unified format that the codec system can handle. The encoding module in the protocol parsing layer also supports the configuration and adjustment of multiple encoding parameters to meet the needs of different application scenarios. At the same time, the video codec system also provides an extensible protocol interface, allowing developers to add new protocol parsing modules and encoding modules as needed.

[0029] Furthermore, the video codec driver method provided by the embodiments of the present disclosure is similar to a plug-in mechanism, allowing developers to write corresponding driver plug-ins based on specific hardware platforms and operating system kernel driver versions. These plug-ins seamlessly integrate with the codec system by implementing unified interface specifications. When the hardware platform or operating system kernel needs to be replaced or upgraded, only the corresponding driver plug-in needs to be replaced, without modifying the core code of the codec system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the specific embodiments of the present disclosure, the following briefly introduces the drawings required for describing the specific embodiments.

[0031] Figure 1 Shown is a schematic diagram of the system framework of video encoding and decoding in an embodiment of the present disclosure.

[0032] Figure 2A Shown is a schematic diagram of interfaces of various layers of a video encoding and decoding system in an embodiment of the present disclosure.

[0033] Figure 2B Shown is a structural diagram of a data description structure in an embodiment of the present disclosure.

[0034] Figure 3 Shown is a schematic diagram of calling a video encoding application in an embodiment of the present disclosure.

[0035] Figure 4A Shown is a target task state transition flow chart in an embodiment of the present disclosure.

[0036] Figure 4B Shown is a schematic diagram of the target task status execution in an embodiment of the present disclosure.

[0037] Figure 5A Shown is a schematic diagram of scheduling hardware tasks for single-IP hardware in an embodiment of the present disclosure.

[0038] Figure 5B Shown is a schematic diagram of hardware task scheduling for multi-IP hardware in an embodiment of the present disclosure.

[0039] Figure 6 Shown is a flowchart of a driving method provided by an embodiment of the present disclosure.

[0040] Figure 7 Shown is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The following describes the embodiments of the present disclosure through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present disclosure from the content disclosed in this specification. The present disclosure can also be implemented or applied through different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0042] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0043] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. Therefore, the illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0044] Figure 1 FIG. 1 is a schematic diagram of a system framework for video encoding and decoding in an embodiment of the present disclosure. Figure 1 As shown, the system includes a user-mode portion and a kernel-mode portion, which run in the user-mode and user-mode portions of the operating system, respectively. The user-mode portion includes an application interface layer 10, a protocol parsing layer 11, and a kernel adaptation layer 12, while the kernel-mode portion includes a kernel interface layer 20 and a hardware driver layer 21. The application interface layer 10 provides a concise API (Application Programming Interface) for application calls. The protocol parsing layer 11 includes an encoding module and a decoding module. The encoding module is primarily responsible for bit rate control and reference frame management, while the decoding module is primarily responsible for parsing the video stream according to the protocol format and converting it into parsed data in a unified format. The kernel adaptation layer 12 converts the parsed data into compatible kernel input data based on the hardware platform and operating system kernel version, and interacts with the kernel driver. The kernel interface layer 20 provides the kernel adaptation layer with a kernel driver interface for the current hardware platform and system kernel version. The hardware driver layer 21 is responsible for hardware task scheduling, register configuration generation, register configuration, and hardware result processing.

[0045] Figure 2A FIG. 1 is a schematic diagram of interfaces of various layers of a video encoding and decoding system according to an embodiment of the present disclosure. Figure 2A As shown, the application interface layer provides application interfaces, including the channel application interface open_channel, the data sending interface send_input, the data receiving interface wait_output, the control command interface control_cmd and the channel closing interface close_channel.

[0046] The protocol parsing layer includes an encoding module and a decoding module, and is divided according to different protocols. For example, Figure 2A As shown, the encoding module can support h264 and h265 protocols, and the decoding module can support h264, h265 and vp9 protocols.

[0047] In some implementations, the protocol parsing layer may include: a command control module, a protocol parsing module, a video framing module, a syntax parsing module, a syntax error tolerance module, a bit rate control module, and a reference frame management module.

[0048] The kernel adaptation layer is further subdivided according to different hardware and different system kernel versions based on the protocol division of the protocol parsing layer. It includes two versions, v1 and v2, which correspond to the hal_v1 and hal_v2 versions of the kernel interface layer respectively.

[0049] In some implementations, the kernel adaptation layer may include: a command control module, a driver data generation module, a kernel communication module, and a memory management module.

[0050] In some implementations, the kernel interface provided by the kernel interface layer includes a channel application interface open_channel, an input data interface send_data, an output data interface wait_data, a control command interface control_cmd, and a channel closing interface close_channel.

[0051] User-mode applications cannot directly access hardware; they must enter kernel mode and call the corresponding driver to read and write to the hardware. Character device files are a special type of device file and a common way for user mode to access kernel-mode drivers. Common hardware drivers create specific character device files, such as / dev / mpp_service. User mode opens the corresponding character device file for the desired hardware driver using the open interface, obtaining a file descriptor (fd) and establishing a connection with the hardware driver through this fd. Conversely, close(fd) closes the connection between fd and the hardware driver. User mode sends the fd, command cmd, and data data to the kernel interface layer using the ioctl(fd, cmd, data) interface. In some implementations, cmd can be encapsulated into a custom command for user mode to call.

[0052] The hardware driver layer is the main body that manages and utilizes IP hardware to perform hardware tasks. This layer is divided into modules according to different protocols. For example, Figure 2A As shown, the IP hardware supports h264 encoding, h265 encoding, h264 decoding, h265 decoding and vp9 decoding, and there are corresponding driver modules dev_h264e, dev_h265e, dev_h264d, dev_h265d and dev_vp9d.

[0053] In some implementations, the hardware driver layer may include: a channel management module, a register generation module, a data processing module, a task queue management module, a task status management module, a hardware configuration module, and a hardware interrupt response module.

[0054] In some technical solutions, data exchange between user mode and kernel mode includes copying user mode data to kernel mode via copy_from_user and copying kernel mode data to user mode via copy_to_user. However, when the amount of exchanged data is large, this method can cause problems such as high CPU load and slow data transfer efficiency.

[0055] To address at least the above-mentioned issues, a video codec system is proposed in an embodiment of the present disclosure. This system, based on an interactive method of a data description structure, significantly reduces the amount of data copied and improves transmission efficiency. In the video codec system provided in an embodiment of the present disclosure, the application interface layer is configured to receive application commands, parameters, and / or codec data and send them to the protocol parsing layer. The protocol parsing layer is configured to create a user-mode context. If codec data exists, it selects an adapted protocol parsing module to parse the codec data to generate parsed data, and then sends the commands, parameters, and / or parsed data to the kernel adaptation layer. The kernel adaptation layer is configured to select an adapted kernel driver version, open the corresponding character device file to obtain a file descriptor, generate kernel commands and parameters of the corresponding version, and if parsed data exists, generate kernel input data and / or data description structures of the corresponding version, and then send the kernel commands, parameters, kernel input data, and / or data description structures to the kernel interface layer via the file descriptor. The kernel interface layer is configured to create a kernel-mode context, receive kernel commands, parameters, kernel input data, and / or data description structures, and send them to the hardware driver layer. The hardware driver layer is configured to select an adapted hardware driver, generate hardware tasks according to kernel commands, parameters, kernel input data and / or data description structures, and process the hardware tasks using corresponding IP hardware.

[0056] Figure 2B The following is a structural diagram of the data description structure in the embodiment of the present disclosure. For example, the kernel adaptation layer in user mode applies for a large memory space through the system's memory requester interface and obtains the corresponding file descriptor fd; the kernel adaptation layer performs a memory mapping mmap operation on fd to obtain a user mode virtual address, and the kernel adaptation layer writes the data to be transmitted to the kernel to the memory space through the user mode virtual address, and records the offset and data length of the corresponding data in the memory space in the data description structure. The kernel interface layer obtains the data description structure through copy_from_user, performs memory mapping on the fd in the structure to obtain the kernel mode virtual address, and combines the data offset and data length in the description structure to obtain the data transmitted by the user mode through the kernel mode virtual address. Similarly, the data output by the kernel mode is written out through the kernel mode virtual address, and the user mode can also read it through the user mode virtual address.

[0057] Illustratively, the data describes a structure such as Figure 2B As shown, it includes the file descriptor fd corresponding to the memory space, the output data offset, the input data length, the number of register write segments, the offset and length of each write segment, the number of register read segments, and the offset and length of each read segment.

[0058] In some implementations, the hardware driver layer is configured to return processed output data to the kernel interface layer. The kernel interface layer is configured to integrate the output data into the output portion of the data description structure and return it to the kernel adaptation layer. The kernel adaptation layer is configured to parse and process the output portion of the data description structure to obtain processed data and return it to the application interface layer. The application interface layer is configured to return the processed data to the application.

[0059] In some implementations, the video codec system provided by the embodiments of the present disclosure also includes an application, which is configured to send codec commands and parameters through the application channel interface of the application interface layer, obtain a codec channel number, send the codec channel number and input data through the send data interface of the application interface layer, send the codec channel number and wait for output data through the receive data interface of the application interface layer, and after all input data processing is completed, send the codec channel number through the close channel interface of the application interface layer to close the corresponding channel.

[0060] The following uses video encoding as an example to illustrate the application calling method, calling process, and working method of each layer. Figure 3 Shown is a schematic diagram of calling a video encoding application in an embodiment of the present disclosure. Figure 3 As shown, the application interface layer may include a channel application interface, a data sending interface, a data receiving interface, a control command interface, and a channel closing interface.

[0061] The application interface layer's channel request interface is used to receive calls from applications. In some implementations, the application interface layer's channel request interface is configured to, upon receiving the application's commands and parameters, request a user-mode data space based on the commands and parameters for storing user-mode context information for the codec channel, and then send the commands and parameters to the command control module in the protocol parsing layer. The command control module in the protocol parsing layer is configured to send the commands and parameters to the command control module in the kernel adaptation layer. The command control module in the kernel adaptation layer is configured to select an adapted kernel driver version based on the commands and parameters, open the corresponding character device file to obtain a file descriptor, generate kernel commands and parameters for the corresponding version, and send the kernel commands and parameters to the channel request interface in the kernel interface layer via the file descriptor. The kernel interface layer's channel request interface is configured to request a kernel-mode data space based on the received kernel commands and parameters for storing kernel-mode context information for the codec channel, register the data space with the file descriptor, and then send the kernel commands and parameters to the channel management module in the hardware driver layer. The channel management module in the hardware driver layer is configured to apply for the codec channel number according to the kernel command and parameters, and return the codec channel number layer by layer.

[0062] Exemplarily, user-mode context information is used to store codec channel data in the application interface layer, protocol parsing layer, and kernel adaptation layer, including: protocol information, codec channel number, kernel driver version, character device file, file descriptor, and computation cache data.

[0063] In some implementations, kernel state context information is used to store codec channel data in the kernel interface layer and the hardware driver layer, including: protocol information, codec channel number, hardware driver information, IP hardware information, output queue, and computation cache data.

[0064] The send data interface of the application interface layer is used to receive encoded data from the application. In some implementations, the send data interface of the application interface layer is configured to receive the codec channel number and input data of the application and forward them to the protocol parsing module in the protocol parsing layer. The protocol parsing module in the protocol parsing layer is configured to find the corresponding user-mode context based on the codec channel number, select an adapted protocol parsing module based on the protocol information in the user-mode context to parse the input data, and send the parsed data to the driver data generation module in the kernel adaptation layer. The driver data generation module in the kernel adaptation layer is configured to generate kernel input data and a data description structure based on the parsed data, and send the kernel input data and data description structure to the input data interface in the kernel interface layer via a file descriptor in the user-mode context. The input data interface in the kernel interface layer is configured to send the kernel input data and data description structure to the data processing module in the hardware driver layer based on the hardware driver information in the kernel-mode context under the file descriptor. The data processing module in the hardware driver layer is configured to process the kernel input data and data description structure, the register generation module is configured to obtain the input register data based on the processed data and generate a hardware task, and the task status management module is configured to mark the hardware task as a waiting state and send it to the corresponding hardware driver task queue.

[0065] Exemplarily, the protocol parsing layer may call a corresponding encoding module according to the protocol parameters in the user state context, and calculate the bit rate control parameters and reference frame management.

[0066] For example, the hardware driver can be divided into the upper driver half and the lower driver half. The upper driver half is used to obtain register configuration information based on input data, generate hardware tasks, and send them to the corresponding task queue. The lower driver half waits for the IP hardware to be idle, selects a hardware task from the task queue as the target task, and executes the target task using the IP hardware. The lower driver half captures interrupts, performs post-interrupt processing on the target task, and finally sends the result to the output queue in the encoding context.

[0067] Exemplarily, the information in the hardware task may include: register segmentation information, control register data, input register data, output register data, and status flags.

[0068] Exemplarily, the hardware-driven task queue may include: a first-in-first-out based serial single queue and a priority-based serial multiple queue.

[0069] The receiving data interface of the application interface layer is used to receive the receiving coded data call of the application. In some implementations, the receiving data interface of the application interface layer is configured to receive the codec channel number of the application and forward it to the protocol parsing module in the protocol parsing layer.

[0070] The protocol parsing module in the protocol parsing layer is configured to find the corresponding user-mode context based on the codec channel number and send the protocol information in the context to the driver data generation module in the kernel adaptation layer. The driver data generation module in the kernel adaptation layer is configured to generate kernel commands and parameters based on the protocol information and send the kernel commands and parameters to the output data interface in the kernel interface layer through the file descriptor in the user-mode context. The output data interface in the kernel interface layer is configured to select one as the target task from the output queue in the kernel-mode context under the file description based on the kernel commands and parameters and copy the data description structure to the kernel adaptation layer. The kernel adaptation layer processes the encoded data based on the information in the data description structure. If the target task is in the interrupt completion state, the return value is set to correct; otherwise, the return value is set to error. The output data and return value in the target task are returned layer by layer, and the task queue management module releases the data space of the target task.

[0071] The control command interface of the application interface layer is used for some customized operations of the application on the encoding, such as querying the encoding status, special processing of some specific scenarios, error handling, etc.

[0072] The closed channel interface of the application interface layer receives calls from the application; the protocol parsing layer releases the user-state space for storing the encoding context; the kernel interface layer also releases the kernel-state encoding context.

[0073] In some implementations, the hardware driver is configured to select a task from the hardware driver task queue as a target task, the task status management module is configured to mark the target task as an execution state, initialize a timer with a preset time, and use the hardware configuration module to configure the input register data in the target task for the corresponding IP hardware. If the hardware interrupt response module in the hardware driver layer captures a hardware interrupt, the timer is turned off, and the output register of the hardware is read according to the data description structure in the target task. The task status management module marks the target task as an interrupt completion state. If it is an error interrupt, the corresponding IP hardware is reset, and the task queue management module sends the target task to the output queue of the corresponding kernel state context. If the hardware interrupt response module in the hardware driver layer captures a timer timeout, the task status management module marks the target task as a timeout state, turns off the timer, resets the corresponding IP hardware, and the task queue management module sends the target task to the output queue of the corresponding kernel state context.

[0074] Exemplarily, the output queue of the kernel-mode context may be a serialized single queue based on a first-in-first-out principle.

[0075] The following example describes the execution method and process of hardware tasks in the hardware driver task queue. In this example, a task status tag-based method is used to record the state transition of each target task, where the task status includes waiting state, execution state, interrupt completion state, and timeout state. Figure 4A This is a target task state transition flow chart in an embodiment of the present disclosure, including the following steps S41a to S49a.

[0076] Step S41a, generating a hardware task according to the data description structure, marking it as a waiting state, and putting it into the task queue of the corresponding hardware driver, and then proceeding to the next step S411a.

[0077] Step S411a, determine whether the IP hardware is idle, if it is idle, go to step S42a, otherwise go to step S412a.

[0078] Step S412a, enter sleep mode and wait for awakening. If awakened, enter step S42a.

[0079] Step S42a, select a hardware task from the hardware-driven task queue as the target task, mark it as the execution state, and enter the next step S43a.

[0080] Step S43a, initialize the timer of the target task, and proceed to the next step S44a. The timer can be preset according to actual needs. For example, the preset time is 1 second.

[0081] In step S44a, the IP hardware registers are configured according to the data description structure, and the process proceeds to step S45a, wherein the IP hardware can be configured according to the number of register write segments and the register write segment data.

[0082] Step S45a, determining whether the hardware driver captures the hardware interrupt, if so, proceeding to step S461a, otherwise proceeding to step S471a.

[0083] Step S461a, turn off the timer of the target task and go to the next step S462a.

[0084] In step S462a, the hardware registers are read according to the data description structure, and the process proceeds to step S463a, wherein the hardware registers include output registers and status registers.

[0085] Step S463a, set the target task status to interrupt completion, and go to the next step S464a.

[0086] Step S464a, determine whether the hardware interrupt is an error interrupt, if it is, go to step S473a, otherwise go to the next step S48a.

[0087] Step S471a, wait for the timer to time out, set the target state to the timeout state, and enter the next step S472a. Since the driver has not captured the hardware interrupt, if the trigger timer exceeds the timeout, it can be determined that the hardware is stuck.

[0088] Step S472a, turn off the timer of the target task and go to the next step S473a.

[0089] Step S473a: Reset the IP hardware and proceed to step S48a. When the hardware reports an error or freezes, the hardware needs to be reset.

[0090] Step S48a, sending the target task to the output queue of the corresponding kernel state context, and proceeding to the next step S49a.

[0091] Step S49a: The target task is executed and the dormant thread is awakened.

[0092] Among them, step S41a works in the lower half of the hardware driver, and steps S411a to S49a work in the lower half of the hardware driver to jointly complete a hardware task; the two parts are connected through the task queue and can belong to the same thread or two threads.

[0093] Figure 4B A schematic diagram of target task state transition in an embodiment of the present disclosure is shown. Figure 4B Used for Figure 3 The hardware driver layer is refined to show the transfer of task status from the generation to the completion of hardware tasks.

[0094] In some implementations, the codec has only one IP hardware that supports encoding and decoding at the same time. Figure 5A FIG. 1 is a schematic diagram showing the scheduling of hardware tasks for a single IP hardware in an embodiment of the present disclosure. Figure 5AAs shown in the figure, there are four applications, calling h264 encoding, h265 encoding, h264 decoding, and h265 decoding, respectively. The hardware driver instantiates a task queue for the IP hardware. The input data of the four applications, h264e in_task0, h265e in_task0, h264d in_task0, and h265d in_task0, is processed by the hardware driver as hardware tasks h264e task0, h265e task0, h264d task0, and h265d task0, respectively, and sent to the task queue. The hardware driver selects the hardware task from the task queue as the target task and executes it using the IP hardware. The execution results are read and sent to the output queue of the respective kernel contexts, as shown in the figure, h264e out_task0, h265e out_task0, h264d out_task0, and h265d out_task0.

[0095] In some implementations, the codec has multiple IP hardware, each IP hardware is responsible for different functions. For example, Figure 5B FIG. 1 is a schematic diagram showing the scheduling of hardware tasks when multiple IP hardware is used in the embodiment of the present disclosure. Figure 5B As shown, IP hardware 1 supports encoding, and IP hardware 2 supports decoding. For example, there are four applications, each calling h264 encoding, h265 encoding, h264 decoding, and h265 decoding. Based on the number of IP hardware components, the kernel driver instantiates task queue 0 and task queue 1 for IP hardware 1 and IP hardware 2, respectively. The input data h264e in_task0 and h265e in_task0 from Application 1 and Application 2 are processed by the hardware driver to generate encoding hardware tasks h264e task0 and h265e task0, which are then sent to task queue 1. The input data h264d in_task0 and h265d in_task0 from Application 3 and Application 4 are processed by the hardware driver to generate decoding hardware tasks h264d task0 and h265d task0, which are then sent to task queue 2. The hardware driver selects the encoding hardware task from task queue 1 as the target task and executes it using IP hardware 1. The hardware driver selects the decoding hardware task from task queue 2 as the target task and executes it using IP hardware 2. After the execution is completed, the execution results are read and sent to the output queues of their respective kernel contexts.

[0096] The embodiment of the present disclosure also provides a video codec driving method. Figure 6 The flowchart of the video codec driving method is shown. Figure 6 As shown, the video codec driving method provided by the embodiment of the present disclosure includes the following steps S61 to S65.

[0097] In step S61, the application interface layer receives application commands, parameters and / or codec data and sends them to the protocol parsing layer.

[0098] In step S62, the protocol parsing layer creates a user state context. If codec data exists, an adapted protocol parsing module is selected to parse the codec data to generate parsed data, and the command, parameters and / or parsed data are sent to the kernel adaptation layer.

[0099] In step S63, the kernel adaptation layer selects the adapted kernel driver version, opens the corresponding character device file to obtain the file descriptor, generates the corresponding version of kernel commands and parameters, generates the corresponding version of kernel input data and / or data description structure if there is parsed data, and sends the kernel commands, parameters, kernel input data and / or data description structure to the kernel interface layer through the file descriptor.

[0100] In step S64, the kernel interface layer creates a kernel state context, receives kernel commands, parameters, kernel input data and / or data description structures, and sends them to the hardware driver layer.

[0101] In step S65, the hardware driver layer selects an adapted hardware driver, generates a hardware task according to the kernel command, parameters, kernel input data and / or data description structure, and processes the hardware task using the corresponding IP hardware.

[0102] It should be understood that the specific implementation process of the above steps has been described in detail in the aforementioned embodiments and will not be repeated here for the sake of brevity. In addition, the scope of protection of the driving method provided by the embodiment of the present disclosure is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, subtracting, or replacing steps in the prior art based on the principles of the present disclosure are included in the scope of protection of the present disclosure.

[0103] The embodiment of the present disclosure further provides a video coding and decoding device, which includes an application interface layer, a protocol parsing layer, a kernel adaptation layer, a kernel interface layer and a hardware driver layer.

[0104] The application interface layer is configured to receive application commands, parameters and / or codec data and send them to the protocol parsing layer.

[0105] The protocol parsing layer is configured to create a user-mode context, and if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the command, the parameters and / or the parsed data to the kernel adaptation layer.

[0106] The kernel adaptation layer is configured to select an adapted kernel driver version, open the corresponding character device file to obtain a file descriptor, generate a corresponding version of kernel commands and parameters, generate a corresponding version of kernel input data and / or data description structure if the parsed data exists, and send the kernel commands, parameters, kernel input data and / or data description structure to the kernel interface layer through the file descriptor.

[0107] The kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameter, the kernel input data and / or the data description structure, and send them to the hardware driver layer.

[0108] The hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

[0109] It should be understood that the relevant contents of the video encoding and decoding device have been described in detail in the aforementioned embodiments, and for the sake of brevity, they will not be repeated here.

[0110] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.

[0111] Modules / units described as separate components may or may not be physically separate, and components displayed as modules / units may or may not be physical modules, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules / units may be selected based on actual needs to achieve the objectives of the embodiments of the present disclosure. For example, the functional modules / units in the various embodiments of the present disclosure may be integrated into a single processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into a single module / unit.

[0112] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0113] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which implements the kernel-driven method provided by the present disclosure when the computer program is executed by the processor. A person skilled in the art will understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state drive, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0114] An embodiment of the present disclosure also provides an electronic device. Figure 7 The diagram shows the structure of the electronic device 7 in one embodiment of the present disclosure. Figure 7 As shown, in this embodiment, the electronic device 7 includes a memory 71 and a processor 72.

[0115] The memory 71 is used to store computer programs. In some implementations, the memory 71 may include a computer system readable medium in the form of a volatile memory, such as RAM and / or cache memory. The electronic device 7 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 71 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure. The processor 72 is connected to the memory 71 and is used to execute the computer program stored in the memory 71 so that the electronic device 7 executes the kernel driving method provided by the embodiments of the present disclosure.

[0116] The descriptions of the processes or structures corresponding to the above figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0117] In summary, the disclosed embodiments achieve comprehensive adaptation to different video protocol formats, operating system kernel driver versions, and hardware platforms by constructing a unified codec system. This system comprises an application interface layer, a protocol parsing layer, a kernel adaptation layer, a kernel interface layer, and a hardware driver layer. The application interface layer defines the basic operations required during the video encoding and decoding process, including channel application, data transmission, data reception, command control, and channel release. All applications only need to interface with these interfaces. The protocol parsing layer includes an encoding module and a decoding module. The encoding module provides various bit rate control and reference frame management algorithms, while the decoding module supports nearly all video protocols and converts video streams into an internal unified format. The kernel adaptation layer automatically identifies and adapts to different operating systems, kernel driver versions, and hardware platforms, establishing connections with the corresponding hardware drivers. The kernel interface layer provides a unified kernel interface for all hardware drivers and provides task forwarding and scheduling for each hardware driver. The hardware driver layer is the core driver, responsible for task execution and configuring and reading IP hardware registers.

[0118] In the disclosed embodiments, the kernel adaptation layer adapts to hardware drivers for different operating systems, different kernel driver versions, and different hardware platforms. The application layer does not need to be concerned with these differences, reducing development costs and maintenance difficulties. Furthermore, the hardware drivers in the hardware driver layer are also developed using a plug-in approach. The underlying hardware drivers only need to adapt to the kernel interface layer and can be dynamically loaded and unloaded, eliminating the need for the user to be concerned with the specific implementation details of the underlying hardware.

[0119] The disclosed embodiments also propose an interaction method based on a data description structure to reduce data exchange between user mode and kernel mode. By allocating a memory space as an interaction cache, user mode maps user-mode virtual addresses, and kernel mode maps kernel-mode virtual addresses. Between user mode and the kernel, only the data description structure is copied, rather than the entire data. This method significantly reduces the amount of data exchange, reduces the amount of data copied by the CPU, and improves transmission efficiency.

[0120] In the disclosed embodiment, the hardware driver can adopt a processing mechanism based on task state transfer, which can clearly understand the execution process of the target task and judge the execution result of the target task through the status mark. A timer is added to the hardware driver for the hardware execution of the target task to handle the abnormal situation of hardware stuck, thereby ensuring the integrity of the task state. In addition, the hardware task queue determines the number of task queues according to the number of IP hardware, and achieves time-sharing multiplexing of IP hardware through the serial operation of the queue to prevent competition. In addition, the output queue of each codec channel is directly output to the respective file description after the hardware task is completed under the corresponding file description. The user state receives and obtains the output data, which can be directly obtained at the kernel interface layer without calling the corresponding hardware driver query.

[0121] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed herein shall be covered by the claims of this disclosure.

Claims

1. A video encoding and decoding system, characterized in that: include: The user-mode part includes the application interface layer, protocol parsing layer, and kernel adaptation layer; as well as The kernel state part includes the kernel interface layer and the hardware driver layer. The application interface layer is configured to receive application commands, parameters and / or codec data and send them to the protocol parsing layer. The protocol parsing layer is configured to create a user state context, and if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the command, the parameter and / or the parsed data to the kernel adaptation layer. The kernel adaptation layer is configured to select an adapted kernel driver version, open a corresponding character device file to obtain a file descriptor, generate kernel commands and parameters of a corresponding version, generate kernel input data and / or a data description structure of a corresponding version if the parsed data exists, and send the kernel commands, parameters, kernel input data and / or data description structure to the kernel interface layer via the file descriptor. The kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameter, the kernel input data and / or the data description structure, and send the received data to the hardware driver layer. The hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

2. The video encoding and decoding system according to claim 1, wherein: The hardware driver layer is configured to return the processed output data to the kernel interface layer; The kernel interface layer is configured to integrate the output data into the output part of the data description structure and return it to the kernel adaptation layer; The kernel adaptation layer is configured to parse and process the output part of the data description structure to obtain processed data, and return the processed data to the application interface layer; as well as The application interface layer is configured to return the processed data to the application program.

3. The video encoding and decoding system according to claim 1, wherein: The application interface layer includes: a channel application interface, a data sending interface, a data receiving interface, a control command interface and a channel closing interface.

4. The video encoding and decoding system according to claim 1, wherein: The protocol analysis layer includes: a command control module, a protocol analysis module, a video framing module, a syntax analysis module, a syntax error tolerance module, a bit rate control module and a reference frame management module.

5. The video encoding and decoding system according to claim 1, wherein: The kernel adaptation layer includes: a command control module, a drive data generation module, a kernel communication module and a memory management module.

6. The video encoding and decoding system according to claim 1, wherein: The kernel interface layer includes: an application channel interface, an input data interface, an output data interface, a command control interface and a channel closing interface.

7. The video encoding and decoding system according to claim 1, wherein: The hardware driver layer includes: a channel management module, a register generation module, a data processing module, a task queue management module, a task status management module, a hardware configuration module and a hardware interrupt response module.

8. The video encoding and decoding system according to claim 1, wherein: The data description structure includes: a file descriptor corresponding to the memory space, an input data offset and length, the number of register write segments, the offset and length of each write segment, the number of register read segments, and the offset and length of each read segment.

9. The video encoding and decoding system according to claim 8, wherein: The kernel adaptation layer is further configured to: apply for memory space through a system memory requester interface and obtain a file descriptor corresponding to the memory space; perform memory mapping on the file descriptor corresponding to the memory space to obtain a user state virtual address; and writing the data to be transmitted to the kernel into the memory space through the user state virtual address, and recording the offset and data length of the corresponding data in the memory space in the data description structure; The kernel interface layer is further configured to: obtain the data description structure, perform memory mapping on the file descriptor corresponding to the memory space to obtain a kernel state virtual address; and obtain the data transmitted in user state through the kernel state virtual address according to the offset and data length in the data description structure.

10. The video encoding and decoding system according to claim 1, wherein: Also included is the application, which is configured to: Send codec commands and parameters through the application channel interface of the application interface layer to obtain a codec channel number; Send the codec channel number and input data through the data sending interface of the application interface layer; Send the codec channel number through the receiving data interface of the application interface layer and wait for output data; as well as After all input data processing is completed, the codec channel number is sent through the channel closing interface of the application interface layer to close the corresponding channel.

11. The video encoding and decoding system according to claim 10, wherein: The application channel application interface of the application interface layer is configured to, after receiving the command and parameters of the application program, apply for a user-state data space for storing user-state context information of the codec channel according to the command and parameters, and send the command and parameters to the command control module in the protocol parsing layer; The command control module in the protocol parsing layer is configured to send the command and parameters to the command control module in the kernel adaptation layer; The command control module in the kernel adaptation layer is configured to select an adapted kernel driver version according to the command and parameters, open a corresponding character device file to obtain the file descriptor, generate a kernel command and parameters of a corresponding version, and send the kernel command and parameters to the application channel interface in the kernel interface layer through the file descriptor; The channel application interface in the kernel interface layer is configured to apply for a kernel-state data space for storing kernel-state context information of the codec channel according to the received kernel command and parameters, register the space under the file descriptor, and send the kernel command and parameters to the channel management module in the hardware driver layer; The channel management module in the hardware driver layer is configured to apply for the codec channel number according to the kernel command and parameters, and return the codec channel number layer by layer.

12. The video encoding and decoding system according to claim 11, wherein: The user state context information is used to store the codec channel data in the application interface layer, the protocol parsing layer and the kernel adaptation layer, including: protocol information, codec channel number, kernel driver version, character device file, file descriptor and calculation cache data.

13. The video encoding and decoding system according to claim 11, wherein: The kernel state context information is used to store the codec channel data in the kernel interface layer and the hardware driver layer, including: protocol information, codec channel number, hardware driver information, IP hardware information, output queue and calculation cache data.

14. The video encoding and decoding system according to claim 11, wherein: The data transmission interface of the application interface layer is configured to receive the codec channel number and input data of the application program, and forward the data to the protocol parsing module in the protocol parsing layer; The protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, select an adapted protocol parsing module according to the protocol information in the user state context to parse the input data, and send the parsed data to the driver data generation module in the kernel adaptation layer; The driver data generation module in the kernel adaptation layer is configured to generate the kernel input data and the data description structure according to the parsed data, and send the kernel input data and the data description structure to the input data interface in the kernel interface layer through the file descriptor in the user state context; The input data interface in the kernel interface layer is configured to send the kernel input data and the data description structure to the data processing module in the hardware driver layer according to the hardware driver information in the kernel state context under the file descriptor; The data processing module in the hardware driver layer is configured to process the kernel input data and the data description structure, the register generation module is configured to obtain input register data based on the processed data and generate a hardware task, and the task status management module is configured to mark the hardware task as a waiting state and send it to the corresponding hardware driver task queue.

15. The video encoding and decoding system according to claim 14, wherein: The information in the hardware task includes: register segmentation information, control register data, input register data, output register data and status mark.

16. The video encoding and decoding system according to claim 14, wherein: The hardware driver task queue includes: a first-in-first-out serial single queue and a priority-based serial multi-queue.

17. The video encoding and decoding system according to claim 16, wherein: The hardware driver is configured to select a task from the hardware driver task queue as a target task, the task state management module is configured to mark the target task as an execution state, initialize a timer for a preset time, and use the hardware configuration module to configure input register data in the target task for the corresponding IP hardware; If the hardware interrupt response module in the hardware driver layer captures a hardware interrupt, the timer is turned off, the output register of the hardware is read according to the data description structure in the target task, the task status management module marks the target task as an interrupt completion state, and resets the corresponding IP hardware if it is an error interrupt. The task queue management module sends the target task to the output queue of the corresponding kernel state context; If the hardware interrupt response module in the hardware driver layer captures the timer timeout, the task status management module marks the target task as a timeout state, turns off the timer, resets the corresponding IP hardware, and the task queue management module sends the target task to the output queue of the corresponding kernel state context.

18. The video encoding and decoding system according to claim 17, wherein: The output queue of the kernel context is a serialized single queue based on the first-in-first-out principle.

19. The video encoding and decoding system according to claim 12, wherein: The receiving data interface of the application interface layer is configured to receive the codec channel number of the application program and forward it to the protocol parsing module in the protocol parsing layer; The protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, and send the protocol information in the context to the driver data generation module in the kernel adaptation layer; The driver data generation module in the kernel adaptation layer is configured to generate the kernel command and parameters according to the protocol information, and send the kernel command and parameters to the output data interface in the kernel interface layer through the file descriptor in the user state context; The output data interface in the kernel interface layer is configured to select one as a target task from the output queue in the kernel state context under the file description according to the kernel command and parameters, set the return value to be correct if the target task is in the interrupt completion state, otherwise set the return value to be error, return the output data and the return value in the target task layer by layer, and the task queue management module releases the data space of the target task.

20. A video codec driving method, characterized in that: The method comprises: The application interface layer receives application commands, parameters and / or codec data and sends them to the protocol analysis layer; The protocol parsing layer creates a user state context, and if the codec data exists, selects an adapted protocol parsing module to parse the codec data to generate parsed data, and sends the command, the parameters and / or the parsed data to the kernel adaptation layer; The kernel adaptation layer selects an adapted kernel driver version, opens a corresponding character device file to obtain a file descriptor, generates kernel commands and parameters of a corresponding version, generates kernel input data and / or a data description structure of a corresponding version if the parsed data exists, and sends the kernel commands, parameters, kernel input data and / or data description structure to the kernel interface layer via the file descriptor; The kernel interface layer creates a kernel state context, receives the kernel command, the parameter, the kernel input data and / or the data description structure, and sends the received data to the hardware driver layer; and The hardware driver layer selects an adapted hardware driver, generates a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and processes the hardware task using corresponding IP hardware.

21. A video encoding and decoding device, characterized in that: include: The application interface layer is configured to receive application commands, parameters and / or codec data and send them to the protocol parsing layer; The protocol parsing layer is configured to create a user state context, select an adapted protocol parsing module to parse the codec data to generate parsed data if the codec data exists, and send the command, the parameter and / or the parsed data to the kernel adaptation layer; The kernel adaptation layer is configured to select an adapted kernel driver version, open a corresponding character device file to obtain a file descriptor, generate kernel commands and parameters of a corresponding version, generate kernel input data and / or a data description structure of a corresponding version if the parsed data exists, and send the kernel commands, parameters, kernel input data and / or data description structure to the kernel interface layer via the file descriptor; The kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameter, the kernel input data and / or the data description structure, and send the received data to the hardware driver layer; as well as The hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video codec driving method according to claim 20 is implemented.

23. An electronic device, characterized in that: The electronic device comprises: a memory configured to store an executable program; and The processor is configured to call the program so that the electronic device executes the video codec driving method according to claim 20.

Images