Video encoding and decoding system and device, driving method, storage medium and electronic equipment

Abstract

The invention provides a video coding and decoding system, a driving method and device, a storage medium and electronic equipment. The video encoding and decoding system comprises: a user mode part, which comprises an application interface layer, a protocol analysis layer and a kernel adaptation layer; and the kernel mode part comprises a kernel interface layer and a hardware driving layer. Through the unified coding and decoding framework and the driving interface, adaptation of different protocol formats, different kernel driving versions and different hardware versions is realized, the development cost is reduced, and the expandability and maintainability of the system are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video coding and decoding technology, and in particular to a video coding and decoding system and device, a driving method, a storage medium and an electronic device. Background Art

[0002] With the rapid development of video technology, video codec technology plays an increasingly important role in various application scenarios. However, video data in different application scenarios often use different protocol formats (such as H264, H265, VP9, ​​AV1, etc.). At the same time, the hardware platform of the device and the kernel driver version of the operating system also vary greatly. This requires the video codec system to have a high degree of flexibility and compatibility to adapt to the ever-changing market demands and technical standards. Summary of the invention

[0003] The present invention provides a video codec system and device, a driving method, a storage medium and an electronic device, which can achieve adaptation to different protocol formats, different kernel driver versions and different hardware versions through a unified codec framework and driving interface, reduce development costs and improve the scalability and maintainability of the system.

[0004] In the first aspect, the embodiments of the present disclosure provide a video codec system. The video codec system includes: a user state part, including an application interface layer, a protocol parsing layer and a kernel adaptation layer; and a kernel state part, including a kernel interface layer and a hardware driver layer, wherein the application interface layer is configured to receive commands, parameters and / or codec data of an application program, and send them to the protocol parsing layer, wherein the protocol parsing layer is configured to create a user state context, and if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the commands, parameters and / or parsed data to the kernel adaptation layer, wherein the kernel adaptation layer is configured to select an adapted kernel driver version, open the corresponding character device file to obtain a file descriptor, and generate a corresponding version of kernel commands and parameters, if the parsed data exists, generates kernel input data and / or data description structure of the corresponding version, and sends the kernel command, the parameters, the kernel input data and / or the data description structure to the kernel interface layer through the file descriptor, wherein the kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameters, the kernel input data and / or the data description structure, and send them to the hardware driver layer, wherein the hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

[0005] In an implementation of the first aspect, the hardware driver layer is configured to return processed output data to the kernel interface layer; the kernel interface layer is configured to integrate the output data into the output part in the data description structure and return it to the kernel adaptation layer; the kernel adaptation layer is configured to parse and process the output part in the data description structure to obtain processed data, and return it to the application interface layer; and the application interface layer is configured to return the processed data to the application.

[0006] In an implementation manner of the first aspect, the application interface layer includes: a channel application interface, a data sending interface, a data receiving interface, a control command interface, and a channel closing interface.

[0007] In an implementation of the first aspect, the protocol parsing layer includes: a command control module, a protocol parsing module, a video framing module, a syntax parsing module, a syntax error tolerance module, a bit rate control module and a reference frame management module.

[0008] In an implementation of the first aspect, the kernel adaptation layer includes: a command control module, a drive data generation module, a kernel communication module and a memory management module.

[0009] In an implementation of the first aspect, the kernel interface layer includes: an application channel interface, an input data interface, an output data interface, a command control interface, and a closed channel interface.

[0010] In an implementation of the first aspect, the hardware driver layer includes: a channel management module, a register generation module, a data processing module, a task queue management module, a task status management module, a hardware configuration module and a hardware interrupt response module.

[0011] In an implementation of the first aspect, the data description structure includes: a file descriptor corresponding to the memory space, an input data offset and length, the number of register write segments, the offset and length of each write segment, the number of register read segments, and the offset and length of each read segment.

[0012] In an implementation of the first aspect, the kernel adaptation layer is further configured to: apply for memory space through the system's memory requester interface, and obtain the file descriptor corresponding to the memory space; memory-map the file descriptor corresponding to the memory space to obtain a user-state virtual address; and write the data to be transmitted to the kernel into the memory space through the user-state virtual address, and record the offset and data length of the corresponding data in the memory space in the data description structure; the kernel interface layer is further configured to: obtain the data description structure, memory-map the file descriptor corresponding to the memory space to obtain a kernel-state virtual address; and obtain the user-state transmitted data through the kernel-state virtual address according to the offset and data length in the data description structure.

[0013] In an implementation of the first aspect, the video codec system also includes the application, which is configured to: send codec commands and parameters through the application channel interface of the application interface layer to obtain a codec channel number; send the codec channel number and input data through the send data interface of the application interface layer; send the codec channel number through the receive data interface of the application interface layer and wait for output data; and after all input data are processed, send the codec channel number through the close channel interface of the application interface layer to close the corresponding channel.

[0014] In an implementation of the first aspect, the application channel interface of the application interface layer is configured to apply for a user-mode data space for storing user-mode context information of the codec channel according to the command and parameters after receiving the command and parameters of the application program, and send the command and parameters to the command control module in the protocol parsing layer; the command control module in the protocol parsing layer is configured to send the command and parameters to the command control module in the kernel adaptation layer; the command control module in the kernel adaptation layer is configured to select an adapted kernel driver version according to the command and parameters, open the corresponding character device file to obtain the file descriptor, generates a kernel command and parameter of a corresponding version, and sends the kernel command and parameter to the channel application interface in the kernel interface layer through the file descriptor; the channel application interface in the kernel interface layer is configured to apply for a kernel-state data space for storing kernel-state context information of the codec channel according to the received kernel command and parameter, and register it under the file descriptor, and send the kernel command and parameter to the channel management module in the hardware driver layer; the channel management module in the hardware driver layer is configured to apply for the codec channel number according to the kernel command and parameter, and return the codec channel number layer by layer.

[0015] In an implementation of the first aspect, the user-state context information is used to store codec channel data in the application interface layer, the protocol parsing layer, and the kernel adaptation layer, including: protocol information, codec channel number, kernel driver version, character device file, file descriptor, and calculation cache data.

[0016] In an implementation of the first aspect, the kernel state context information is used to store codec channel data in the kernel interface layer and the hardware driver layer, including: protocol information, codec channel number, hardware driver information, IP hardware information, output queue, and calculation cache data.

[0017] In an implementation of the first aspect, the data sending interface of the application interface layer is configured to receive the codec channel number and input data of the application, and forward it to the protocol parsing module in the protocol parsing layer; the protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, select an adapted protocol parsing module according to the protocol information in the user state context to parse the input data, and send the parsed data to the driver data generation module in the kernel adaptation layer; the driver data generation module in the kernel adaptation layer is configured to generate the kernel input data and the data description structure according to the parsed data, and send the parsed data to the driver data generation module in the kernel adaptation layer through the file in the user state context. The descriptor sends the kernel input data and the data description structure to the input data interface in the kernel interface layer; the input data interface in the kernel interface layer is configured to send the kernel input data and the data description structure to the data processing module in the hardware driver layer according to the hardware driver information in the kernel state context under the file descriptor; the data processing module in the hardware driver layer is configured to process the kernel input data and the data description structure, the register generation module is configured to obtain input register data according to the processed data and generate a hardware task, and the task status management module is configured to mark the hardware task as a waiting state and send it to the corresponding hardware driver task queue.

[0018] In an implementation manner of the first aspect, the information in the hardware task includes: register segmentation information, control register data, input register data, output register data, and a status flag.

[0019] In an implementation of the first aspect, the hardware driver task queue includes: a first-in-first-out based serial single queue and a priority based serial multiple queues.

[0020] In an implementation of the first aspect, the hardware driver is configured to select a task from the hardware driver task queue as a target task, the task status management module is configured to mark the target task as an execution state, initialize a timer of a preset time, and use the hardware configuration module to configure the input register data in the target task for the corresponding IP hardware; if the hardware interrupt response module in the hardware driver layer captures a hardware interrupt, the timer is turned off, and the output register of the hardware is read according to the data description structure in the target task, the task status management module marks the target task as an interrupt completion state, and if it is an error interrupt, the corresponding IP hardware is reset, and the task queue management module sends the target task to the output queue of the corresponding kernel state context; if the hardware interrupt response module in the hardware driver layer captures a timer timeout, the task status management module marks the target task as a timeout state, turns off the timer, resets the corresponding IP hardware, and the task queue management module sends the target task to the output queue of the corresponding kernel state context.

[0021] In an implementation of the first aspect, the output queue of the kernel mode context is a serialized single queue based on a first-in-first-out principle.

[0022] In an implementation of the first aspect, the receiving data interface of the application interface layer is configured to receive the codec channel number of the application and forward it to the protocol parsing module in the protocol parsing layer; the protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, and send the protocol information in the context to the driver data generation module in the kernel adaptation layer; the driver data generation module in the kernel adaptation layer is configured to generate the kernel command and parameters according to the protocol information, and send the kernel command and parameters to the output data interface in the kernel interface layer through the file descriptor in the user state context; the output data interface in the kernel interface layer is configured to select one as a target task from the output queue in the kernel state context under the file description according to the kernel command and parameters, if the target task is in an interrupt completion state, set the return value to correct, otherwise set the return value to error, return the output data and the return value in the target task layer by layer, and the task queue management module releases the data space of the target task.

[0023] In a second aspect, the disclosed embodiment provides a video codec driver method. The video codec driver method includes: the application interface layer receives the command, parameter and / or codec data of the application program, and sends it to the protocol parsing layer; the protocol parsing layer creates a user state context, and if the codec data exists, selects an adapted protocol parsing module to parse the codec data to generate parsed data, and sends the command, parameter and / or parsed data to the kernel adaptation layer; the kernel adaptation layer selects an adapted kernel driver version, opens the corresponding character device file to obtain a file descriptor, generates a corresponding version of kernel commands and parameters, and if the parsed data exists, generates a corresponding version of The kernel input data and / or data description structure are received by the kernel interface layer, and the kernel command, the parameters, the kernel input data and / or the data description structure are sent to the kernel interface layer through the file descriptor; the kernel interface layer creates a kernel state context, receives the kernel command, the parameters, the kernel input data and / or the data description structure, and sends them to the hardware driver layer; and the hardware driver layer selects an adapted hardware driver, generates a hardware task according to the kernel command, the parameters, the kernel input data and / or the data description structure, and processes the hardware task using the corresponding IP hardware.

[0024] In a third aspect, the disclosed embodiment provides a video codec device. The video codec device includes: an application interface layer, configured to receive commands, parameters and / or codec data of an application program, and send them to a protocol parsing layer; a protocol parsing layer, configured to create a user state context, if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the commands, parameters and / or parsed data to a kernel adaptation layer; a kernel adaptation layer, configured to select an adapted kernel driver version, open the corresponding character device file to obtain a file descriptor, generate kernel commands and parameters of the corresponding version, and generate the parsed data of the corresponding version if the parsed data exists. The kernel input data and / or data description structure is received by the kernel interface layer, and the kernel command, the parameter, the kernel input data and / or the data description structure are sent to the kernel interface layer through the file descriptor; the kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameter, the kernel input data and / or the data description structure, and send them to the hardware driver layer; and the hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameter, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

[0025] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the video codec driving method provided by the embodiment of the present disclosure is implemented.

[0026] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory configured to store an executable program; and a processor configured to call the program so that the electronic device executes the video codec driving method provided according to an embodiment of the present disclosure.

[0027] The video codec system provided according to the embodiment of the present disclosure includes an application interface layer, a protocol parsing layer, a kernel adaptation layer, a kernel interface layer and a hardware driver layer. The application interface layer provides a concise application interface for upper-layer applications to call; the protocol parsing layer is responsible for parsing video streams of different protocol formats and converting them into a unified internal format; the kernel adaptation layer is responsible for interacting with different hardware platforms and operating system kernel drivers to realize the input and output of data between user state and kernel state; the hardware driver layer adapts to different IP hardware and uses IP hardware to realize hardware acceleration of video codec. In this way, all hardware drivers of different chip versions, different kernel versions, and different hardware platforms can be compatible in one framework, which maximizes the reusability and portability of the driver. The simple application interface does not need to consider the differences between various underlying platforms, greatly shortens the development time of the application program, and speeds up the update and iteration speed of the product.

[0028] In addition, in the disclosed embodiments, the protocol parsing layer integrates a variety of protocol parsing modules to achieve comprehensive support for different video protocol formats, such as H264, H265, VP9, ​​AV1, etc. These parsing modules can automatically identify and parse a variety of mainstream video transmission protocols and convert them into a unified format that the codec system can handle. The encoding module in the protocol parsing layer also supports the configuration and adjustment of multiple encoding parameters to meet the needs of different application scenarios. At the same time, the video codec system also provides an extensible protocol interface, allowing developers to add new protocol parsing modules and encoding modules as needed.

[0029] Furthermore, the video codec driver method provided according to the embodiment of the present disclosure is similar to a plug-in mechanism, allowing developers to write corresponding driver plug-ins according to specific hardware platforms and operating system kernel driver versions. These plug-ins seamlessly connect with the codec system by implementing unified interface specifications. When the hardware platform or operating system kernel needs to be replaced or upgraded, only the corresponding driver plug-in needs to be replaced without modifying the core code of the codec system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the specific embodiments of the present disclosure, the drawings required for describing the specific embodiments are briefly introduced below.

[0031] Figure 1 Shown is a schematic diagram of the system framework of video encoding and decoding in an embodiment of the present disclosure.

[0032] Figure 2A Shown is a schematic diagram of interfaces of various layers of a video encoding and decoding system in an embodiment of the present disclosure.

[0033] Figure 2B Shown is a structural diagram of a data description structure in an embodiment of the present disclosure.

[0034] Figure 3 Shown is a schematic diagram of calling a video encoding application in an embodiment of the present disclosure.

[0035] Figure 4A Shown is a target task state transfer flow chart in an embodiment of the present disclosure.

[0036] Figure 4B Shown is a schematic diagram of the target task status execution in an embodiment of the present disclosure.

[0037] Figure 5A Shown is a schematic diagram of scheduling hardware tasks for single IP hardware in an embodiment of the present disclosure.

[0038] Figure 5B Shown is a schematic diagram of scheduling hardware tasks for multi-IP hardware in an embodiment of the present disclosure.

[0039] Figure 6 A flowchart of a driving method provided by an embodiment of the present disclosure is shown.

[0040] Figure 7 Shown is a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The following is an explanation of the embodiments of the present disclosure by specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0042] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0043] It should be noted that the illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present disclosure. Therefore, the drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0044] Figure 1 FIG. 1 is a schematic diagram of a system framework for video encoding and decoding in an embodiment of the present disclosure. Figure 1 As shown, the system includes a user state part and a kernel state part, which run in the user state and user state of the operating system respectively. The user state part includes an application interface layer 10, a protocol parsing layer 11 and a kernel adaptation layer 12, and the kernel state part includes a kernel interface layer 20 and a hardware driver layer 21. The application interface layer 10 provides a concise API (Application Programming Interface) for application calls. The protocol parsing layer 11 includes an encoding module and a decoding module. The encoding module is mainly responsible for bit rate control and reference frame management, and the decoding module is mainly responsible for parsing the video code stream according to the protocol format and converting it into parsed data in a unified format. The kernel adaptation layer 12 converts the parsed data into adapted kernel input data according to different hardware platforms and kernel versions of the operating system, and interacts with the kernel driver. The kernel interface layer 20 provides the kernel adaptation layer with a kernel driver interface of the current hardware platform and the kernel version of the system. The hardware driver layer 21 is responsible for hardware task scheduling, generating register configuration, configuring registers and hardware result processing.

[0045] Figure 2A FIG. 2 is a schematic diagram of interfaces of various layers of a video encoding and decoding system in an embodiment of the present disclosure. Figure 2A As shown, the application interface layer provides application interfaces, including the channel application interface open_channel, the data sending interface send_input, the data receiving interface wait_output, the control command interface control_cmd and the channel closing interface close_channel.

[0046] The protocol analysis layer includes an encoding module and a decoding module, and is divided according to different protocols. For example, Figure 2A As shown, the encoding module can support h264 and h265 protocols, and the decoding module can support h264, h265 and vp9 protocols.

[0047] In some implementations, the protocol parsing layer may include: a command control module, a protocol parsing module, a video framing module, a syntax parsing module, a syntax error tolerance module, a bit rate control module, and a reference frame management module.

[0048] The kernel adaptation layer is further subdivided for different hardware and different system kernel versions based on the protocol division of the protocol parsing layer, including v1 and v2, which correspond to the hal_v1 and hal_v2 versions of the kernel interface layer respectively.

[0049] In some implementations, the kernel adaptation layer may include: a command control module, a driver data generation module, a kernel communication module, and a memory management module.

[0050] In some implementations, the kernel interface provided by the kernel interface layer includes a channel application interface open_channel, an input data interface send_data, an output data interface wait_data, a control command interface control_cmd, and a channel closing interface close_channel.

[0051] User-mode applications cannot directly access hardware. They must enter kernel mode and call the corresponding driver to read and write to the hardware. Character device files are a special type of device file and a common way for user mode to access kernel-mode drivers. Common hardware drivers create specific character device files, such as / dev / mpp_service. User mode opens the corresponding character device file of the required hardware driver through the open interface, obtains a file descriptor fd (File Descriptor), and establishes a connection with the hardware driver through this fd. Correspondingly, close(fd) closes the connection between fd and the hardware driver. User mode sends fd, command cmd, and data data to the kernel interface layer through the ioctl(fd,cmd,data) interface. In some implementations, cmd can be encapsulated into custom commands for user mode to call.

[0052] The hardware driver layer is the main body that manages and uses IP hardware to perform hardware tasks. This layer is divided into modules according to different protocols. For example, Figure 2A As shown, the IP hardware supports h264 encoding, h265 encoding, h264 decoding, h265 decoding and vp9 decoding, and there are corresponding driver modules dev_h264e, dev_h265e, dev_h264d, dev_h265d and dev_vp9d.

[0053] In some implementations, the hardware driver layer may include: a channel management module, a register generation module, a data processing module, a task queue management module, a task status management module, a hardware configuration module, and a hardware interrupt response module.

[0054] In some technical solutions, the data interaction between user state and kernel state includes: copying the data in user state to kernel state through copy_from_user, and copying the data in kernel state to user state through copy_to_user. However, when the amount of interactive data is large, this method will cause problems such as high CPU load and slow data transmission efficiency.

[0055] At least for the above problems, a video encoding and decoding system is proposed in the embodiment of the present disclosure. The system is based on the interactive mode of the data description structure, which greatly reduces the amount of data copying and improves the transmission efficiency. In the video encoding and decoding system provided by the embodiment of the present disclosure, the application interface layer is configured to receive the command, parameter and / or encoding and decoding data of the application program, and send it to the protocol parsing layer. The protocol parsing layer is configured to create a user state context. If there is encoding and decoding data, the adapted protocol parsing module is selected to parse the encoding and decoding data to generate parsing data, and the command, parameter and / or parsing data are sent to the kernel adaptation layer. The kernel adaptation layer is configured to select the adapted kernel driver version, open the corresponding character device file to obtain the file descriptor, generate the kernel command and parameter of the corresponding version, if there is parsing data, the kernel input data and / or data description structure of the corresponding version are generated, and the kernel command, parameter, kernel input data and / or data description structure are sent to the kernel interface layer through the file descriptor. The kernel interface layer is configured to create a kernel state context, receive kernel commands, parameters, kernel input data and / or data description structures, and send them to the hardware driver layer. The hardware driver layer is configured to select an adapted hardware driver, generate hardware tasks according to kernel commands, parameters, kernel input data and / or data description structures, and process the hardware tasks using corresponding IP hardware.

[0056] Figure 2B The structure schematic diagram of the data description structure in the embodiment of the present disclosure is shown. Exemplarily, the kernel adaptation layer in user mode applies for a large memory space through the system's memory requester interface and obtains the corresponding file descriptor fd; the kernel adaptation layer performs a memory mapping mmap operation on fd to obtain a user mode virtual address, and the kernel adaptation layer writes the data to be transmitted to the kernel to the memory space through the user mode virtual address, and records the offset and data length of the corresponding data in the memory space in the data description structure. The kernel interface layer obtains the data description structure through copy_from_user, performs a memory mapping on the fd in the structure to obtain the kernel mode virtual address, and combines the data offset and data length in the description structure to obtain the data transmitted by the user mode through the kernel mode virtual address. Similarly, the data output by the kernel mode is written out through the kernel mode virtual address, and the user mode can also read through the user mode virtual address.

[0057] Exemplarily, the data describes a structure such as Figure 2B As shown, it includes the file descriptor fd corresponding to the memory space, the output data offset, the input data length, the number of register write segments, the offset and length of each write segment, the number of register read segments, and the offset and length of each read segment.

[0058] In some implementations, the hardware driver layer is configured to return the processed output data to the kernel interface layer. The kernel interface layer is configured to integrate the output data into the output part of the data description structure and return it to the kernel adaptation layer. The kernel adaptation layer is configured to parse and process the output part of the data description structure to obtain processed data and return it to the application interface layer. The application interface layer is configured to return the processed data to the application.

[0059] In some implementations, the video codec system provided by the embodiments of the present disclosure also includes an application, which is configured to send codec commands and parameters through the application channel interface of the application interface layer, obtain a codec channel number, send the codec channel number and input data through the send data interface of the application interface layer, send the codec channel number and wait for output data through the receive data interface of the application interface layer, and after all input data processing is completed, send the codec channel number through the close channel interface of the application interface layer to close the corresponding channel.

[0060] The following will take video encoding as an example to illustrate the calling method, calling process and working method of each layer of the application. Figure 3 The diagram is a call diagram of a video encoding application in an embodiment of the present disclosure. Figure 3 As shown, the application interface layer may include a channel application interface, a data sending interface, a data receiving interface, a control command interface, and a channel closing interface.

[0061] The application channel interface of the application interface layer is used to receive the call of the application program. In some implementations, the application channel interface of the application interface layer is configured to apply for a piece of user-mode data space for storing the user-mode context information of the codec channel according to the command and parameter after receiving the command and parameter of the application program, and send the command and parameter to the command control module in the protocol parsing layer. The command control module in the protocol parsing layer is configured to send the command and parameter to the command control module in the kernel adaptation layer. The command control module in the kernel adaptation layer is configured to select the adapted kernel driver version according to the command and parameter, open the corresponding character device file to obtain the file descriptor, generate the kernel command and parameter of the corresponding version, and send the kernel command and parameter to the application channel interface in the kernel interface layer through the file descriptor. The application channel interface in the kernel interface layer is configured to apply for a piece of kernel-mode data space for storing the kernel-mode context information of the codec channel according to the received kernel command and parameter, and register it under the file descriptor, and send the kernel command and parameter to the channel management module in the hardware driver layer. The channel management module in the hardware driver layer is configured to apply for the codec channel number according to the kernel command and parameters, and return the codec channel number layer by layer.

[0062] Exemplarily, user-mode context information is used to store codec channel data in the application interface layer, protocol parsing layer, and kernel adaptation layer, including: protocol information, codec channel number, kernel driver version, character device file, file descriptor, and computation cache data.

[0063] In some implementations, kernel state context information is used to store codec channel data in the kernel interface layer and the hardware driver layer, including: protocol information, codec channel number, hardware driver information, IP hardware information, output queue, and computation cache data.

[0064] The data transmission interface of the application interface layer is used to receive the coded data of the application program. In some implementations, the data transmission interface of the application interface layer is configured to receive the codec channel number and input data of the application program, and forward it to the protocol analysis module in the protocol analysis layer. The protocol analysis module in the protocol analysis layer is configured to find the corresponding user state context according to the codec channel number, select the adapted protocol analysis module to parse the input data according to the protocol information in the user state context, and send the parsed data to the driver data generation module in the kernel adaptation layer. The driver data generation module in the kernel adaptation layer is configured to generate kernel input data and data description structure according to the parsed data, and send the kernel input data and data description structure to the input data interface in the kernel interface layer through the file descriptor in the user state context. The input data interface in the kernel interface layer is configured to send the kernel input data and data description structure to the data processing module in the hardware driver layer according to the hardware driver information in the kernel state context under the file descriptor. The data processing module in the hardware driver layer is configured to process the kernel input data and the data description structure, the register generation module is configured to obtain the input register data according to the processed data and generate a hardware task, and the task status management module is configured to mark the hardware task as a waiting state and send it to the corresponding hardware driver task queue.

[0065] Exemplarily, the protocol parsing layer may call a corresponding encoding module according to the protocol parameters in the user state context, and calculate the bit rate control parameters and reference frame management.

[0066] Exemplarily, the hardware driver can be divided into the upper half of the driver and the lower half of the driver. The upper half of the driver is used to obtain register configuration information according to input data, generate hardware tasks, and send them to the corresponding task queue. After the lower half of the driver waits for the IP hardware to be idle, it selects a hardware task from the task queue as the target task, and uses the IP hardware to execute the target task. The lower half of the hardware driver captures interrupts, then performs post-interrupt processing on the target task, and finally sends the result to the output queue in the coding context.

[0067] Exemplarily, the information in the hardware task may include: register segmentation information, control register data, input register data, output register data, and status flags.

[0068] Exemplarily, the hardware driver task queue may include: a first-in-first-out based serial single queue and a priority based serial multiple queues.

[0069] The receiving data interface of the application interface layer is used to receive the receiving coded data call of the application. In some implementations, the receiving data interface of the application interface layer is configured to receive the codec channel number of the application and forward it to the protocol parsing module in the protocol parsing layer.

[0070] The protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, and send the protocol information in the context to the driver data generation module in the kernel adaptation layer. The driver data generation module in the kernel adaptation layer is configured to generate kernel commands and parameters according to the protocol information, and send the kernel commands and parameters to the output data interface in the kernel interface layer through the file descriptor in the user state context. The output data interface in the kernel interface layer is configured to select one as the target task from the output queue in the kernel state context under the file description according to the kernel command and parameter, and copy the data description structure to the kernel adaptation layer. The kernel adaptation layer processes the encoded data according to the information in the data description structure. If the target task is in the interrupt completion state, the return value is set to correct, otherwise the return value is set to error, and the output data and return value in the target task are returned layer by layer, and the task queue management module releases the data space of the target task.

[0071] The control command interface of the application interface layer is used for some customized operations of the application on the encoding, such as querying the encoding status, special processing of some specific scenarios, error handling, etc.

[0072] The closed channel interface of the application interface layer receives calls from the application; the protocol parsing layer releases the user-mode space for storing the encoding context; the kernel interface layer also releases the kernel-mode encoding context.

[0073] In some implementations, the hardware driver is configured to select a task from the hardware driver task queue as the target task, the task state management module is configured to mark the target task as an execution state, initialize a timer of a preset time, and use the hardware configuration module to configure the input register data in the target task for the corresponding IP hardware. If the hardware interrupt response module in the hardware driver layer captures the hardware interrupt, the timer is turned off, the output register of the hardware is read according to the data description structure in the target task, the task state management module marks the target task as an interrupt completion state, and if it is an error interrupt, the corresponding IP hardware is reset, and the task queue management module sends the target task to the output queue of the corresponding kernel state context. If the hardware interrupt response module in the hardware driver layer captures the timer timeout, the task state management module marks the target task as a timeout state, turns off the timer, resets the corresponding IP hardware, and the task queue management module sends the target task to the output queue of the corresponding kernel state context.

[0074] Exemplarily, the output queue of the kernel-mode context may be a serialized single queue based on a first-in-first-out principle.

[0075] The following will introduce the execution method and execution process of the hardware tasks in the hardware driver task queue through an example. In this example, a task state tag-based method is used to record the state transition of each target task, where the task state includes the waiting state, execution state, interrupt completion state and timeout state. Figure 4A This is a target task state transfer flow chart in the embodiment of the present disclosure, including the following steps S41a to S49a.

[0076] Step S41a, generating a hardware task according to the data description structure, marking it as a waiting state, putting it into the task queue of the corresponding hardware driver, and entering the next step S411a.

[0077] Step S411a, determine whether the IP hardware is idle, if so, proceed to step S42a, otherwise proceed to step S412a.

[0078] Step S412a, enter sleep mode and wait for wake-up. If awakened, enter step S42a.

[0079] Step S42a, select a hardware task from the task queue of the hardware driver as the target task, mark it as the execution state, and enter the next step S43a.

[0080] Step S43a, initialize the timer of the target task, and proceed to the next step S44a. The timer can be preset according to actual needs, and the preset time is 1 second.

[0081] Step S44a, configure the registers of the IP hardware according to the data description structure, and proceed to the next step S45a, wherein the IP hardware can be configured according to the number of register write segments and the register write segment data.

[0082] Step S45a, determine whether the hardware driver captures the hardware interrupt, if captured, proceed to step S461a, otherwise proceed to step S471a.

[0083] Step S461a, turn off the timer of the target task and go to the next step S462a.

[0084] Step S462a, read the hardware register according to the data description structure, and proceed to the next step S463a. The hardware register includes an output register and a status register.

[0085] Step S463a, set the target task status to interrupt completion, and proceed to the next step S464a.

[0086] Step S464a, determine whether the hardware interrupt is an error interrupt, if so, proceed to step S473a, otherwise proceed to the next step S48a.

[0087] Step S471a, wait for the timer to time out, set the target state to the timeout state, and enter the next step S472a. Since the driver has not captured the hardware interrupt, if the trigger timer exceeds the timeout, it can be determined that the hardware is stuck.

[0088] Step S472a, turn off the timer of the target task and go to the next step S473a.

[0089] Step S473a, reset the IP hardware and go to the next step S48a. When the hardware reports an error or the hardware is stuck, it is necessary to reset the hardware.

[0090] Step S48a, sending the target task to the output queue of the corresponding kernel state context, and proceeding to the next step S49a.

[0091] Step S49a, the target task is executed and the dormant thread is awakened.

[0092] Among them, step S41a works in the lower half of the hardware driver, and steps S411a to S49a work in the lower half of the hardware driver to jointly complete a hardware task; the two parts are connected through the task queue and can belong to the same thread or two threads.

[0093] Figure 4B A schematic diagram of target task state transfer in an embodiment of the present disclosure is shown. Figure 4B Used for Figure 3 The hardware driver layer is refined to show the transfer of task status from the generation to the completion of hardware tasks.

[0094] In some implementations, the codec has only one IP hardware that supports encoding and decoding at the same time. Figure 5A FIG. 2 is a schematic diagram showing the scheduling of hardware tasks in the case of single IP hardware in the embodiment of the present disclosure. Figure 5AAs shown in the figure, there are 4 applications, which call h264 encoding, h265 encoding, h264 decoding and h265 decoding respectively. The hardware driver instantiates a task queue for the IP hardware. The input data of the 4 applications, h264e in_task0, h265e in_task0, h264d in_task0 and h265d in_task0, are processed by the hardware driver to obtain hardware tasks h264e task0, h265e task0, h264d task0 and h265d task0 respectively, and sent to the task queue. The hardware driver selects the hardware task as the target task from the task queue, executes it using the IP hardware, reads the execution results, and sends them to the output queues of their respective kernel contexts, such as h264e out_task0, h265e out_task0, h264d out_task0 and h265d out_task0 in the figure.

[0095] In some implementations, the codec has multiple IP hardware, each IP hardware is responsible for different functions. Figure 5B FIG. 2 is a schematic diagram showing the scheduling of hardware tasks when multiple IP hardware is used in the embodiment of the present disclosure. Figure 5B As shown, IP hardware 1 supports encoding, and IP hardware 2 supports decoding. For example, there are 4 applications, which call h264 encoding, h265 encoding, h264 decoding, and h265 decoding respectively. The kernel driver instantiates task queue 0 and task queue 1 for IP hardware 1 and IP hardware 2 respectively according to the number of IP hardware. The input data h264e in_task0 and h265e in_task0 of application 1 and application 2 are processed by the hardware driver to obtain the encoding hardware tasks h264e task0 and h265e task0, which are sent to task queue 1. The input data h264d in_task0 and h265d in_task0 of application 3 and application 4 are processed by the hardware driver to obtain the decoding hardware tasks h264d task0 and h265d task0, which are sent to task queue 2. The hardware driver selects the encoding hardware task from task queue 1 as the target task and executes it using IP hardware 1; the hardware driver selects the decoding hardware task from task queue 2 as the target task and executes it using IP hardware 2; after the execution is completed, the execution results are read and sent to the output queues of their respective kernel contexts.

[0096] The disclosed embodiment also provides a video codec driving method. Figure 6 The flowchart of the video codec driving method is shown. Figure 6 As shown, the video codec driving method provided by the embodiment of the present disclosure includes the following steps S61 to S65.

[0097] In step S61, the application interface layer receives application commands, parameters and / or encoding and decoding data, and sends them to the protocol parsing layer.

[0098] In step S62, the protocol parsing layer creates a user state context. If there is codec data, an adapted protocol parsing module is selected to parse the codec data to generate parsed data, and the command, parameter and / or parsed data are sent to the kernel adaptation layer.

[0099] In step S63, the kernel adaptation layer selects an adapted kernel driver version, opens the corresponding character device file to obtain a file descriptor, generates a corresponding version of kernel commands and parameters, generates a corresponding version of kernel input data and / or data description structure if there is parsed data, and sends the kernel commands, parameters, kernel input data and / or data description structure to the kernel interface layer through the file descriptor.

[0100] In step S64, the kernel interface layer creates a kernel state context, receives kernel commands, parameters, kernel input data and / or data description structures, and sends them to the hardware driver layer.

[0101] In step S65, the hardware driver layer selects an adapted hardware driver, generates a hardware task according to kernel commands, parameters, kernel input data and / or data description structures, and processes the hardware task using corresponding IP hardware.

[0102] It should be understood that the specific implementation process of the above steps has been described in detail in the aforementioned embodiments, and for the sake of brevity, it will not be repeated here. In addition, the protection scope of the driving method provided by the embodiment of the present disclosure is not limited to the execution order of the steps listed in the embodiment, and all solutions implemented by adding, reducing, or replacing steps in the prior art based on the principles of the present disclosure are included in the protection scope of the present disclosure.

[0103] The embodiment of the present disclosure also provides a video coding and decoding device, which includes an application interface layer, a protocol parsing layer, a kernel adaptation layer, a kernel interface layer and a hardware driver layer.

[0104] The application interface layer is configured to receive commands, parameters and / or encoding and decoding data of the application program and send them to the protocol parsing layer.

[0105] The protocol parsing layer is configured to create a user state context, and if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the command, the parameter and / or the parsed data to the kernel adaptation layer.

[0106] The kernel adaptation layer is configured to select an adapted kernel driver version, open the corresponding character device file to obtain a file descriptor, generate a corresponding version of kernel commands and parameters, generate a corresponding version of kernel input data and / or data description structure if the parsed data exists, and send the kernel command, the parameters, the kernel input data and / or the data description structure to the kernel interface layer through the file descriptor.

[0107] The kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameter, the kernel input data and / or the data description structure, and send them to the hardware driver layer.

[0108] The hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameter, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

[0109] It should be understood that the relevant contents of the video encoding and decoding device have been described in detail in the aforementioned embodiments, and for the sake of brevity, they will not be repeated here.

[0110] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0111] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure. For example, the functional modules / units in the various embodiments of the present disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0112] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0113] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor, the kernel driving method provided by the embodiment of the present disclosure is implemented. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above-mentioned embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above-mentioned storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state hard disk (SSD)), etc.

[0114] The embodiment of the present disclosure also provides an electronic device. Figure 7 The diagram is a schematic diagram of the structure of an electronic device 7 in one embodiment of the present disclosure. Figure 7 As shown, in this embodiment, the electronic device 7 includes a memory 71 and a processor 72 .

[0115] The memory 71 is used to store computer programs. In some implementations, the memory 71 may include a computer system readable medium in the form of a volatile memory, such as a RAM and / or a cache memory. The electronic device 7 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 71 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure. The processor 72 is connected to the memory 71 and is used to execute the computer program stored in the memory 71 so that the electronic device 7 executes the kernel driving method provided by the embodiments of the present disclosure.

[0116] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0117] In summary, the disclosed embodiment realizes comprehensive adaptation to different video protocol formats, operating system kernel driver versions and hardware platforms by constructing a unified encoding and decoding system. The system includes an application interface layer, a protocol analysis layer, a kernel adaptation layer, a kernel interface layer and a hardware driver layer. The application interface layer defines the basic operations required in the video encoding and decoding process, including applying for channels, sending data, receiving data, command control and releasing channels. All applications only need to connect to these interfaces. The protocol analysis layer includes an encoding module and a decoding module. The encoding module provides various bit rate control and reference frame management algorithms. The decoding module supports almost all video protocols and converts the video stream into an internal unified format. The kernel adaptation layer can automatically identify and adapt to different operating systems, kernel driver versions and hardware platforms, and establish a connection with the corresponding hardware driver. The kernel interface layer provides a unified kernel interface for all hardware drivers, and provides task forwarding and scheduling for each hardware driver. The hardware driver layer is the main body of the kernel driver, responsible for the execution of tasks, the configuration and reading of IP hardware registers.

[0118] In the disclosed embodiment, the kernel adaptation layer adapts to hardware drivers of different operating systems, different kernel driver versions and different hardware platforms. The application layer does not need to care about these differences, which reduces development costs and maintenance difficulties. At the same time, the hardware drivers in the hardware driver layer are also developed in a plug-in type. The underlying hardware drivers only need to adapt to the kernel interface layer and can be dynamically loaded and unloaded. The user state does not need to care about the specific implementation details of the underlying hardware.

[0119] In the disclosed embodiment, an interaction method based on a data description structure is also proposed to reduce the data interaction between the user state and the kernel state. By allocating a memory space as an interaction cache, the user state maps the user state virtual address, and the kernel state maps the kernel state virtual address. Between the user state and the kernel, only a data description structure is copied, rather than the entire data. In this way, the amount of data interaction is greatly reduced, the amount of data copied by the CPU is reduced, and the transmission efficiency is improved.

[0120] In the disclosed embodiment, the hardware driver can adopt a processing mechanism based on task state transfer, and through the state mark method, it can not only clearly understand the execution process of the target task, but also judge the execution result of the target task through the state mark. A timer is added to the hardware execution of the target task in the hardware driver to handle the abnormal situation of hardware stuck, thereby ensuring the integrity of the task state. In addition, the hardware task queue determines the number of task queues according to the number of IP hardware, and achieves time-sharing multiplexing of IP hardware through the serial operation of the queue to prevent competition. In addition, the output queue of each encoding and decoding channel is directly output to the respective file description after the hardware task is completed under the corresponding file description, and the user state receives and obtains the output data, which can be directly obtained only at the kernel interface layer, without calling the corresponding hardware driver query.

[0121] The above embodiments are merely illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present disclosure shall still be covered by the claims of the present disclosure.

Claims

1. A video encoding and decoding system, characterized in that: include: The user state part includes the application interface layer, protocol parsing layer and kernel adaptation layer; as well as The kernel state part includes the kernel interface layer and the hardware driver layer. The application interface layer is configured to receive application commands, parameters and / or encoding and decoding data and send them to the protocol analysis layer. The protocol parsing layer is configured to create a user state context, and if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the command, the parameter and / or the parsed data to the kernel adaptation layer, The kernel adaptation layer is configured to select an adapted kernel driver version, open a corresponding character device file to obtain a file descriptor, generate a kernel command and parameter of a corresponding version, generate kernel input data and / or a data description structure of a corresponding version if the parsed data exists, and send the kernel command, the parameter, the kernel input data and / or the data description structure to the kernel interface layer through the file descriptor. The kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameter, the kernel input data and / or the data description structure, and send them to the hardware driver layer. The hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameter, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

2. The video encoding and decoding system according to claim 1, characterized in that: The hardware driver layer is configured to return the processed output data to the kernel interface layer; The kernel interface layer is configured to integrate the output data into the output part of the data description structure and return it to the kernel adaptation layer; The kernel adaptation layer is configured to parse and process the output part in the data description structure to obtain processed data, and return the processed data to the application interface layer; as well as The application interface layer is configured to return the processed data to the application program.

3. The video encoding and decoding system according to claim 1, characterized in that: The application interface layer includes: a channel application interface, a data sending interface, a data receiving interface, a control command interface and a channel closing interface.

4. The video encoding and decoding system according to claim 1, characterized in that: The protocol analysis layer includes: a command control module, a protocol analysis module, a video framing module, a syntax analysis module, a syntax error tolerance module, a bit rate control module and a reference frame management module.

5. The video encoding and decoding system according to claim 1, characterized in that: The kernel adaptation layer includes: a command control module, a drive data generation module, a kernel communication module and a memory management module.

6. The video encoding and decoding system according to claim 1, characterized in that: The kernel interface layer includes: an application channel interface, an input data interface, an output data interface, a command control interface and a closing channel interface.

7. The video encoding and decoding system according to claim 1, characterized in that: The hardware driver layer includes: a channel management module, a register generation module, a data processing module, a task queue management module, a task status management module, a hardware configuration module and a hardware interrupt response module.

8. The video encoding and decoding system according to claim 1, characterized in that: The data description structure includes: a file descriptor corresponding to the memory space, an input data offset and length, the number of register write segments, the offset and length of each write segment, the number of register read segments, and the offset and length of each read segment.

9. The video encoding and decoding system according to claim 8, characterized in that: The kernel adaptation layer is further configured to: apply for memory space through the system's memory requester interface and obtain a file descriptor corresponding to the memory space; perform memory mapping on the file descriptor corresponding to the memory space to obtain a user state virtual address; and writing the data to be transmitted to the kernel into the memory space through the user state virtual address, and recording the offset and data length of the corresponding data in the memory space in the data description structure; The kernel interface layer is further configured to: obtain the data description structure, perform memory mapping on the file descriptor corresponding to the memory space to obtain the kernel state virtual address; and obtain the data transmitted in the user state through the kernel state virtual address according to the offset and data length in the data description structure.

10. The video encoding and decoding system according to claim 1, characterized in that: Also included is the application, which is configured to: Sending codec commands and parameters through the application channel interface of the application interface layer to obtain a codec channel number; Send the codec channel number and input data through the data transmission interface of the application interface layer; Send the codec channel number through the receiving data interface of the application interface layer and wait for output data; as well as After all input data processing is completed, the codec channel number is sent through the channel closing interface of the application interface layer to close the corresponding channel.

11. The video encoding and decoding system according to claim 10, characterized in that: The application channel application interface of the application interface layer is configured to apply for a user-mode data space for storing user-mode context information of the codec channel according to the command and parameters after receiving the command and parameters of the application program, and send the command and parameters to the command control module in the protocol parsing layer; The command control module in the protocol parsing layer is configured to send the command and parameters to the command control module in the kernel adaptation layer; The command control module in the kernel adaptation layer is configured to select an adapted kernel driver version according to the command and parameter, open the corresponding character device file to obtain the file descriptor, generate a kernel command and parameter of a corresponding version, and send the kernel command and parameter to the application channel interface in the kernel interface layer through the file descriptor; The channel application interface in the kernel interface layer is configured to apply for a kernel state data space for storing kernel state context information of the codec channel according to the received kernel command and parameters, register the space under the file descriptor, and send the kernel command and parameters to the channel management module in the hardware driver layer; The channel management module in the hardware driver layer is configured to apply for the codec channel number according to the kernel command and parameters, and return the codec channel number layer by layer.

12. The video encoding and decoding system according to claim 11, characterized in that: The user state context information is used to store the codec channel data in the application interface layer, the protocol parsing layer and the kernel adaptation layer, including: protocol information, codec channel number, kernel driver version, character device file, file descriptor and calculation cache data.

13. The video encoding and decoding system according to claim 11, characterized in that: The kernel state context information is used to store the codec channel data in the kernel interface layer and the hardware driver layer, including: protocol information, codec channel number, hardware driver information, IP hardware information, output queue and calculation cache data.

14. The video encoding and decoding system according to claim 11, characterized in that: The data transmission interface of the application interface layer is configured to receive the codec channel number and input data of the application program, and forward them to the protocol parsing module in the protocol parsing layer; The protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, select an adapted protocol parsing module according to the protocol information in the user state context to parse the input data, and send the parsed data to the driver data generation module in the kernel adaptation layer; The driver data generation module in the kernel adaptation layer is configured to generate the kernel input data and the data description structure according to the parsed data, and send the kernel input data and the data description structure to the input data interface in the kernel interface layer through the file descriptor in the user state context; The input data interface in the kernel interface layer is configured to send the kernel input data and the data description structure to the data processing module in the hardware driver layer according to the hardware driver information in the kernel state context under the file descriptor; The data processing module in the hardware driver layer is configured to process the kernel input data and the data description structure, the register generation module is configured to obtain input register data based on the processed data and generate a hardware task, and the task status management module is configured to mark the hardware task as a waiting state and send it to the corresponding hardware driver task queue.

15. The video encoding and decoding system according to claim 14, characterized in that: The information in the hardware task includes: register segmentation information, control register data, input register data, output register data and status mark.

16. The video encoding and decoding system according to claim 14, characterized in that: The hardware driver task queue includes: a first-in-first-out based serial single queue and a priority based serial multi-queue.

17. The video encoding and decoding system according to claim 16, characterized in that: The hardware driver is configured to select a task from the hardware driver task queue as a target task, the task state management module is configured to mark the target task as an execution state, initialize a timer of a preset time, and use the hardware configuration module to configure input register data in the target task for the corresponding IP hardware; If the hardware interrupt response module in the hardware driver layer captures a hardware interrupt, the timer is turned off, the output register of the hardware is read according to the data description structure in the target task, the task status management module marks the target task as an interrupt completion state, and if it is an error interrupt, the corresponding IP hardware is reset, and the task queue management module sends the target task to the output queue of the corresponding kernel state context; If the hardware interrupt response module in the hardware driver layer captures the timer timeout, the task status management module marks the target task as a timeout state, turns off the timer, resets the corresponding IP hardware, and the task queue management module sends the target task to the output queue of the corresponding kernel state context.

18. The video encoding and decoding system according to claim 17, characterized in that: The output queue of the kernel state context is a serialized single queue based on the first-in-first-out principle.

19. The video encoding and decoding system according to claim 12, characterized in that: The receiving data interface of the application interface layer is configured to receive the codec channel number of the application program and forward it to the protocol parsing module in the protocol parsing layer; The protocol parsing module in the protocol parsing layer is configured to find the corresponding user state context according to the codec channel number, and send the protocol information in the context to the driver data generation module in the kernel adaptation layer; The driver data generation module in the kernel adaptation layer is configured to generate the kernel command and parameters according to the protocol information, and send the kernel command and parameters to the output data interface in the kernel interface layer through the file descriptor in the user state context; The output data interface in the kernel interface layer is configured to select one as a target task from the output queue in the kernel state context under the file description according to the kernel command and parameters, set the return value to correct if the target task is in an interrupt completion state, otherwise set the return value to error, return the output data and the return value in the target task layer by layer, and the task queue management module releases the data space of the target task.

20. A video codec driving method, characterized in that: The method comprises: The application interface layer receives application commands, parameters and / or encoding and decoding data and sends them to the protocol analysis layer; The protocol parsing layer creates a user state context, and if the codec data exists, selects an adapted protocol parsing module to parse the codec data to generate parsed data, and sends the command, the parameter and / or the parsed data to the kernel adaptation layer; The kernel adaptation layer selects an adapted kernel driver version, opens a corresponding character device file to obtain a file descriptor, generates a kernel command and parameter of a corresponding version, generates kernel input data and / or a data description structure of a corresponding version if the parsed data exists, and sends the kernel command, the parameter, the kernel input data and / or the data description structure to the kernel interface layer through the file descriptor; The kernel interface layer creates a kernel state context, receives the kernel command, the parameter, the kernel input data and / or the data description structure, and sends them to the hardware driver layer; and The hardware driver layer selects an adapted hardware driver, generates a hardware task according to the kernel command, the parameter, the kernel input data and / or the data description structure, and processes the hardware task using corresponding IP hardware.

21. A video encoding and decoding device, characterized in that: include: The application interface layer is configured to receive application commands, parameters and / or codec data and send them to the protocol parsing layer; The protocol parsing layer is configured to create a user state context, if the codec data exists, select an adapted protocol parsing module to parse the codec data to generate parsed data, and send the command, the parameter and / or the parsed data to the kernel adaptation layer; The kernel adaptation layer is configured to select an adapted kernel driver version, open a corresponding character device file to obtain a file descriptor, generate a kernel command and parameter of a corresponding version, generate kernel input data and / or a data description structure of a corresponding version if the parsed data exists, and send the kernel command, the parameter, the kernel input data and / or the data description structure to the kernel interface layer through the file descriptor; The kernel interface layer is configured to create a kernel state context, receive the kernel command, the parameter, the kernel input data and / or the data description structure, and send them to the hardware driver layer; as well as The hardware driver layer is configured to select an adapted hardware driver, generate a hardware task according to the kernel command, the parameter, the kernel input data and / or the data description structure, and process the hardware task using corresponding IP hardware.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video codec driving method according to claim 20 is implemented.

23. An electronic device, characterized in that: The electronic device comprises: a memory configured to store an executable program; and The processor is configured to call the program so that the electronic device executes the video codec driving method according to claim 20.

Images