A system for video processing

Abstract

The present disclosure provides a system for video processing, which relates to the technical field of video processing. The system architecture comprises a computing power base, a driving layer, a service layer and an application layer arranged in sequence. The hardware of the computing power base comprises a plurality of video processing cards, each of which comprises a plurality of parallel video processing chips, each of which comprises a RISC-V computing core and a video processing unit. The application layer is used to initiate a video processing task. The video processing service engine in the service layer distributes the video processing task to the target video processing chip according to the task requirement of the video processing task and the load of the RISC-V computing core in each video processing card. The driving layer is used to standardize the preprocessing of the input video corresponding to the video processing task, and push the processed video to the target video processing chip through the PCIe switch. The RISC-V computing core and the video processing unit of the video processing chip cooperatively process the video. According to the embodiment of the present disclosure, the video processing efficiency can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and more particularly to a system for video processing. Background Technology

[0002] With the rapid development of video surveillance and video networking services, the popularization of ultra-high-definition video (such as 4K / 8K) and the expansion of intelligent application scenarios, the amount of video data has exploded, bringing unprecedented challenges to video transmission and storage systems.

[0003] Traditional CPU computing methods are inefficient, while GPUs, although possessing certain processing capabilities, suffer from high costs, low resource utilization, and limited flexibility, making them unsuitable for large-scale video services. Traditional ASIC-based VPUs (Video Processing Units), despite their performance advantages, are hampered by high costs and customization limitations, making them unable to meet the diverse and flexible needs of cloud computing scenarios.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] Currently, there is a lack of VPU products based on the RISC-V (Reduced Instruction Set Computer V) architecture. This disclosure provides a system for video processing based on the open features and modular design of the RISC-V architecture. At the hardware level, it adopts software-defined VPU architecture (SD-VPU) technology, integrating a high-performance computing core (RISC-V) and a dedicated ASIC acceleration unit, which significantly improves the programmability of the VPU and effectively reduces the communication frequency with the host, thereby improving efficiency.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, a system for video processing is provided, comprising a computing power base, a driver layer, a service layer and an application layer arranged sequentially.

[0008] The hardware of the computing power base includes multiple video processing cards, each video processing card includes multiple parallel video processing chips, and each video processing chip includes a RISC-V computing core and a video processing unit; all the parallel video processing chips are connected to a PCIe switch.

[0009] The application layer is used to initiate video processing tasks;

[0010] In the service layer, the video processing service engine allocates video processing tasks to the target video processing chip based on the task requirements of the video processing task and the load of the RISC-V computing core in each video processing card.

[0011] The driver layer is used to standardize and preprocess the input video corresponding to the video processing task, and push the processed video to the target video processing chip through the PCIe switch. The RISC-V computing core and video processing unit of the video processing chip work together to process the video.

[0012] In one embodiment of this disclosure, each video processing chip further includes an AI computing unit, which uses algorithms to analyze video content and achieve image quality enhancement, intelligent adjustment of encoding parameters, and / or scene recognition.

[0013] The processed video is pushed to the RISC-V computing core of the target video processing chip via a PCIe switch. The RISC-V computing core calls on the video processing unit and the AI ​​computing unit to process the video collaboratively according to the task requirements of the video processing task.

[0014] In one embodiment of this disclosure, the video processing service engine in the service layer allocates video processing tasks to multiple target video processing chips based on the task requirements of the video processing tasks and the load of the RISC-V computing cores in each video processing card. The multiple target processing chips are used to process multiple video streams simultaneously or different parts of a video simultaneously.

[0015] In one embodiment of this disclosure, multiple target processing chips belong to the same video processing card, or multiple target processing chips belong to multiple video processing cards.

[0016] In one embodiment of this disclosure, the video processing service engine in the service layer allocates the video processing task to one or more target video processing chips based on the task requirements and hardware resource status information of the video processing task. The hardware resource status information includes the load and hardware temperature of the RISC-V computing core in each video processing card.

[0017] In one embodiment of this disclosure, the application layer initiates multiple video processing tasks; the video processing service engine of the service layer is also used to adjust the execution order of the multiple video processing tasks by combining task priority and hardware resource status information.

[0018] In one embodiment of this disclosure, the driver layer also collects hardware resource status information in real time and feeds the hardware resource status information back to the video processing service engine of the service layer.

[0019] In one embodiment of this disclosure, each video processing card is provided with a separate baseboard management controller, which is used to monitor the hardware status of the transcoding card in real time; the driver layer is provided with a status monitoring interface, through which the driver layer obtains hardware resource status information from each baseboard management controller.

[0020] In one embodiment of this disclosure, the video processing task includes one or more of the following tasks:

[0021] Video transcoding, video compression, video monitoring, hyperparameter adaptive optimization, stitching, scaling, and AI bitrate control.

[0022] In one embodiment of this disclosure, the video processing service engine of the service layer also dynamically adjusts the transcoding parameters of the input video corresponding to the video processing task in the driver layer by combining task load and hardware resource status information.

[0023] The video processing system provided in this disclosure adopts a software-defined VPU architecture (SD-VPU) at the hardware level. Each video processing card includes multiple parallel video processing chips, and each video processing chip integrates a RISC-V computing core and a video processing unit, significantly improving the programmability of the VPU. Multiple parallel video processing chips can process in parallel, improving efficiency. In the service layer, the video processing service engine allocates video processing tasks to target video processing chips according to the task requirements of the video processing tasks and the load of the RISC-V computing cores in each video processing card. The RISC-V computing cores and video processing units of the video processing chips process the video collaboratively. Through unified allocation and scheduling of tasks, VPU cluster resource optimization and performance improvement are achieved.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0026] Obviously, the accompanying drawings described below are merely some embodiments of this disclosure. Those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0027] Figure 1 This diagram illustrates a system schematic for video processing according to an embodiment of the present disclosure;

[0028] Figure 2 A schematic diagram of the architecture of a video processing card according to an embodiment of the present disclosure is shown;

[0029] Figure 3 This diagram illustrates a pure ASIC architecture according to an embodiment of the present disclosure;

[0030] Figure 4 This diagram illustrates a control core + ASIC dedicated unit architecture according to an embodiment of the present disclosure.

[0031] Figure 5 This diagram illustrates an architecture of a high-performance computing core + ASIC dedicated unit in an embodiment of the present disclosure.

[0032] Figure 6 This illustration shows a schematic diagram of another system for video processing according to an embodiment of the present disclosure;

[0033] Figure 7 A schematic diagram of the architecture of another video processing card in an embodiment of this disclosure is shown;

[0034] Figure 8 This diagram illustrates the high bitrate video transcoding effect in an embodiment of the present disclosure.

[0035] Figure 9 This diagram illustrates the low bitrate video transcoding effect in an embodiment of the present disclosure.

[0036] Figure 10a and 10b This diagram illustrates the low bitrate video transcoding effect in an embodiment of the present disclosure. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0038] To facilitate understanding, the relevant concepts involved in this disclosure are explained below:

[0039] RISC-V (Reduced Instruction Set Computer V) is an open-source instruction set architecture developed by the University of California, Berkeley. It adopts the principle of reduced instruction set computing, and is designed to be simple, efficient and modular. It supports multiple data widths (such as 32-bit, 64-bit and 128-bit), does not require instruction set licensing, and has low cost and low power consumption characteristics.

[0040] VPU (Video Processing Unit): A video processing unit is a brand-new core engine of a video processing platform. It has hardware decoding capabilities and the ability to reduce CPU load, which can reduce server load and network bandwidth consumption, thus differentiating it from traditional GPUs (Graphics Processing Units).

[0041] H.264 is a highly compressed digital video codec standard proposed by the Joint Video Team (JVT), a joint effort of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG). This standard is commonly referred to as H.264 / AVC (or AVC / H.264, H.264 / MPEG-4 AVC, or MPEG-4 / H.264 AVC), explicitly indicating its two developers.

[0042] H.265: High Efficiency Video Coding (HEVC), also known as H.265 and MPEG-H part 2, is a video compression standard and one of several potential successors to the widely used AVC (H.264 or MPEG-4 Part 10). Compared to AVC, HEVC offers approximately twice the data compression ratio at the same video quality level, or significantly improves video quality at the same bitrate. It supports resolutions up to 8192×4320, including 8K UHD.

[0043] With the rapid development of cloud computing and video applications, the demand for cloud storage and computing resources in video encoding and decoding technologies continues to rise. On the one hand, the transmission of high-definition video places a sharp increase in the demand for network bandwidth, especially in multi-channel concurrency and real-time monitoring scenarios, where traditional solutions struggle to meet the requirements for efficient transmission. On the other hand, the long-term storage of massive amounts of video data leads to a surge in storage pressure, and the bottlenecks of existing storage architectures in terms of capacity, cost, and efficiency are gradually becoming apparent. Furthermore, the heterogeneity of encoding formats in different business scenarios (such as H.264, H.265, AV1, etc.) further increases the complexity of the system and processing costs. Meanwhile, as market competition intensifies, customers are placing higher demands on video storage services, expecting not only storage capacity to quickly adapt to data growth but also services with higher cost-effectiveness and stronger video processing capabilities. This presents a dual challenge to cost control and service quality in the video networking business, and also raises higher expectations for technological innovation in underlying hardware and system architecture.

[0044] Traditional CPU computing is inefficient, while GPUs, though possessing some processing power, suffer from high costs, low resource utilization, and limited flexibility, making them unsuitable for large-scale video services. Against this backdrop, the VPU (Video Processing Unit) emerged. As a dedicated processor for accelerating image and video processing, it features hardware-based encoding / decoding, image stitching, and post-processing acceleration, effectively reducing the burden on server CPUs and lowering network bandwidth consumption, thereby significantly reducing the number of racks and operating costs in data centers.

[0045] Furthermore, raw video data contains a large amount of redundant information. While video transcoding and compression technologies can remove redundant data to reduce video size, they may still contain data with weak scene relevance and low user interest. With the continuous development of AI technology, AI-based video analysis can identify key details in the footage, achieving deeper compression and redundancy removal. However, using different hardware accelerator cards to handle video transcoding and compression and AI recognition tasks separately will undoubtedly increase the cost burden.

[0046] Driven by market demand, ASIC-based VPU products have emerged, but RISC-V-based VPU products are still lacking. While traditional ASIC VPUs offer certain performance advantages, their high cost and customization limitations make them unsuitable for the diverse and flexible needs of cloud computing scenarios. Developing video transcoding solutions that adapt to cloud computing scenarios, based on the open nature and modular design of the RISC-V architecture, represents a breakthrough.

[0047] Against this backdrop, this disclosure proposes a video processing system for high-concurrency video transcoding applications such as video cloud and security. At the hardware level, it employs a software-defined VPU architecture (SD-VPU) that integrates a high-performance computing core (RISC-V) and a dedicated ASIC acceleration unit, significantly enhancing the VPU's programmability. The TeleVPU supports flexible super-resolution virtualization and is equipped with larger capacity video memory, capable of simultaneously caching multiple AI models and a large number of video files, effectively reducing the frequency of communication with the host and thus improving efficiency. At the software level, it provides full-stack driver services, supporting key functions including video transcoding, video compression, video monitoring, hyperparameter adaptive optimization, stitching, scaling, AI bitrate control, and multi-card, multi-core scheduling, providing users with an innovative and cost-effective video processing solution.

[0048] The deficiencies of the above solutions and the proposed solutions are the result of the inventors' practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventors' contributions to this disclosure.

[0049] The following detailed description of this exemplary implementation method is provided in conjunction with the accompanying drawings and embodiments.

[0050] Figure 1 This disclosure illustrates a system for video processing, such as... Figure 1 As shown in the embodiments of this disclosure, the system for video processing includes a computing power base, a driver layer, a service layer, and an application layer arranged sequentially.

[0051] In some embodiments, the hardware of the computing power base includes multiple video processing cards (TeleVPU cards), each video processing card including multiple video processing chips connected in parallel. For example... Figure 2 As shown, each video processing chip includes a RISC-V computing core and a video processing unit (NPU), and multiple parallel video processing chips are connected to a PCIe switch. In some embodiments, the video processing unit is an ASIC-specific acceleration unit.

[0052] In some embodiments, the application layer initiates video processing tasks; the video processing service engine in the service layer allocates video processing tasks to target video processing chips based on the task requirements and the load of the RISC-V computing cores in each video processing card; the driver layer performs standardized preprocessing on the input video corresponding to the video processing task and pushes the processed video to the target video processing chip through a PCIe switch, where the RISC-V computing cores and video processing units of the video processing chip collaboratively process the video. In some embodiments, the video processing task includes one or more of the following tasks: video transcoding, video compression, video monitoring, hyperparameter adaptive optimization, stitching, scaling, and AI bitrate control.

[0053] In some embodiments, each video processing chip further includes an AI computing unit, which uses algorithms to analyze video content and achieve image quality enhancement, intelligent adjustment of encoding parameters, and / or scene recognition. The processed video is pushed to the RISC-V computing core of the target video processing chip through a PCIe switch. The RISC-V computing core calls the video processing unit and the AI ​​computing unit to process the video collaboratively according to the task requirements of the video processing task.

[0054] In some embodiments, the video processing card (TeleVPU card) hardware can consist of five high-performance video processing chips connected in parallel, such as... Figure 2 As shown, each chip has a powerful video processing unit (video processing accelerator) and a quad-core RISC-V processor. The five computing chips exchange data with the server host through the PCIe port. Intensive calculations such as video data decoding, encoding, and post-processing are all performed internally by the chips, while the host only needs to perform light-load calculations such as scheduling.

[0055] In some embodiments, the video processing card (TeleVPU card) has a separate board-level BMC management system to adapt to integration into the server cluster operation and maintenance system. The five acceleration chips can work independently, and the normal operation of other chips is not affected when one chip experiences video task blocking or abnormality. At the same time, under the control of the VPU card service software, the transcoding tasks and video streams can be switched in real time between chips, ensuring the disaster recovery and robustness of massive transcoding tasks.

[0056] In some embodiments, video data is accessed at high speed via a PCIe Switch to a video processing card (TeleVPU card). The PCIe Switch acts as a data exchange hub, which can distribute data to each RISC-V computing core to achieve parallel processing and improve efficiency.

[0057] In some embodiments, the video processing service engine in the service layer allocates video processing tasks to multiple target video processing chips based on the task requirements and the load of the RISC-V computing cores in each video processing card. These multiple target processing chips are used to process multiple video streams simultaneously or different parts of a single video. Each RISC-V computing core, acting as a core control unit, leverages its flexible and efficient architecture to schedule its associated video processing unit (NPU) and AI computing unit. The RISC-V computing core allocates video transcoding tasks to the NPU according to task requirements and can simultaneously call upon the AI ​​unit for auxiliary processing. The NPU performs hardware acceleration for video encoding and decoding, supporting multiple format conversions such as H.264 and H.265, and quickly processes video encoding compression or decompression, ensuring transcoding speed and compatibility. The AI ​​computing unit uses algorithms to analyze video content, achieving functions such as image quality enhancement, intelligent adjustment of encoding parameters, and scene recognition, improving transcoding quality and adaptability to meet diverse needs. Multiple RISC-V computing cores and their associated units work in parallel, simultaneously processing multiple video streams or different parts of high-resolution videos, significantly improving overall transcoding performance and adapting to high-load scenarios.

[0058] Traditional VPUs mainly fall into two categories, such as... Figure 3 The example shown has a pure ASIC dedicated unit, or as shown in the example. Figure 4 The traditional VPU, featuring a control core and dedicated ASIC unit, lacks general-purpose computing power and has poor programmability. With the emergence of AI-driven video compression technology, applications are increasingly demanding higher programmability and caching capabilities from VPUs, making it difficult for traditional VPUs to meet these emerging needs. The RISC-V instruction set video processing card (TeleVPU card) proposed in this disclosure adopts a software-defined VPU architecture (SD-VPU) technology approach, integrating a high-performance computing core (RISC-V) and dedicated ASIC unit. This results in a more powerful VPU with stronger programmability, support for more flexible super-resolution virtualization, and larger VPU-side video memory, supporting simultaneous caching of multiple AI models and a large number of video files, avoiding frequent communication with the host. Figure 5 As shown.

[0059] The software for the video processing card (TeleVPU card) is divided into host-side and VPU-side. The host-side software runs on the server host, integrating the video computing resources of up to eight TeleVPU cards installed on a single server. This is mapped to an easy-to-use software interface that can be directly accessed on the host, providing overall task scheduling for multiple cards, video stream allocation, hardware status monitoring, and real-time load statistics services. The VPU-side software runs within the card, executing specific computing tasks. The host-side software provides a universal FFmpeg interface, allowing seamless integration into existing software transcoding systems. It also provides backend service software at the RTSP and RTMP streaming protocol levels, supporting direct management of video stream tasks from outside the server via an HTTP command interface. This allows for the rapid integration of the multi-card server as a single product into monitoring and live streaming systems, eliminating the need to manage internal hardware resource allocation and load balancing. The overall RISC-V software architecture is as follows: Figure 6 As shown.

[0060] The underlying hardware layer utilizes the TeleVPU video transcoding computing power of the SD-VPU architecture as its foundation, enabling efficient execution of key functions such as video transcoding, compression, and splicing. This provides high-density, low-cost, and customizable video transcoding and compression capabilities for video processing scenarios such as the video network. The driver layer constructs a high-level encapsulation SDK, OpenVPU, covering key capabilities such as video transcoding, compression, splicing, AI bitrate control, and hyperparameter adjustment, shielding application complexity and facilitating agile user development. The service layer leverages the VPU Server to empower upper-layer applications, handling single-card multi-core, multi-level multi-card, distributed server, and multi-task scheduling. Through unified task allocation and scheduling, it optimizes VPU cluster resources and improves performance. The application layer handles key user tasks such as video transcoding, video compression, video splicing, and video merging.

[0061] In some embodiments, the application layer initiates a video processing task, such as a video transcoding request, and calls the video transcoding function of the VPUServer engine in the service layer.

[0062] In some embodiments, the service layer is responsible for task parsing and resource scheduling, including single-card multi-chip management, multi-device multi-card management, and multi-task scheduling.

[0063] In some embodiments, the application layer initiates video processing tasks; the video processing service engine in the service layer allocates the video processing tasks to one or more target video processing chips based on the task requirements of the video processing tasks and the load of the RISC-V computing cores in each video processing card. In some embodiments, multiple target processing chips belong to the same video processing card, or multiple target processing chips belong to multiple video processing cards.

[0064] Multiple target processing chips belong to the same video processing card, which is the single-card multi-chip management mentioned above. If a task is processed within a single card, the VPU Server engine allocates the transcoding task to the appropriate computing core based on the transcoding task requirements and the load of each core within the single card.

[0065] Multiple target processing chips belong to multiple video processing cards, i.e., multi-machine multi-card management (if cross-card access is required). If the task load is high or the resources of a single card are insufficient, the multi-machine multi-card management scans the global resource pool to determine the distribution of multiple cards participating in transcoding, and uses distributed services to access resources across cards.

[0066] In some embodiments, the video processing service engine in the service layer allocates video processing tasks to one or more target video processing chips based on the task requirements and hardware resource status information of the video processing tasks. The hardware resource status information includes the load and hardware temperature of the RISC-V computing cores in each video processing card.

[0067] In some embodiments, the application layer initiates multiple video processing tasks; the video processing service engine in the service layer is also used to adjust the execution order of multiple video processing tasks by combining task priority and hardware resource status information.

[0068] In some embodiments, the driver layer also collects hardware resource status information in real time and feeds the hardware resource status information back to the video processing service engine of the service layer.

[0069] In some embodiments, each video processing card has a separate baseboard management controller, which is used to monitor the hardware status of the transcoding card in real time; the driver layer has a status monitoring interface, through which the driver layer obtains hardware resource status information from each baseboard management controller, such as... Figure 7 As shown. The BMC (Baseboard Management Controller) monitors the hardware status of the transcoding card (temperature, power consumption, etc.) in real time to ensure stable operation of the equipment and supports remote management, facilitating maintenance and troubleshooting.

[0070] In some embodiments, the video processing service engine in the service layer also dynamically adjusts the transcoding parameters of the input video corresponding to the video processing task during preprocessing in the driver layer, based on task load and hardware resource status information. The service layer calls the OpenVPUSDK video transcoding interface, and after receiving the instructions, the driver layer performs standardized preprocessing (such as format conversion and decoding) on ​​the input video based on the FFmpeg video processing library. Hyperparameter adaptive optimization analysis of task load and hardware status is used to dynamically adjust transcoding parameters (such as resolution and frame rate) to improve efficiency. AI bitrate control is invoked to optimize the encoding process, reducing bitrate waste while ensuring image quality.

[0071] In some embodiments, the driver layer pushes the processed video data to the RISC-V VPU computing core and supporting units (such as the NPU) for transcoding. The OpenVPU SDK's status monitoring interface collects hardware status (such as computing power usage and temperature) in real time and feeds it back to the VPU Server engine in the service layer. Based on the monitoring feedback, the VPU Server engine dynamically adjusts resource allocation through multi-task scheduling. For example, if a core overheats, some tasks will be migrated to other idle cores.

[0072] After transcoding is complete, the driver layer integrates the transcoded data using the FFmpeg library, and the service layer returns the result to the application layer, completing a full resource management and invocation process.

[0073] The architecture of this disclosure combines hardware acceleration, parallel processing, and intelligent optimization to achieve efficient, flexible, and high-quality video transcoding processing.

[0074] In this embodiment, at the hardware level, a software-defined VPU architecture (SD-VPU) is adopted, integrating a high-performance computing core (RISC-V) and a dedicated ASIC acceleration unit, significantly improving the programmability of the VPU. The TeleVPU supports flexible super-resolution virtualization, is equipped with larger capacity video memory, and can simultaneously cache multiple AI models and a large number of video files, effectively reducing the communication frequency with the host and thus improving efficiency. At the software level, it provides full-stack driver service capabilities, supporting key functions including video transcoding, video compression, video monitoring, hyperparameter adaptive optimization, stitching, scaling, AI bitrate control, and multi-card, multi-core scheduling.

[0075] Figure 8 To achieve the high bitrate video transcoding effect of the RISC-V VPU in the embodiments of this disclosure, from Figure 8 As can be seen, for high-bitrate raw videos, while ensuring basic clarity, the video size can be reduced by more than 90% by using VPU hardware and video compression, which greatly reduces video storage costs and promotes the rapid implementation and application of this product in video cloud services.

[0076] Figure 9 To achieve the low bitrate video transcoding effect of this embodiment, from Figure 9 As can be seen, for low-bitrate original videos, the original video is already unclear. By reducing the video bitrate and frame rate, VPU can still achieve a certain compression factor, and the video size can be reduced by 50%-80%.

[0077] Table 1 shows the test results for the number of transcoding channels in this embodiment. As can be seen from Table 1, when the VPU does not reduce the video frame rate (25fps) for transcoding, it can transcode a maximum of 40 channels of 1080P video. When the video frame rate is reduced, the overall transcoding time is significantly reduced because fewer video frames need to be encoded, allowing for the transcoding of more video files within the same timeframe.

[0078] Table 1

[0079]

[0080] Figure 10a and Figure 10b To achieve the low bitrate video transcoding effect of this embodiment, when the VPU does not reduce the video frame rate (25fps) for transcoding, the overall transcoding time is not significantly different because the number of video frames being encoded remains the same. However, when the video frame rate is reduced (10fps), the overall transcoding time is significantly reduced because fewer video frames are being encoded.

[0081] In some embodiments, the video processing card (TeleVPU card) adopts a single-width, full-length, full-height PCIe card form factor, which can be easily integrated into existing 2U and 4U rack servers, providing high-performance support for video processing. A single domestically developed server has eight slots, allowing up to eight transcoding cards to be inserted simultaneously, providing users with greater processing power. In terms of software, the transcoding card offers a rich selection of software interfaces. Users can choose to use the standard FFmpeg interface or their own interfaces, which cover various video processing functions such as encoding, decoding, scaling, stitching, video compression, and AI, meeting the needs of different users. This flexibility allows users to choose the most suitable software interface according to their own needs, realizing customized video processing solutions.

[0082] In the embodiments disclosed herein, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The concepts of "first," "second," etc., mentioned in this disclosure are only used to distinguish different devices, modules, or units and are not used to define the order of functions performed by these devices, modules, or units or their interdependencies.

[0083] In this disclosure, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0084] It should be noted that although several modules or units of the device used for action execution are mentioned in the detailed description above, this division is not mandatory.

[0085] In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0086] Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0087] Those skilled in the art will understand that all or part of the steps of the above embodiments can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to as "circuit", "module" or "system".

[0088] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein.

[0089] This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A system for video processing, characterized in that, The computing power base, the driving layer, the service layer and the application layer are sequentially arranged. The hardware of the computing power base comprises a plurality of video processing cards, each of which comprises a plurality of parallel video processing chips, each of which comprises a RISC-V computing core and a video processing unit; the plurality of parallel video processing chips are connected to a PCIe switch; each video processing chip further comprises an AI computing unit, which analyzes video content by using an algorithm to realize quality enhancement, intelligent adjustment of encoding parameters and / or scene recognition; the processed video is pushed to the RISC-V computing core of the target video processing chip through the PCIe switch, and the RISC-V computing core calls the video processing unit and the AI computing unit to cooperatively process the video according to the task requirements of the video processing task; The application layer is configured to initiate a video processing task. The video processing service engine in the service layer distributes the video processing task to a target video processing chip according to the task requirements of the video processing task and the load of the RISC-V computing core in each video processing card; the video processing service engine in the service layer distributes the video processing task to a plurality of target video processing chips according to the task requirements of the video processing task and the load of the RISC-V computing core in each video processing card, and the plurality of target processing chips are configured to simultaneously process multiple video streams or simultaneously process different parts of one video. The driving layer is configured to standardize and pre-process the input video corresponding to the video processing task, and push the processed video to the target video processing chip through the PCIe switch, and the RISC-V computing core and the video processing unit of the video processing chip cooperatively process the video.

2. The system of claim 1, wherein, The plurality of target processing chips belong to the same video processing card, or the plurality of target processing chips belong to a plurality of video processing cards.

3. The system of claim 1, wherein, The video processing service engine in the service layer distributes the video processing task to one or more target video processing chips according to the task requirements of the video processing task and the hardware resource state information, and the hardware resource state information comprises the load of the RISC-V computing core in each video processing card and the hardware temperature.

4. The system of claim 1, wherein, The application layer initiates a plurality of video processing tasks; the video processing service engine of the service layer is further configured to adjust the execution order of the plurality of video processing tasks in combination with the task priority and the hardware resource state information.

5. The system of claim 3 or 4, wherein, The driving layer further collects hardware resource state information in real time and feeds back the hardware resource state information to the video processing service engine of the service layer.

6. The system of claim 5, wherein, Each video processing card is provided with a separate baseboard management controller, which is configured to monitor the hardware state of the transcoding card in real time; the driving layer is provided with a state monitoring interface, and the driving layer obtains hardware resource state information from each baseboard management controller through the state monitoring interface.

7. The system of claim 1, wherein, The video processing task comprises one or more of the following tasks: Video transcoding, video compression, video monitoring, hyperparameter adaptive optimization, splicing, scaling, AI bitrate control.

8. The system of claim 1, wherein, The video processing service engine of the service layer also combines task load and hardware resource state information to dynamically adjust transcoding parameters for preprocessing input video of the video processing task in the driving layer.

Images