Video encoding and decoding system, and electronic device

Abstract

Provided in the embodiments of the present disclosure are a video encoding and decoding system, and an electronic device. The video encoding and decoding system comprises: a first processing unit, a second processing unit, a data pre-processing unit, a first storage unit and a second storage unit, wherein the first processing unit is configured to generate first intermediate data on the basis of original image data in the first storage unit; the second processing unit is configured to generate second intermediate data on the basis of the original image data in the first storage unit; the first storage unit is configured to store the first intermediate data and the second intermediate data; the second storage unit is configured to store control flow data; and the data pre-processing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate a quantization parameter on the basis of the first intermediate data and the second intermediate data and use same as first output data. By means of the video encoding and decoding system, a Cutree algorithm is implemented by means of hardware logic, and the Cutree algorithm is decoupled from firmware, so that the execution efficiency of the Cutree algorithm is improved, thereby solving problems such as first-frame delay and scheduling and data blockage.

Description

Video encoding and decoding systems and electronic devices

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411519486.X, filed on October 29, 2024, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Embodiments of this disclosure relate to a video encoding / decoding system and an electronic device. Background Technology

[0004] In current video coding algorithms, Cupree is a component of video pre-analysis algorithms that provides block-level quantization parameters (QPs). It analyzes the reference dependencies of pixel blocks within a certain look-ahead range, quantizes and accumulates the importance of pixel blocks in the reference frames, and provides crucial pixel-level QP parameters for the encoder's subsequent mode decision (MD) part. This algorithm effectively reduces the accumulation of errors caused by inter-frame prediction, enhancing the visual effect of the video to the human eye.

[0005] This pre-analysis algorithm, combined with the bitstream control algorithm, can appropriately increase the frame-level quantization parameter (QP) to balance parts of different importance in the video, thereby improving video quality without increasing the final video bitstream data volume. Summary of the Invention

[0006] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] At least one embodiment of this disclosure provides a video encoding and decoding system, including: a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit;

[0008] The first processing unit is configured to generate first intermediate data based on the original image data in the first storage unit;

[0009] The second processing unit is configured to generate second intermediate data based on the original data in the first storage unit;

[0010] The first storage unit is configured to store the first intermediate data and the second intermediate data;

[0011] The second storage unit is configured to store control flow data;

[0012] The data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as the first output data based on the first intermediate data and the second intermediate data.

[0013] At least one embodiment of this disclosure also provides an electronic device, including the video encoding and decoding system provided in any of the above embodiments of this disclosure. Attached Figure Description

[0014] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0015] Figure 1 shows a schematic block diagram of a video encoding and decoding system provided in at least one embodiment of the present disclosure;

[0016] Figure 2A is a schematic diagram of the structure of a data preprocessing unit provided in at least one embodiment of the present disclosure;

[0017] Figure 2B is a schematic diagram of another data preprocessing unit provided in at least one embodiment of the present disclosure;

[0018] Figure 2C is a schematic diagram of another data preprocessing unit provided in at least one embodiment of the present disclosure;

[0019] Figure 2D is a schematic diagram of another data preprocessing unit provided in at least one embodiment of the present disclosure;

[0020] Figure 3 illustrates the propagation method of propagation weights;

[0021] Figure 4 is a schematic diagram of a reference architecture for a look-ahead window provided in at least one embodiment of the present disclosure;

[0022] Figure 5 is a control flowchart of a control state machine core provided in at least one embodiment of the present disclosure;

[0023] Figure 6 is a schematic diagram of the architecture of a frame set GOP provided in at least one embodiment of this disclosure;

[0024] Figure 7 shows a schematic diagram of the storage space of a reference weight block storage module provided in at least one embodiment of the present disclosure;

[0025] Figure 8 is a control flowchart of another control state machine core provided in at least one embodiment of the present disclosure;

[0026] Figure 9 is a schematic diagram of two adjacent look-ahead windows provided in at least one embodiment of this disclosure;

[0027] Figure 10 shows the control flowchart of the core of the control state machine when the look-ahead window has multiple layers of reference relationships;

[0028] Figure 11 shows a schematic diagram of an electronic device provided in at least one embodiment of the present disclosure; and

[0029] Figure 12 shows a schematic diagram of the specific structure of another electronic device provided in at least one embodiment of the present disclosure. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] The inventors noted that in current video codec hardware, the Cupree algorithm is implemented using on-chip firmware. While firmware offers greater flexibility, the Cupree algorithm has many data dependencies and uses a large amount of data, which consumes firmware processing time, leading to problems such as data congestion, scheduling blockage, and severe first-frame latency.

[0037] At least one embodiment of this disclosure provides a video encoding and decoding system, including: a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit; the first processing unit is configured to generate first intermediate data based on raw image data in the first storage unit; the second processing unit is configured to generate second intermediate data based on the raw data in the first storage unit; the first storage unit is configured to store the first intermediate data and the second intermediate data; the second storage unit is configured to store control flow data; the data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as first output data based on the first intermediate data and the second intermediate data.

[0038] At least one embodiment of this disclosure also provides an electronic device, including the video encoding and decoding system provided in any of the above embodiments of this disclosure.

[0039] The video encoding and decoding system provided in the above embodiments of this disclosure implements the Cupree algorithm through hardware logic, thereby separating the Cupree algorithm from the firmware. This improves the efficiency of the Cupree algorithm execution, reduces the first frame latency, significantly reduces scheduling and solves problems such as data blocking. Furthermore, it provides the firmware with some data required by the code control algorithm (RC), further reducing the pressure on the firmware and improving the efficiency of the video encoding preprocessing part.

[0040] The embodiments and some examples of this disclosure will now be described in detail with reference to the accompanying drawings.

[0041] Figure 1 shows a schematic block diagram of a video codec system provided in at least one embodiment of the present disclosure. For example, as shown in Figure 1, Figure 1 illustrates the upstream and downstream dependencies of a data preprocessing unit (e.g., a Cupree module for implementing the Cupree algorithm) in the video codec hardware architecture: the data preprocessing unit provides block-level quantization parameter offsets (QP offsets) for the subsequent video encoder (VE); the data preprocessing unit 130 receives data from the upstream first processing unit 110 (e.g., an intra-processor) and second processing unit 120 (an inter-processor), calculates the quantization parameter offset (QP offset) for each pixel block (i.e., the coding unit (CU)), and provides the necessary data for the code control algorithm, such as frame cost data, for the second storage unit 150 (e.g., a firmware module (FW)). For example, in some examples, the data preprocessing unit 130 may not calculate the frame-level prediction loss, but can calculate it itself through the firmware. The embodiments of this disclosure do not limit this.

[0042] For example, as shown in Figure 1, the video encoding and decoding system 100 includes: a first processing unit 110, a second processing unit 120, a data preprocessing unit 130, a first storage unit 140, and a second storage unit 150.

[0043] For example, the first processing unit 110 is configured to generate first intermediate data based on the original image data in the first storage unit 140. For example, the first processing unit 110 may be an intra-frame processor, and the first intermediate data it generates may be, for example, intra-frame loss data of the intra-frame prediction CU or intra-frame quantization parameters at the initial pixel block level calculated only based on intra-frame prediction, etc. The embodiments of this disclosure are not limited in this regard.

[0044] For example, the second processing unit 120 is configured to generate second intermediate data based on the raw data in the first storage unit 140. For example, the second processing unit 120 can be an inter-frame processor, and the second intermediate data it generates can be, for example, inter-frame loss data of the inter-frame prediction CU or inter-frame motion vector (MV) for the corresponding reference frame, etc. The embodiments of this disclosure are not limited in this regard.

[0045] For example, the first storage unit 140 is configured to store first intermediate data and second intermediate data. For example, the first storage unit 140 can be a dynamic random access memory (DRAM), or other memory that can be used for storage, and the embodiments of this disclosure are not limited thereto.

[0046] For example, the second storage unit 150 is configured to store control flow data. This control flow data, for example, is used in code control algorithms, and can be used to generate frame-level prediction loss. For example, the second storage unit 150 can be firmware.

[0047] For example, the data preprocessing unit 130 is configured to read first intermediate data and second intermediate data from the first storage unit 140, and generate a first output data frame-level prediction loss based on the first intermediate data and the second intermediate data. For example, the data preprocessing unit 130 is used to execute the Cutree algorithm to generate block-level quantization parameters. For example, the first output data may include quantization parameters for output to the first storage unit 140 for subsequent calculations, and the embodiments of this disclosure are not limited thereto.

[0048] For example, in the video encoding and decoding system shown in Figure 1, the data preprocessing unit 130 for implementing the Cupree algorithm is separated from the second storage unit 150 (i.e., firmware), thereby improving the efficiency of the Cupree algorithm execution, reducing the first frame latency, significantly reducing scheduling and solving problems such as data blocking, and improving the efficiency of the video encoding preprocessing part.

[0049] For example, in other examples, the data preprocessing unit 130 is also configured to read control flow data from the second storage unit 150, and generate frame-level prediction loss as second output data based on the control flow data and output it to the second storage unit 150.

[0050] For example, in this example, the data preprocessing unit 130 provides the second storage unit (firmware) with the data required by the code control algorithm (RC) (e.g., frame-level prediction loss), thereby further reducing the pressure on the firmware. The firmware can then focus on the code control algorithm and overall scheduling, without performing any actual block-level calculations, significantly reducing the burden on the firmware. This allows for reduced complexity in firmware design, lower development and maintenance difficulty, and the use of smaller firmware in the hardware top-level design. In actual use, the Cutree module in the hardware (i.e., the data preprocessing unit 130) operates for a shorter time than required by the firmware and consumes relatively fewer resources, thus improving overall stability and efficiency.

[0051] Figure 2A is a schematic diagram of the structure of a data preprocessing unit provided in at least one embodiment of the present disclosure. For example, as shown in Figure 2A, the data preprocessing unit 130 includes a control register module 131, at least one control state machine core 132, a data acquisition module 133, a reference weight calculation module 134, a reference weight block storage module 135, a quantization parameter offset calculation module 136, and a quantization parameter offset result output module 137.

[0052] For example, the control register module 131 is connected to the second storage unit 150 and to at least one control state machine core 132 in the data preprocessing unit 130 (not shown in the figures for clarity and simplicity), configured to broadcast information to all control state machine cores 132. This information may include, for example, flow control parameters. It should be noted that the control register module 131 can also be connected to other modules to directly provide them with the data they require; embodiments of this disclosure do not limit this.

[0053] For example, in some examples, all the main modules in the data preprocessing unit 130 (e.g., data acquisition module 133, reference weight calculation module 134, quantization parameter offset calculation module 136, quantization parameter offset result output module 137, and reference weight result output module 138 and frame-level prediction loss calculation module 139 provided in subsequent embodiments) instantiate a single control state machine core 132. This ensures that the processing flow of all main modules is consistent, reduces the number of signal feedbacks between modules, avoids potential problems, and reduces the maintenance difficulty caused by different flow controls for different main modules. For example, the control state machine core 132 is configured to control the flow consistency of each module in the data preprocessing unit 130 according to the flow control parameters.

[0054] For example, in other examples, the data preprocessing unit 130 may include only one control state machine core 132, which transmits control signals along the pipeline to the various modules 133-139 via pipeline delay. This method can reduce the repeated instantiation of the control state machine core 132 and can be used to process smaller data, such as data smaller than 4K. Of course, the embodiments of this disclosure are not limited to this.

[0055] For example, the data acquisition module 133 is configured to send data requests and arrange the returned data in a certain order for distribution to various modules. For example, the returned data could be first intermediate data, second intermediate data, the control flow, or other data required for subsequent calculations by various modules. For example, the data acquisition module 133 balances the data requirements of various modules in the Cutree algorithm. The control state machine core 132 outputs the data requirement states for different processes. Based on the parameters provided by the control register module 131, the position of the corresponding data in the frame, and the base addresses of various data types, the specific address of the data can be calculated for the data acquisition module 133 to read.

[0056] For example, the reference weight calculation module 134 is configured to obtain corresponding pre-data from the data acquisition module 133, quantify the importance of the current codec unit based on the pre-data to obtain weight parameters, calculate the weight parameters using motion vectors, and send them to the reference block of the current codec unit. For example, the pre-data may include the intra-frame and inter-frame prediction loss for comparison, as well as the previously calculated reference weight of the current CU. The reference weight calculation module 134 can quantify the importance of the current CU and transmit it as a weight parameter to the upper-level CU (i.e., the reference block of the current CU) referenced by the current CU. For example, the reference weight calculation module 134 quantifies the importance of the CU by comparing the intra-frame and inter-frame prediction loss with the reference weight of the current CU, and transmits it to the upper-level CU referenced by the current CU using motion vectors.

[0057] Figure 2C is a schematic diagram of another data preprocessing unit provided in at least one embodiment of this disclosure. For example, in some other examples, as shown in Figure 2C, based on Figure 2A, the data processing unit 130 includes two reference weight calculation modules 134, configured to perform bidirectional propagation to access the reference weight block storage module 135 respectively. For example, one reference weight calculation module 134 processes the calculation and propagation of the reference weights of the left keyframe, and the other reference weight calculation module 134 processes the calculation and propagation of the reference weights of the right keyframe, thereby realizing bidirectional propagation and improving computational efficiency.

[0058] For example, the reference weight block storage module 135 includes storage spaces corresponding to multiple adjacent encoding / decoding units of the reference frame, and is configured to store the weight parameters calculated by motion vectors into the corresponding storage spaces. For example, since the basic operation unit of the Cupree algorithm is a pixel block (CU), and the basic unit of motion vectors is a pixel, the weight data calculated by motion vectors will be accumulated into the adjacent (e.g., 4) CUs in the top, bottom, left, and right of the reference frame. Here, the reference weight block storage module is designed as 4 independent storage spaces corresponding to 4 CUs respectively, thereby realizing the design of a four-part storage format to optimize execution efficiency.

[0059] For example, Figure 3 illustrates the propagation method of propagation weights. As shown in Figure 3, the right side is the reference frame on the left, and each box represents a CU. It can be seen that the referenced block may not completely overlap with a single block on the reference frame; for example, it may be distributed across four adjacent CUs. During the accumulation process, the corresponding weight must be multiplied based on the actual area occupied by the referenced block in these four adjacent CUs. Therefore, each calculation requires reading four data points and writing them back. Thus, in the embodiments of this disclosure, the propagation weights storage method in the reference weight block storage module 135 shown in Figure 2A is adopted. Each CU in a frame is divided into four parts according to the parity of its row and column, and stored separately. In this way, the four adjacent CUs must exist in these four storage areas respectively, without overlap, and can therefore be implemented with a single read, thereby reducing the complexity of internal data reading.

[0060] For example, in the example shown in Figure 2A, the data preprocessing unit does not include a reference weight result output module, thereby expanding the storage space within the data processing unit 130 (e.g., the storage space of the reference weight block storage module 135 can be expanded to q frames, where q is an integer greater than or equal to 1). All propagation weights can be directly cached in the reference weight block storage module 135, which reduces the complexity of control flow design and lowers the total bandwidth. This example can be used when the data volume is small. For example, in this example, all calculation results can be stored in the reference weight block storage module 135, and the data stored in the reference weight block storage module 135 can be immediately used for subsequent calculations through the control of the control state machine core 132, thus reusing the internal storage.

[0061] Figure 2B is a schematic diagram of another data preprocessing unit provided in at least one embodiment of this disclosure. For example, in some other examples, as shown in Figure 2B, based on Figure 2A, the data processing unit 130 further includes a reference weight result output module 138, configured to output the calculation result of the weight parameters stored in the reference weight block storage module to the first storage unit 140. For example, based on the control signal given by the control state machine core 132, it is determined whether there is a need to output the calculated reference weight data to the first storage unit 140. If so, the corresponding data is read from the reference weight block storage module 138 and written to the first storage unit 140. For example, the calculation result of the weight parameters is the final reference weight of each block in a certain frame obtained after storage accumulation.

[0062] For example, in this example, the data in the reference weight block storage module 135 can be transmitted to the first storage unit 140 for storage through the reference weight result output module 138. When needed, it can be read from the first storage unit 140, thereby freeing up the space of the reference weight block storage module 135. For example, in this example, the storage space of the reference weight block storage module 135 can be reduced to the space for storing multiple rows in one frame (e.g., k rows, where k is an integer greater than 0), thereby reducing the internal storage space.

[0063] For example, the quantization parameter offset calculation module 136 is configured to obtain quantization parameters based on the final weight parameters. For example, these quantization parameters include a quantization parameter offset. For example, the degree to which each CU in the keyframe is referenced in subsequent frames is obtained based on the calculated reference weights; for example, a higher degree results in a lower QP, thus leading to better video quality. For example, the control signals provided by the control state machine core 132, at the keyframe, provide more accurate block-level QP data for subsequent video encoding based on the previously calculated reference weights.

[0064] For example, the quantization parameter offset result output module 137 is configured to determine whether the quantization parameter needs to be output, calculate the address offset corresponding to the quantization parameter, and output it to the first storage unit. For example, based on the control signal of the control state machine core 132, it determines whether the corresponding QP offset result needs to be output, calculates the output address offset, and passes it to the first storage unit 140.

[0065] Figure 2D is a schematic diagram of another data preprocessing unit provided in at least one embodiment of this disclosure. For example, in some other examples, as shown in Figure 2D, based on Figure 2B, the data processing unit 130 further includes a frame-level prediction loss calculation module 139, configured to calculate the frame-level prediction loss as the second output data based on the quantization parameter offset and the control flow data. For example, the frame-level prediction loss calculation module 139 obtains the QP offset calculated by the quantization parameter offset calculation module 136, and recalculates the frame-level prediction loss based on the QP offset, providing key data for the code control algorithm in the second storage unit 150. The frame-level prediction loss is transmitted back to the second storage unit 150 through the control register module 131.

[0066] For example, in this example, the frame-level prediction loss calculation module 139 in the data preprocessing unit 130 provides the second storage unit (firmware) with the data required by the code control algorithm (RC) (e.g., frame-level prediction loss), thereby further reducing the pressure on the firmware. The firmware can then focus on the code control algorithm and overall scheduling, without performing any actual calculations at the block level, significantly reducing the burden on the firmware. This allows for reduced complexity in firmware design, lower development and maintenance difficulty, and the use of smaller firmware in the hardware top-level design. In actual use, the Cutree module (i.e., the data preprocessing unit 130) in the hardware operates for a shorter time than the firmware requires and consumes relatively fewer resources, thus improving overall stability and efficiency.

[0067] For example, the second storage unit 150 is further configured to configure a look-ahead window for the control state machine core, wherein the look-ahead window includes multiple group of frames (GOPs). For example, the second storage unit 150 describes a reference architecture for the look-ahead window and sets a fast mode for the control state machine core based on the reference architecture to reduce the computation of duplicate frame sets in adjacent look-ahead windows. For example, the reference architecture of the look-ahead window determines duplicate GOPs in two adjacent look-ahead windows, and the fast mode is set so that the relevant data of the duplicate GOP can be directly called from the previous look-ahead window to reduce the computation of duplicate frame sets in adjacent look-ahead windows. For example, the second storage unit 150 is configured to describe the reference architecture and fast mode of the look-ahead window and send them to the control state machine core via the control register module.

[0068] Figure 4 is a schematic diagram of a reference architecture for a look-ahead window provided in at least one embodiment of the present disclosure.

[0069] For example, each execution of the Cupree algorithm operates on a look-ahead window basis, traversing the entire window in a single execution. A look-ahead window consists of an integer number of Groups of Pictures (GOPs). For instance, frames 0-8 in Figure 4 constitute a GOP. To improve flexibility, the propagation process within each GOP can be configured individually, including the start and end points, the target frames for each B-frame propagation, and the propagation order. This configuration is transmitted as control data from the second storage unit 150 to the control state machine core 132 via the control register module 131. If there is no need for flexibility in configuring GOPs within a look-ahead window, each GOP can be set to be identical, reducing the need for GOP configuration. This process can be configured through the second storage unit (i.e., firmware), improving flexibility.

[0070] For example, as shown in Figure 4, the length of the lookahead window currently being processed by Cupre is 0-38, and the keyframes are frames 0, 8, 16, 32, and 38. For example, as shown in Figure 4, the transmission order (i.e., GOP configuration) can be described by importance as 084213657… Figure 4 is merely exemplary, and this disclosure is not intended to limit it.

[0071] For example, due to the existence of lookahead windows, there are repetitive parts in different Cutree processing flows. Fast mode exists to reduce repetitive processes (e.g., reducing the processing and calculation of duplicate GOPs, which can be directly called) and improve efficiency at the cost of bandwidth and memory space. If bandwidth is insufficient or increased complexity is undesirable, this register configuration can be omitted. For example, if frames 8-38 have already been calculated when processing frames 0-38, the GOP values ​​for the next lookahead window (8-46) can be directly called when calculating the next lookahead window, avoiding redundant calculations and thus achieving fast mode.

[0072] Figure 5 is a control flowchart of a control state machine core provided in at least one embodiment of this disclosure.

[0073] In the process shown in Figure 5, the following settings can be made:

[0074] (1) There are several GOPs with the same reference architecture as shown in Figure 6 in a look-out window. A GOP can be defined as a combination of all frames between two non-B frames, including two non-B frames, with the left one called REF0 and the right one called REF1.

[0075] (2) In the reference architecture shown in Figure 6, all B frames are unidirectionally referenced to REF0 or REF1, or bidirectionally propagated to REF0 and REF1;

[0076] (3) The storage space of the reference weight block storage module 135 can cache all the reference weights of REF0 and REF1 frames;

[0077] (4) Do not use the fast mode, that is, repeated operations between different processing flows are not skipped, but recalculated.

[0078] Figure 6 is a schematic diagram of the architecture of a frame set (GOP) provided in at least one embodiment of this disclosure. As shown in Figure 6, B-frames are bidirectional reference frames, P-frames are unidirectional reference frames, and I-frames are completely independent frames. As shown in Figure 6, the area within the dashed box represents a GOP. Within a GOP, the frame on the left is used as reference frame REF0, and the frame on the right is used as reference frame REF1. For example, each bidirectional B-frame in the middle references either REF0 or REF1, while REF1 only references REF0. Therefore, REF0 and REF1 between two GOPs are interchangeable. For example, REF1 of the GOP in the dashed box in Figure 4 can be used as REF0 of the next GOP. Therefore, during execution, the later GOP is executed first, followed by the earlier GOP, and the data of REF0 of the later GOP can be directly switched and used as REF1 of the previous GOP.

[0079] Based on the above settings, for example, if a single process of Cupreel is abstracted into the architecture shown in Figure 6, its processing order satisfies the following rules:

[0080] (1) GOPs within the lookahead window are processed from back to front;

[0081] (2) Within a GOP, the B-frames are processed first, followed by REF1. REF0 is processed until the leftmost GOP in the look-ahead window. In this processing order, the REF0 of the current GOP is the REF1 of the next GOP.

[0082] (3) The frame is processed line by line in the order of the CU;

[0083] (4) After the calculation of a frame is completed, output the propagation weight parameters or QP offset. Output the QP offset for the leftmost GOP and output the propagation weight parameters for the other GOPs.

[0084] For example, in the setup shown in Figure 5, the control state machine core can be configured to perform the following control flow:

[0085] First, the storage module within the reference weight block is reset, for example, the internal storage is refreshed to a state of 0. Then, after calculating and writing the weight parameters propagating from the B-frame, it is determined whether all B-frames have been processed. If not, the calculation and writing operation is continued on the unprocessed B-frame (i.e., the next B-frame). If all B-frames have been processed, the calculation and writing operation of the weight parameters propagating from the reference frame REF1 is performed again. After REF1 is completed, it is determined whether the current GOP containing REF1 is the last GOP. If not, the above operation is performed on the next GOP. If so, the weight parameters of the reference frame REF0 in the last GOP of the lookahead window are calculated and written. It should be noted that the above operation is performed from back to front. For example, corresponding to the structure in Figure 4, it starts from the GOP corresponding to frame 38 and proceeds backward to frame 0, thus completing the calculation of one lookahead window, and so on.

[0086] For example, in some examples, the storage space of the reference weight block storage module includes the radiation range of inter-frame reference, such as k rows (k is an integer greater than 0) as shown in Figure 7. The specific range can be determined according to actual conditions, and the embodiments of this disclosure do not limit this. Figure 7 shows a schematic diagram of the storage space of a reference weight block storage module 135 provided in at least one embodiment of this disclosure. For example, as shown in Figure 7, considering the increase in video resolution, the storage space of the reference weight block storage module within the Cupre will increase exponentially by m² with the increase in the number of pixel rows m (m is an integer greater than 0), which will lead to a significant increase in cost. To overcome the increase in storage space, in the embodiments of this disclosure, the height of the storage space for propagating weights (weight parameters) within the Cupre is fixed, while the width still increases with the increase in the number of pixel rows m. The height is the vertical radiation range of the motion vector (i.e., the radiation range k of the inter-frame reference), which, as shown in Figure 7, is the influence range of the CU row of the current frame on the referenced frame to the right. For example, after the first row of the referenced frame is no longer affected by the CU row of the current frame on the left, its weight parameters can be output.

[0087] In this example, by adding a constraint on storage space, the control granularity of the control state machine core can be changed from frame-level control as shown in Figure 5 to control of each CU row, thus making the calculation more accurate. The specific control flow is shown in Figure 8.

[0088] Figure 8 is a control flowchart of another control state machine core provided in at least one embodiment of this disclosure. For example, as shown in Figure 8, the control state machine core can be configured to perform the following control flow:

[0089] First, reset the storage module within the reference weight block, for example, refresh the internal storage to a state of 0. Then, after calculating and writing the weight parameters of the current line codec unit (CU) propagating from the B-frame, it is determined whether all lines in all B-frames have been processed. If not, it switches to the next line codec unit of the B-frame to continue the calculation and writing operation. If yes, it calculates and writes the weight parameters of the current line codec unit propagating from the reference frame REF1, and determines whether all lines in the reference frame REF1 have been processed. If not, it switches to the next line codec unit of the reference frame REF1 to continue the calculation and writing operation. If yes, it determines whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window. If not, it continues to perform calculation and writing operations on the next GOP in the lookahead window. If yes, it calculates and writes the weight parameters of the current line codec unit of the reference frame REF0 in the last GOP, and determines whether all lines in the reference frame REF0 have been processed. If not, it switches to the next line codec unit of the reference frame REF0 to continue the calculation and writing operation. If yes, the calculation ends.

[0090] Figure 9 is a schematic diagram of two adjacent look-ahead windows provided in at least one embodiment of this disclosure. For example, as shown in Figure 9, the overlapping parts of the two adjacent look-ahead windows are within the dashed box. Therefore, in the subsequent processing flow of the second look-ahead window, a fast mode can be used to omit the calculation of the overlapping parts, and the calculation result of the first look-ahead window can be directly read, thereby improving the efficiency of data processing.

[0091] Figure 10 shows the control flow diagram of the core of the control state machine when the look-ahead window has multiple layers of reference relationships (e.g., the hierarchy of the first GOP shown in Figure 4 is 084213657). For example, in the flow shown in Figure 10, a fast mode can be added to the flow shown in Figure 8.

[0092] For example, referring to Figure 10, the control state machine core can be configured to perform the following control flow:

[0093] First, the frame set information of the current GOP is obtained from the second storage unit 150, and it is determined whether the current GOP is a new GOP architecture based on the frame set information. That is, if it is not a new GOP, its corresponding weight parameters can be directly called from the first storage unit 140, and the B-frame judgment calculation is no longer performed. If it is a new GOP, the following judgment is performed.

[0094] For example, the frame set information includes a configurable reference architecture for the GOP, such as the architecture shown in Figure 4, Figure 6, or Figure 9. The specific architecture can be determined according to the actual situation, and this disclosure does not impose any restrictions on it. For example, the reference architecture information includes parameters such as the source index, the destination index, the reference layer of the source frame, and whether the propagation is the last segment of the GOP (Ending flag).

[0095] If it's a new GOP architecture, the complexity of the current GOP is calculated, and the processing feature information of the current GOP is invoked based on the complexity. This processing feature information includes, for example, the underlying features corresponding to the current GOP, such as whether weight parameters have already been calculated. Based on the processing feature information of the current GOP, it is determined whether the weight parameters corresponding to the current GOP need to be preloaded. If the weight parameters have already been calculated, they can be preloaded, and calculation and write operations are performed on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 based on the preloaded weight parameters. If not, the storage module within the reference weight block is reset, and the above calculation and write operations on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 are performed. For example, the calculation and write operations on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 can be seen in the specific description in Figure 8, and will not be repeated here. The weight parameters of the current CU line propagating from the B-frame are calculated.

[0096] The video encoding and decoding system provided in the above embodiments of this disclosure achieves the following: by heterogenizing the Cupree algorithm and converting it into RTL parallel digital logic; by instantiating a control state machine core in each major module in the core state machine design of Cupree; by using pixel block row switching to save storage space in each module; by using a fast mode control flow design to improve efficiency by reducing repetitive operations; and by using the customizable control logic design in the propagation architecture within the GOP and the module for accumulating Cupree propagation weights to design a four-part storage format, the Cupree algorithm can be implemented through hardware logic, separating the Cupree algorithm from the firmware, improving the efficiency of Cupree algorithm execution, reducing first frame latency, significantly reducing scheduling and solving data blocking problems, and also providing some data required by the code control algorithm (RC) for the firmware, further reducing the firmware pressure and improving the efficiency of the video encoding preprocessing part.

[0097] At least one embodiment of this disclosure also provides an electronic device, including the video encoding and decoding system provided in any of the above embodiments of this disclosure.

[0098] Figure 11 shows a schematic diagram of an electronic device provided by at least one embodiment of the present disclosure. For example, as shown in Figure 11, the electronic device 200 includes a video encoding / decoding system 100. For example, the video encoding / decoding system 100 can be implemented using the structure shown in Figure 1. For example, the electronic device 200 can be any electronic device with computing capabilities, such as a mobile phone, digital camera, laptop, tablet computer, desktop computer, network server, etc., which can load and execute the video processing method. The embodiments of the present disclosure do not limit this. For example, the electronic device can include a central processing unit (CPU) or graphics processing unit (GPU), digital signal processor (DSP), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, storage units, etc. The electronic device is also equipped with an operating system, application programming interfaces (e.g., OpenGL (Open Graphics Library), Metal, etc.), etc., and implements video encoding / decoding by running code or instructions. For example, the electronic device may also include an output component, such as a display component, which may be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a quantum dot light-emitting diode (QLED) display, etc. The embodiments disclosed herein are not limited in this respect.

[0099] It should be noted that, for clarity and brevity, this disclosure does not show all the constituent units of the electronic device 200. To achieve the necessary functions of the electronic device 200, those skilled in the art can provide and set other constituent units (not shown) according to specific needs, and this disclosure does not limit this.

[0100] Referring now to FIG12, a schematic diagram of the specific structure of an electronic device (e.g., a terminal device or server) 600 suitable for implementing a video encoding / decoding system according to embodiments of the present disclosure is shown. The terminal device in embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG12 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present disclosure.

[0101] As shown in Figure 12, the electronic device 600 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0102] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG12 shows electronic device 600 with various devices, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0103] Specifically, according to embodiments of this disclosure, the processes described above in the reference flowchart of the control state machine core or the video encoding / decoding method can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 608, or installed from ROM 602. When the computer program is executed by processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0104] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0105] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0106] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0107] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire at least two Internet Protocol (IP) addresses; send a node evaluation request including the at least two IP addresses to a node evaluation device, wherein the node evaluation device selects an IP address from the at least two IP addresses and returns it; and receive the IP address returned by the node evaluation device; wherein the acquired IP address indicates an edge node in a content delivery network.

[0108] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a node evaluation request including at least two Internet Protocol (IP) addresses; select an IP address from the at least two IP addresses; and return the selected IP address; wherein the received IP address indicates an edge node in the content delivery network.

[0109] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0111] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0112] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0113] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0114] According to one or more embodiments of this disclosure, Example 1 provides a video encoding / decoding system, including: a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit; wherein,

[0115] The first processing unit is configured to generate first intermediate data based on the original image data in the first storage unit;

[0116] The second processing unit is configured to generate second intermediate data based on the original data in the first storage unit;

[0117] The first storage unit is configured to store the first intermediate data and the second intermediate data;

[0118] The second storage unit is configured to store control flow data;

[0119] The data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as the first output data based on the first intermediate data and the second intermediate data.

[0120] According to one or more embodiments of this disclosure, Example 2 provides a video encoding and decoding system of Example 1, wherein the data preprocessing unit is further configured to read the control flow data from the second storage unit, and generate frame-level prediction loss as second output data based on the control flow data and output it to the second storage unit.

[0121] According to one or more embodiments of this disclosure, Example 3 provides the video encoding and decoding system of Example 1, wherein the data preprocessing unit includes a control register module, at least one control state machine core, a data acquisition module, a reference weight calculation module, a reference weight block storage module, a quantization parameter offset calculation module, and a quantization parameter offset result output module; wherein,

[0122] The control register module is connected to the second storage unit and is configured to broadcast information to the at least one control state machine core, wherein the information includes process control parameters;

[0123] The at least one control state machine core is configured to control the processing flow of each module in the data preprocessing unit to be consistent according to the process control parameters.

[0124] The data acquisition module is configured to send a data request and arrange the returned data.

[0125] The reference weight calculation module is configured to obtain corresponding pre-data from the data acquisition module, quantify the importance of the current codec unit based on the pre-data to obtain weight parameters, calculate the weight parameters through motion vectors and send them to the reference block of the current codec unit. The pre-data includes intra-frame and inter-frame prediction loss and the reference weight of the current codec unit.

[0126] The reference weight block storage module includes storage spaces corresponding to multiple adjacent codec units of the reference frame, and is configured to store the weight parameters calculated by motion vectors into the corresponding storage spaces.

[0127] The quantization parameter offset calculation module is configured to obtain quantization parameters based on the weight parameters, wherein the quantization parameters include quantization parameter offsets;

[0128] The quantization parameter offset result output module is configured to determine whether the quantization parameter needs to be output, and to calculate the address offset corresponding to the quantization parameter and output it to the first storage unit.

[0129] According to one or more embodiments of this disclosure, Example 4 provides a video encoding / decoding system of Example 3, wherein the storage space includes the radiation range of an inter-frame reference.

[0130] According to one or more embodiments of this disclosure, Example 5 provides a video encoding and decoding system of Example 3, wherein the data preprocessing unit includes two reference weight calculation modules, and the two reference weight calculation modules are further configured to perform bidirectional propagation to access the reference weight block storage module respectively.

[0131] According to one or more embodiments of this disclosure, Example 6 provides a video encoding and decoding system of Example 3, wherein the data preprocessing unit further includes a reference weight result output module configured to output the calculation result of the weight parameters stored in the reference weight block storage module to the first storage unit.

[0132] According to one or more embodiments of this disclosure, Example 7 provides a video encoding and decoding system of Example 3, wherein the data preprocessing unit further includes a frame-level prediction loss calculation module configured to calculate frame-level prediction loss as second output data based on the quantization offset and the control flow data.

[0133] According to one or more embodiments of this disclosure, Example 8 provides the video encoding and decoding system of Example 7, wherein the frame-level prediction loss is returned to the second storage unit via the control register module.

[0134] According to one or more embodiments of this disclosure, Example 9 provides a video encoding / decoding system of Example 3, wherein the second storage unit is further configured to configure a look-ahead window required by the control state machine core, wherein the look-ahead window includes a plurality of frame sets (GOPs).

[0135] According to one or more embodiments of this disclosure, Example 10 provides a video encoding / decoding system of Example 9, wherein the second storage unit is further configured to describe a reference architecture of the look-ahead window and set a fast mode for the control state machine core based on the reference architecture of the look-ahead window to reduce the computation of duplicate frame sets in adjacent look-ahead windows.

[0136] According to one or more embodiments of this disclosure, Example 11 provides the video encoding / decoding system of Example 9, wherein the control state machine core is further configured as follows:

[0137] Reset the storage module within the reference weight block;

[0138] After calculating and writing out the weight parameters propagating from the B-frames, determine whether all B-frames have been processed.

[0139] If not, switch to the next B-frame to continue the aforementioned calculation and write operations;

[0140] If so, then the calculation and write operations are performed on the weight parameters propagated from the reference frame REF1, and it is determined whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window.

[0141] If not, continue performing the computation and write operations on the next GOP in the lookout window.

[0142] If so, then the calculation and write operation are performed on the weight parameters of the reference frame REF0 in the last GOP.

[0143] According to one or more embodiments of this disclosure, Example 12 provides the video encoding / decoding system of Example 9, wherein the control state machine core is further configured as follows:

[0144] Reset the storage module within the reference weight block;

[0145] After calculating and writing out the weight parameters of the current codec unit line propagating from the B-frame, it is determined whether all lines in all B-frames have been processed.

[0146] If not, switch to the next codec unit line of the B frame to continue the calculation and write operations;

[0147] If so, then perform the aforementioned calculation and write operation on the weight parameters of the current codec unit line propagating from reference frame REF1, and determine whether all lines of reference frame REF1 have been processed.

[0148] If not, then switch to the next encoding / decoding unit line of the reference frame REF1 to continue the calculation and write operation;

[0149] If so, determine whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window.

[0150] If not, continue performing the computation and write operations on the next GOP in the lookout window.

[0151] If so, then the calculation and write operation are performed on the weight parameters of the current codec unit line of the reference frame REF0 in the last GOP, and it is determined whether all lines of the reference frame REF0 have been processed.

[0152] If not, then switch to the next encoding / decoding unit line of the reference frame REF0 to continue the calculation and write operations.

[0153] According to one or more embodiments of this disclosure, Example 13 provides a video encoding / decoding system of Example 9, wherein the control state machine core is further configured as follows:

[0154] Obtain the frame set information of the current GOP from the second storage unit, and determine whether the architecture of the current GOP is a new GOP architecture based on the frame set information;

[0155] If so, calculate the complexity of the current GOP, call the processing feature information of the current GOP based on the complexity of the current GOP, and determine whether the weight parameters corresponding to the current GOP need to be preloaded based on the processing feature information of the current GOP.

[0156] If necessary, the weight parameters corresponding to the current GOP are preloaded, and calculation and writing operations are performed on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 based on the preloaded weight parameters corresponding to the current GOP; if not necessary, the storage module within the reference weight block is reset, and calculation and writing operations are performed on all codec unit lines of the B-frame, the reference frame REF1, and the reference frame REF0.

[0157] If not, then directly jump to the calculation and write-out operation for all codec unit lines of the reference frame REF1 and the reference frame REF0.

[0158] According to one or more embodiments of this disclosure, Example 14 provides a video encoding / decoding system as described in any of Examples 3-13, wherein each module of the data preprocessing unit instantiates one of the control state machine cores.

[0159] According to one or more embodiments of this disclosure, Example 15 provides an electronic device including the video encoding / decoding system provided in Examples 1-14.

[0160] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0161] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0162] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A video encoding and decoding system, comprising: The system comprises a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit; wherein... The first processing unit is configured to generate first intermediate data based on the original image data in the first storage unit; The second processing unit is configured to generate second intermediate data based on the original data in the first storage unit; The first storage unit is configured to store the first intermediate data and the second intermediate data; The second storage unit is configured to store control flow data; The data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as the first output data based on the first intermediate data and the second intermediate data.

2. The video encoding and decoding system according to claim 1, wherein, The data preprocessing unit is further configured to read the control flow data from the second storage unit, and generate frame-level prediction loss as second output data based on the control flow data and output it to the second storage unit.

3. The video encoding / decoding system according to claim 1 or 2, wherein, The data preprocessing unit includes a control register module, at least one control state machine core, a data acquisition module, a reference weight calculation module, a reference weight block storage module, a quantization parameter offset calculation module, and a quantization parameter offset result output module; wherein... The control register module is connected to the second storage unit and is configured to broadcast information to the at least one control state machine core, wherein the information includes process control parameters; The at least one control state machine core is configured to control the processing flow of each module in the data preprocessing unit to be consistent according to the process control parameters. The data acquisition module is configured to send a data request and arrange the returned data. The reference weight calculation module is configured to obtain corresponding pre-data from the data acquisition module, quantify the importance of the current codec unit based on the pre-data to obtain weight parameters, and transmit the weight parameters to the reference block of the current codec unit through motion vectors. The pre-data includes intra-frame and inter-frame prediction loss and the reference weight of the current codec unit. The reference weight block storage module includes storage spaces corresponding to multiple adjacent codec units of the reference frame, and is configured to store the weight parameters transmitted by motion vectors into the corresponding storage spaces. The quantization parameter offset calculation module is configured to obtain quantization parameters based on the weight parameters passed by the motion vector, wherein the quantization parameters include quantization parameter offsets. The quantization parameter offset result output module is configured to determine whether the quantization parameter needs to be output, and to calculate the address offset corresponding to the quantization parameter and output it to the first storage unit.

4. The video encoding and decoding system according to claim 3, wherein, The storage space includes the radiation range of the inter-frame reference.

5. The video encoding / decoding system according to claim 3 or 4, wherein, The data preprocessing unit includes two reference weight calculation modules, which are further configured to perform bidirectional propagation to access the storage modules within the reference weight blocks respectively.

6. The video encoding and decoding system according to any one of claims 3-5, wherein, The data preprocessing unit further includes a reference weight result output module, configured to output the calculation results of the weight parameters stored in the reference weight block storage module to the first storage unit.

7. The video encoding and decoding system according to any one of claims 3-6, wherein, The data preprocessing unit further includes a frame-level prediction loss calculation module, configured to calculate the frame-level prediction loss as the second output data based on the quantization offset and the control flow data.

8. The video encoding and decoding system according to claim 7, wherein, The frame-level prediction loss is returned to the second storage unit through the control register module.

9. The video encoding and decoding system according to any one of claims 3-8, wherein, The second storage unit is also configured to configure a look-ahead window for the control state machine core, wherein the look-ahead window includes multiple frame sets (GOPs).

10. The video encoding and decoding system according to claim 9, wherein, The second storage unit is also configured to describe the reference architecture of the look-ahead window and set a fast mode for the control state machine core based on the reference architecture of the look-ahead window to reduce the computation of duplicate frame sets in adjacent look-ahead windows.

11. The video encoding / decoding system according to claim 9 or 10, wherein, The core of the control state machine is also configured as follows: Reset the storage module within the reference weight block; After calculating and writing out the weight parameters propagating from the B-frames, determine whether all B-frames have been processed. If not, switch to the next B-frame to continue the aforementioned calculation and write operations; If so, then the calculation and write operations are performed on the weight parameters propagated from the reference frame REF1, and it is determined whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window. If not, continue performing the computation and write operations on the next GOP in the lookout window. If so, then the calculation and write operation are performed on the weight parameters of the reference frame REF0 in the last GOP.

12. The video encoding / decoding system according to claim 9 or 10, wherein, The core of the control state machine is also configured as follows: Reset the storage module within the reference weight block; After calculating and writing out the weight parameters of the current codec unit line propagating from the B-frame, it is determined whether all lines in all B-frames have been processed. If not, switch to the next codec unit line of the B frame to continue the calculation and write operations; If so, then perform the aforementioned calculation and write operation on the weight parameters of the current codec unit line propagating from reference frame REF1, and determine whether all lines of reference frame REF1 have been processed. If not, then switch to the next encoding / decoding unit line of the reference frame REF1 to continue the calculation and write operation; If so, determine whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window. If not, continue performing the computation and write operations on the next GOP in the lookout window. If so, then the calculation and write operation are performed on the weight parameters of the current codec unit line of the reference frame REF0 in the last GOP, and it is determined whether all lines of the reference frame REF0 have been processed. If not, then switch to the next encoding / decoding unit line of the reference frame REF0 to continue the calculation and write operations.

13. The video encoding and decoding system according to any one of claims 9-12, wherein, The core of the control state machine is also configured as follows: Obtain the frame set information of the current GOP from the second storage unit, and determine whether the architecture of the current GOP is a new GOP architecture based on the frame set information; If so, calculate the complexity of the current GOP, call the processing feature information of the current GOP based on the complexity of the current GOP, and determine whether it is necessary to preload the weight parameters corresponding to the current GOP based on the processing feature information of the current GOP. If necessary, the weight parameters corresponding to the current GOP are preloaded, and calculation and writing operations are performed on all codec unit lines of B-frame, reference frame REF1, and reference frame REF0 based on the preloaded weight parameters corresponding to the current GOP. If not required, the storage module within the reference weight block is reset, and all codec unit rows of the B frame, the reference frame REF1, and the reference frame REF0 are calculated and written out. If not, then directly jump to the calculation and write-out operation for all codec unit lines of the reference frame REF1 and the reference frame REF0.

14. The video encoding and decoding system according to any one of claims 3-13, wherein, The data preprocessing unit includes a data acquisition module, a reference weight calculation module, a quantization parameter offset calculation module, and a quantization parameter offset result output module, all of which instantiate one of the control state machine cores.

15. An electronic device comprising the video encoding / decoding system as described in any one of claims 1-14.

Images