Video encoding and decoding system and electronic equipment

Abstract

The invention discloses a video encoding and decoding system and electronic equipment. The video coding and decoding system comprises a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit and a second storage unit, the first processing unit is configured to generate first intermediate data based on the original image data in the first storage unit; the second processing unit is configured to generate second intermediate data based on the original data in the first storage unit; the first storage unit is configured to store first intermediate data and second intermediate data; the second storage unit is configured to store control flow data; the data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate a quantization parameter as first output data based on the first intermediate data and the second intermediate data. According to the video coding and decoding system, the Cutree algorithm is realized through hardware logic, the Cutree algorithm is stripped from firmware, the execution efficiency of the Cutree algorithm is improved, and the problems of first frame delay, scheduling, data blocking and the like are solved.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of data processing technology, and more specifically, to a video encoding / decoding system and electronic device. Background Technology

[0002] In current video coding algorithms, the Cutree algorithm is a component of video pre-analysis algorithms that provides block-level quantization parameters (QPs). It analyzes the reference dependencies of pixel blocks within a certain look-ahead range, quantizes and accumulates the importance of pixel blocks in the reference frames, and provides crucial pixel-level QP parameters for the encoder's subsequent mode decision (MD) part. This algorithm effectively reduces the accumulation of errors caused by inter-frame prediction, enhancing the visual effect of the video to the human eye.

[0003] This pre-analysis algorithm, combined with the bitstream control algorithm, can appropriately increase the frame-level quantization parameter (QP) to balance parts of different importance in the video, thereby improving video quality without increasing the final video bitstream data volume. Summary of the Invention

[0004] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] At least one embodiment of this disclosure provides a video encoding and decoding system, including: a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit;

[0006] The first processing unit is configured to generate first intermediate data based on the original image data in the first storage unit;

[0007] The second processing unit is configured to generate second intermediate data based on the original data in the first storage unit;

[0008] The first storage unit is configured to store the first intermediate data and the second intermediate data;

[0009] The second storage unit is configured to store control flow data;

[0010] The data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as the first output data based on the first intermediate data and the second intermediate data.

[0011] At least one embodiment of this disclosure also provides an electronic device, including the video encoding and decoding system provided in any of the above embodiments of this disclosure. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0013] Figure 1 A schematic block diagram of a video encoding and decoding system provided in at least one embodiment of the present disclosure is shown.

[0014] Figure 2A This is a schematic diagram of the structure of a data preprocessing unit provided in at least one embodiment of the present disclosure.

[0015] Figure 2B This is a schematic diagram of the structure of another data preprocessing unit provided in at least one embodiment of the present disclosure.

[0016] Figure 2C This is a schematic diagram of the structure of another data preprocessing unit provided in at least one embodiment of the present disclosure.

[0017] Figure 2D This is a schematic diagram of the structure of another data preprocessing unit provided in at least one embodiment of the present disclosure.

[0018] Figure 3 A schematic diagram illustrating the propagation method of propagation weights is shown.

[0019] Figure 4 This is a schematic diagram of a reference architecture for a look-ahead window provided for at least one embodiment of the present disclosure.

[0020] Figure 5 This disclosure provides a control flowchart for the core of a control state machine, which is one embodiment of the present disclosure.

[0021] Figure 6 This is a schematic diagram of the architecture of a frame set GOP provided for at least one embodiment of the present disclosure.

[0022] Figure 7 A schematic diagram of the storage space of a reference weight block storage module provided in at least one embodiment of the present disclosure is shown.

[0023] Figure 8 This is a control flowchart of another control state machine core provided in at least one embodiment of the present disclosure.

[0024] Figure 9 This is a schematic diagram of two adjacent look-ahead windows provided for at least one embodiment of this disclosure.

[0025] Figure 10 The control flowchart of the core of the control state machine is shown when the look-ahead window has multiple layers of reference relationships.

[0026] Figure 11 A schematic diagram of an electronic device provided in at least one embodiment of the present disclosure is shown.

[0027] Figure 12 A schematic diagram of the specific structure of another electronic device provided in at least one embodiment of the present disclosure is shown. Detailed Implementation

[0028] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0029] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0030] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0032] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0033] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0034] The inventors noted that in current video codec hardware, the Cupree algorithm is implemented using on-chip firmware. While firmware offers greater flexibility, the Cupree algorithm has many data dependencies and uses a large amount of data, which consumes firmware processing time, leading to problems such as data congestion, scheduling blockage, and severe first-frame latency.

[0035] At least one embodiment of this disclosure provides a video encoding and decoding system, including: a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit; the first processing unit is configured to generate first intermediate data based on raw image data in the first storage unit; the second processing unit is configured to generate second intermediate data based on the raw data in the first storage unit; the first storage unit is configured to store the first intermediate data and the second intermediate data; the second storage unit is configured to store control flow data; the data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as first output data based on the first intermediate data and the second intermediate data.

[0036] At least one embodiment of this disclosure also provides an electronic device, including the video encoding and decoding system provided in any of the above embodiments of this disclosure.

[0037] The video encoding and decoding system provided in the above embodiments of this disclosure implements the Cupree algorithm through hardware logic, thereby separating the Cupree algorithm from the firmware. This improves the efficiency of the Cupree algorithm execution, reduces the first frame latency, significantly reduces scheduling and solves problems such as data blocking. Furthermore, it provides the firmware with some data required by the code control algorithm (RC), further reducing the pressure on the firmware and improving the efficiency of the video encoding preprocessing part.

[0038] The embodiments and some examples of this disclosure will now be described in detail with reference to the accompanying drawings.

[0039] Figure 1 A schematic block diagram of a video encoding / decoding system provided in at least one embodiment of this disclosure is shown. For example, such as Figure 1 As shown, Figure 1The upstream and downstream dependencies of a data preprocessing unit (e.g., a Cupree module for implementing the Cupree algorithm) in a video codec hardware architecture are illustrated: the data preprocessing unit provides block-level quantization parameter offsets (QP offsets) for the subsequent video encoder (VE); data preprocessing unit 130 receives data from upstream first processing unit 110 (e.g., an intra processor) and second processing unit 120 (an inter processor), calculates the quantization parameter offset (QP offset) for each pixel block (i.e., the coding unit (CU)), and provides necessary data for the code control algorithm, such as frame-level prediction cost data, to the second storage unit 150 (e.g., a firmware module (FW)). For example, in some examples, data preprocessing unit 130 may not calculate frame-level prediction cost, but can calculate it itself through the firmware; the embodiments of this disclosure do not limit this.

[0040] For example, such as Figure 1 As shown, the video encoding and decoding system 100 includes: a first processing unit 110, a second processing unit 120, a data preprocessing unit 130, a first storage unit 140, and a second storage unit 150.

[0041] For example, the first processing unit 110 is configured to generate first intermediate data based on the original image data in the first storage unit 140. For example, the first processing unit 110 may be an intra-frame processor, and the first intermediate data it generates may be, for example, intra-frame loss data of the intra-frame prediction CU or intra-frame quantization parameters at the initial pixel block level calculated only based on intra-frame prediction, etc. The embodiments of this disclosure are not limited in this regard.

[0042] For example, the second processing unit 120 is configured to generate second intermediate data based on the raw data in the first storage unit 140. For example, the second processing unit 120 can be an inter-frame processor, and the second intermediate data it generates can be, for example, inter-frame loss data of the inter-frame prediction CU or inter-frame motion vector (MV) for the corresponding reference frame, etc. The embodiments of this disclosure are not limited in this regard.

[0043] For example, the first storage unit 140 is configured to store first intermediate data and second intermediate data. For example, the first storage unit 140 can be a dynamic random access memory (DRAM), or other memory that can be used for storage, and the embodiments of this disclosure are not limited thereto.

[0044] For example, the second storage unit 150 is configured to store control flow data. This control flow data, for example, is used in code control algorithms, and can be used to generate frame-level prediction loss. For example, the second storage unit 150 can be firmware.

[0045] For example, the data preprocessing unit 130 is configured to read first intermediate data and second intermediate data from the first storage unit 140, and generate a first output data frame-level prediction loss based on the first intermediate data and the second intermediate data. For example, the data preprocessing unit 130 is used to execute the Cutree algorithm to generate block-level quantization parameters. For example, the first output data may include quantization parameters for output to the first storage unit 140 for subsequent calculations, and the embodiments of this disclosure are not limited thereto.

[0046] For example, in Figure 1 In the video encoding and decoding system shown, the data preprocessing unit 130 for implementing the Cupree algorithm is separated from the second storage unit 150 (i.e., firmware), thereby improving the efficiency of the Cupree algorithm execution, reducing the first frame latency, significantly reducing scheduling and solving data blocking problems, and improving the efficiency of the video encoding preprocessing part.

[0047] For example, in other examples, the data preprocessing unit 130 is also configured to read control flow data from the second storage unit 150, and generate frame-level prediction loss as second output data based on the control flow data and output it to the second storage unit 150.

[0048] For example, in this example, the data preprocessing unit 130 provides the second storage unit (firmware) with the data required by the code control algorithm (RC) (e.g., frame-level prediction loss), thereby further reducing the pressure on the firmware. The firmware can then focus on the code control algorithm and overall scheduling, without performing any actual block-level calculations, significantly reducing the burden on the firmware. This allows for reduced complexity in firmware design, lower development and maintenance difficulty, and the use of smaller firmware in the hardware top-level design. In actual use, the Cutree module in the hardware (i.e., the data preprocessing unit 130) operates for a shorter time than required by the firmware and consumes relatively fewer resources, thus improving overall stability and efficiency.

[0049] Figure 2A This is a schematic diagram of the structure of a data preprocessing unit provided in at least one embodiment of the present disclosure. For example, as shown below... Figure 2AAs shown, the data preprocessing unit 130 includes a control register module 131, at least one control state machine core 132, a data acquisition module 133, a reference weight calculation module 134, a reference weight block storage module 135, a quantization parameter offset calculation module 136, and a quantization parameter offset result output module 137.

[0050] For example, the control register module 131 is connected to the second storage unit 150 and to at least one control state machine core 132 in the data preprocessing unit 130 (not shown in the figures for clarity and simplicity), configured to broadcast information to all control state machine cores 132. This information may include, for example, flow control parameters. It should be noted that the control register module 131 can also be connected to other modules to directly provide them with the data they require; embodiments of this disclosure do not limit this.

[0051] For example, in some examples, all the main modules in the data preprocessing unit 130 (e.g., data acquisition module 133, reference weight calculation module 134, quantization parameter offset calculation module 136, quantization parameter offset result output module 137, and reference weight result output module 138 and frame-level prediction loss calculation module 139 provided in subsequent embodiments) instantiate a single control state machine core 132. This ensures that the processing flow of all main modules is consistent, reduces the number of signal feedbacks between modules, avoids potential problems, and reduces the maintenance difficulty caused by different flow controls for different main modules. For example, the control state machine core 132 is configured to control the flow consistency of each module in the data preprocessing unit 130 according to the flow control parameters.

[0052] For example, in other examples, the data preprocessing unit 130 may include only one control state machine core 132, which transmits control signals along the pipeline to the various modules 133-139 via pipeline delay. This method can reduce the repeated instantiation of the control state machine core 132 and can be used to process smaller data, such as data smaller than 4K. Of course, the embodiments of this disclosure are not limited to this.

[0053] For example, the data acquisition module 133 is configured to send data requests and arrange the returned data in a certain order for distribution to various modules. For example, the returned data could be first intermediate data, second intermediate data, the control flow, or other data required for subsequent calculations by various modules. For example, the data acquisition module 133 balances the data requirements of various modules in the Cutree algorithm. The control state machine core 132 outputs the data requirement states for different processes. Based on the parameters provided by the control register module 131, the position of the corresponding data in the frame, and the base addresses of various data types, the specific address of the data can be calculated for the data acquisition module 133 to read.

[0054] For example, the reference weight calculation module 134 is configured to obtain corresponding pre-data from the data acquisition module 133, quantify the importance of the current codec unit based on the pre-data to obtain weight parameters, calculate the weight parameters using motion vectors, and send them to the reference block of the current codec unit. For example, the pre-data may include the intra-frame and inter-frame prediction loss for comparison, as well as the previously calculated reference weight of the current CU. The reference weight calculation module 134 can quantify the importance of the current CU and transmit it as a weight parameter to the upper-level CU (i.e., the reference block of the current CU) referenced by the current CU. For example, the reference weight calculation module 134 quantifies the importance of the CU by comparing the intra-frame and inter-frame prediction loss with the reference weight of the current CU, and transmits it to the upper-level CU referenced by the current CU using motion vectors.

[0055] Figure 2C This is a schematic diagram of the structure of another data preprocessing unit provided in at least one embodiment of the present disclosure. For example, in other examples, such as... Figure 2C As shown, in Figure 2A Based on this, the data processing unit 130 includes two reference weight calculation modules 134, configured to perform bidirectional propagation to access the reference weight block storage module 135 respectively. For example, one reference weight calculation module 134 processes the calculation and propagation of the reference weight of the left keyframe, and the other reference weight calculation module 134 processes the calculation and propagation of the reference weight of the right keyframe, thereby realizing bidirectional propagation and improving computational efficiency.

[0056] For example, the reference weight block storage module 135 includes storage spaces corresponding to multiple adjacent encoding / decoding units of the reference frame, and is configured to store the weight parameters calculated by motion vectors into the corresponding storage spaces. For example, since the basic operation unit of the Cupree algorithm is a pixel block (CU), and the basic unit of motion vectors is a pixel, the weight data calculated by motion vectors will be accumulated into the adjacent (e.g., 4) CUs in the top, bottom, left, and right of the reference frame. Here, the reference weight block storage module is designed as 4 independent storage spaces corresponding to 4 CUs respectively, thereby realizing the design of a four-part storage format to optimize execution efficiency.

[0057] For example, Figure 3 A schematic diagram illustrating the propagation method of propagation weights is shown. For example, as... Figure 3 As shown, the right side is the reference frame on the left, and each box represents a CU. It can be seen that the referenced block may not completely overlap with a single block on the reference frame; for example, it may be distributed across four adjacent CUs. During the accumulation process, the referenced block needs to be multiplied by a corresponding weight based on the actual area it occupies in these four adjacent CUs. Therefore, each calculation requires reading four data points and writing them back. Thus, in the embodiments of this disclosure, the following approach is adopted... Figure 2A The storage method of propagation weight in the reference weight block storage module 135 shown in the figure divides each CU of a frame into 4 parts according to the parity of its row and column and stores them separately. In this way, the 4 adjacent CUs must exist in these 4 storage areas respectively without overlapping. Therefore, it can be implemented by reading once, thereby reducing the complexity of internal data reading.

[0058] For example, in Figure 2A In the example shown, the data preprocessing unit does not include a reference weight result output module, thereby expanding the storage space within the data processing unit 130 (e.g., the storage space of the reference weight block storage module 135 can be expanded to q frames, where q is an integer greater than or equal to 1). All propagation weights are directly cached in the reference weight block storage module 135, which reduces the complexity of control flow design and lowers the overall bandwidth. For example, this example can be used when the data volume is small. For example, in this example, all calculation results can be stored in the reference weight block storage module 135, and the data stored in the reference weight block storage module 135 can be immediately used for subsequent calculations through the control of the control state machine core 132, thus reusing the internal storage.

[0059] Figure 2B This is a schematic diagram of the structure of another data preprocessing unit provided in at least one embodiment of the present disclosure. For example, in other examples, such as... Figure 2B As shown, in Figure 2ABased on this, the data processing unit 130 also includes a reference weight result output module 138, configured to output the calculation results of the weight parameters stored in the reference weight block storage module to the first storage unit 140. For example, based on the control signal given by the control state machine core 132, it is determined whether there is a need to output the calculated reference weight data to the first storage unit 140. If so, the corresponding data is read from the reference weight block storage module 138 and written to the first storage unit 140. For example, the calculation result of the weight parameters is the final reference weight of each block in a certain frame obtained after storage accumulation.

[0060] For example, in this example, the data in the reference weight block storage module 135 can be transmitted to the first storage unit 140 for storage through the reference weight result output module 138. When needed, it can be read from the first storage unit 140, thereby freeing up the space of the reference weight block storage module 135. For example, in this example, the storage space of the reference weight block storage module 135 can be reduced to the space for storing multiple rows in one frame (e.g., k rows, where k is an integer greater than 0), thereby reducing the internal storage space.

[0061] For example, the quantization parameter offset calculation module 136 is configured to obtain quantization parameters based on the final weight parameters. For example, these quantization parameters include a quantization parameter offset. For example, the degree to which each CU in the keyframe is referenced in subsequent frames is obtained based on the calculated reference weights; for example, a higher degree results in a lower QP, thus leading to better video quality. For example, the control signals provided by the control state machine core 132, at the keyframe, provide more accurate block-level QP data for subsequent video encoding based on the previously calculated reference weights.

[0062] For example, the quantization parameter offset result output module 137 is configured to determine whether the quantization parameter needs to be output, calculate the address offset corresponding to the quantization parameter, and output it to the first storage unit. For example, based on the control signal of the control state machine core 132, it determines whether the corresponding QP offset result needs to be output, calculates the output address offset, and passes it to the first storage unit 140.

[0063] Figure 2D This is a schematic diagram of the structure of another data preprocessing unit provided in at least one embodiment of the present disclosure. For example, in other examples, such as... Figure 2D As shown, in Figure 2BBased on this, the data processing unit 130 further includes a frame-level prediction loss calculation module 139, configured to calculate the frame-level prediction loss as the second output data based on the quantization parameter offset and the control flow data. For example, the frame-level prediction loss calculation module 139 obtains the QP offset calculated by the quantization parameter offset calculation module 136, and recalculates the frame-level prediction loss based on the QP offset, providing key data for the code control algorithm in the second storage unit 150. This frame-level prediction loss is transmitted back to the second storage unit 150 through the control register module 131.

[0064] For example, in this example, the frame-level prediction loss calculation module 139 in the data preprocessing unit 130 provides the second storage unit (firmware) with the data required by the code control algorithm (RC) (e.g., frame-level prediction loss), thereby further reducing the pressure on the firmware. The firmware can then focus on the code control algorithm and overall scheduling, without performing any actual calculations at the block level, significantly reducing the burden on the firmware. This allows for reduced complexity in firmware design, lower development and maintenance difficulty, and the use of smaller firmware in the hardware top-level design. In actual use, the Cutree module (i.e., the data preprocessing unit 130) in the hardware operates for a shorter time than the firmware requires and consumes relatively fewer resources, thus improving overall stability and efficiency.

[0065] For example, the second storage unit 150 is further configured to configure a look-ahead window for the control state machine core, wherein the look-ahead window includes multiple group of frames (GOPs). For example, the second storage unit 150 describes a reference architecture for the look-ahead window and sets a fast mode for the control state machine core based on the reference architecture to reduce the computation of duplicate frame sets in adjacent look-ahead windows. For example, the reference architecture of the look-ahead window determines duplicate GOPs in two adjacent look-ahead windows, and the fast mode is set so that the relevant data of the duplicate GOP can be directly called from the previous look-ahead window to reduce the computation of duplicate frame sets in adjacent look-ahead windows. For example, the second storage unit 150 is configured to describe the reference architecture and fast mode of the look-ahead window and send them to the control state machine core via the control register module.

[0066] Figure 4 This is a schematic diagram of a reference architecture for a look-ahead window provided for at least one embodiment of the present disclosure.

[0067] For example, each execution of the Cupree algorithm operates on a lookahead window basis, and an execution traverses the entire lookahead window. A lookahead window consists of an integer number of frames (Group of Pictures, or GOPs). For example, Figure 4Frames 0-8 in the image form a Group of Pictures (GOP). To improve flexibility, the propagation process within each GOP can be configured individually, including the start and end points, the target frame for each B-frame propagation, and the propagation order. This configuration is transmitted as control data from the second storage unit 150 to the control state machine core 132 via the control register module 131. If there is no need for flexibility in configuring GOPs within a look-ahead window, each GOP can be set to be identical, reducing the need for GOP configuration. This process can be configured through the second storage unit (i.e., firmware), improving flexibility.

[0068] For example, such as Figure 4 As shown, the current lookahead window length for Cupreel processing is 0-38, and the keyframes are frames 0, 8, 16, 32, and 38. For example, as... Figure 4 As shown, the transmission order (i.e., GOP configuration) can be described in terms of importance as 084213657... Figure 4 This is merely an example and is not intended to be limiting.

[0069] For example, due to the existence of lookahead windows, there are repetitive parts in different Cutree processing flows. Fast mode exists to reduce repetitive processes (e.g., reducing the processing and calculation of duplicate GOPs, which can be directly called) and improve efficiency at the cost of bandwidth and memory space. If bandwidth is insufficient or increased complexity is undesirable, this register configuration can be omitted. For example, if frames 8-38 have already been calculated when processing frames 0-38, the GOP values ​​for the next lookahead window (8-46) can be directly called when calculating the next lookahead window, avoiding redundant calculations and thus achieving fast mode.

[0070] Figure 5 This disclosure provides a control flowchart for the core of a control state machine, which is one embodiment of the present disclosure.

[0071] exist Figure 5 In the process shown, the following settings can be made:

[0072] (1) A lookout window contains several such... Figure 6 The reference architecture shown is exactly the same GOP. A GOP can be defined as a combination of all frames between two non-B frames, including the two non-B frames, with the left one called REF0 and the right one called REF1.

[0073] (2) In Figure 6 In the reference architecture shown, all B-frames are unidirectionally referenced to REF0 or REF1, or bidirectionally propagated to both REF0 and REF1;

[0074] (3) The storage space of the reference weight block storage module 135 can cache all the reference weights of REF0 and REF1 frames;

[0075] (4) Do not use the fast mode, that is, repeated operations between different processing flows are not skipped, but recalculated.

[0076] Figure 6 This is a schematic diagram of the architecture of a frame set (GOP) provided for at least one embodiment of this disclosure. Figure 6 As shown, B-frames are bidirectional reference frames, P-frames are unidirectional reference frames, and I-frames are completely independent frames. Figure 6 As shown, the area within the dashed box represents a Group of Pictures (GOP). Within a GOP, the frame on the left is used as reference frame REF0, and the frame on the right is used as reference frame REF1. For example, each bidirectional B-frame in the middle references either REF0 or REF1, while REF1 only references REF0. Therefore, REF0 and REF1 switch between two GOPs. Figure 4 The REF1 of the GOP within the dashed line can be used as the REF0 of the next GOP. Therefore, during execution, the following GOP is executed first, followed by the preceding GOP, and the data of the REF0 of the following GOP can be directly switched and used as the REF1 of the previous GOP.

[0077] Regarding the above settings, for example, abstracting a single process of Cupreel into something like... Figure 6 The architecture shown follows a processing order that satisfies the following rules:

[0078] (1) GOPs within the lookahead window are processed from back to front;

[0079] (2) Within a GOP, the B-frames are processed first, followed by REF1. REF0 is processed until the leftmost GOP in the look-ahead window. In this processing order, the REF0 of the current GOP is the REF1 of the next GOP.

[0080] (3) The frame is processed line by line in the order of the CU;

[0081] (4) After the calculation of a frame is completed, output the propagation weight parameters or QP offset. Output the QP offset for the leftmost GOP and output the propagation weight parameters for the other GOPs.

[0082] For example, in Figure 5 As shown in the above settings, the control state machine core can be configured to perform the following control flow:

[0083] First, the storage module within the reference weight block is reset, for example, the internal storage is refreshed to a state of 0. Then, after calculating and writing the weight parameters propagating from the B-frame, it is determined whether all B-frames have been processed. If not, the process switches to the unprocessed B-frame (i.e., the next B-frame) to continue calculation and writing. If all B-frames have been processed, the calculation and writing of the weight parameters propagating from the reference frame REF1 is performed again. After REF1 is completed, it is determined whether the current GOP containing REF1 is the last GOP. If not, the above operations are performed on the next GOP. If so, the weight parameters of the reference frame REF0 in the last GOP in the lookahead window are calculated and written. It is important to note that the above operations are performed from back to front. For example, corresponding to... Figure 4 The middle structure starts from the GOP corresponding to frame 38 and proceeds backward to frame 0 to complete the calculation of a look-ahead window, and so on.

[0084] For example, in some examples, the storage space of the reference weight block storage module includes the radiation range of the inter-frame reference, for example, the radiation range is... Figure 7 The k rows shown (where k is an integer greater than 0) can be determined according to the actual situation, and the embodiments disclosed herein do not impose any restrictions on this. Figure 7 A schematic diagram of the storage space of a reference weight block in-memory module 135 provided in at least one embodiment of this disclosure is shown. For example, as Figure 7 As shown, considering the increase in video resolution, the storage space of the reference weight block storage module within the Cutree will increase exponentially with the increase in the number of pixel rows m (m is a positive integer). 2 The increase in level leads to a significant increase in cost. To overcome the increase in storage space, in the embodiments of this disclosure, the height of the storage space for propagating weights (weight parameters) within the Cupre is fixed, while the width still increases with the number of pixel rows m. The height is the vertical radiation range of the motion vector (i.e., the radiation range k of the inter-frame reference), as shown below. Figure 7 The diagram shows the influence range of the CU row of the current frame on the referenced frame to its right. For example, after the first row of the referenced frame is no longer affected by the CU row of the current frame, its weight parameters can be output.

[0085] In this example, by adding a limit on storage space, the control granularity of the control state machine core can be increased from... Figure 5 The frame-level control shown is transformed into control of each CU row, thus making the calculation more accurate. The specific control flow is as follows: Figure 8 As shown.

[0086] Figure 8 This is a control flowchart for another control state machine core provided in at least one embodiment of the present disclosure. For example, as... Figure 8 As shown, the control state machine core can be configured to perform the following control flow:

[0087] First, reset the storage module within the reference weight block, for example, refresh the internal storage to a state of 0. Then, after calculating and writing the weight parameters of the current line codec unit (CU) propagating from the B-frame, it is determined whether all lines in all B-frames have been processed. If not, it switches to the next line codec unit of the B-frame to continue the calculation and writing operation. If yes, it calculates and writes the weight parameters of the current line codec unit propagating from the reference frame REF1, and determines whether all lines in the reference frame REF1 have been processed. If not, it switches to the next line codec unit of the reference frame REF1 to continue the calculation and writing operation. If yes, it determines whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window. If not, it continues to perform calculation and writing operations on the next GOP in the lookahead window. If yes, it calculates and writes the weight parameters of the current line codec unit of the reference frame REF0 in the last GOP, and determines whether all lines in the reference frame REF0 have been processed. If not, it switches to the next line codec unit of the reference frame REF0 to continue the calculation and writing operation. If yes, the calculation ends.

[0088] Figure 9 This is a schematic diagram of two adjacent look-ahead windows provided for at least one embodiment of this disclosure. For example, as... Figure 9 As shown in the dashed box, the overlapping parts of two adjacent look-ahead windows are included. Therefore, in the subsequent processing of the second look-ahead window, a fast mode can be used to omit the calculation of the overlapping parts, and the calculation results of the first look-ahead window can be directly read, thereby improving the efficiency of data processing.

[0089] Figure 10 This shows that the look-ahead window has multiple layers of reference relationships (e.g., Figure 4 The control flowchart of the core control state machine at the time shown is for the first GOP hierarchy (084213657). For example, in... Figure 10 In the process shown, it can be Figure 8 A fast mode is added to the process shown.

[0090] For example, refer to Figure 10 The control state machine core can be configured to perform the following control flow:

[0091] First, the frame set information of the current GOP is obtained from the second storage unit 150, and it is determined whether the current GOP is a new GOP architecture based on the frame set information. That is, if it is not a new GOP, its corresponding weight parameters can be directly called from the first storage unit 140, and the B-frame judgment calculation is no longer performed. If it is a new GOP, the following judgment is performed.

[0092] For example, the frame set information includes a configurable reference architecture for the GOP, such as... Figure 4 , Figure 6 or Figure 9 The architecture is as described above, and the specific architecture may vary depending on the actual situation. This disclosure does not impose any restrictions on it. For example, the reference architecture information includes parameters such as the source index, the destination index, the reference layer of the source frame, and whether the propagation is the last segment of the GOP (Ending flag).

[0093] If it's a new GOP architecture, the complexity of the current GOP is calculated, and the processing feature information of the current GOP is invoked based on the complexity. This processing feature information includes, for example, the underlying features corresponding to the current GOP, such as whether weight parameters have already been calculated. Based on the processing feature information of the current GOP, it is determined whether the weight parameters corresponding to the current GOP need to be preloaded. If the weight parameters have already been calculated, they can be preloaded, and calculation and write operations are performed on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 based on the preloaded weight parameters. If not, the reference weight block storage module is reset, and the above calculation and write operations on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 are performed. For example, the calculation and write operations on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 can be found in [link to documentation]. Figure 8 The specific description in the code will not be repeated here. Calculate the weight parameters of the current CU line propagating from the B-frame.

[0094] The video encoding and decoding system provided in the above embodiments of this disclosure achieves the following: by heterogenizing the Cupree algorithm and converting it into RTL parallel digital logic; by instantiating a control state machine core in each major module in the core state machine design of Cupree; by using pixel block row switching to save storage space in each module; by using a fast mode control flow design to improve efficiency by reducing repetitive operations; and by using the customizable control logic design in the propagation architecture within the GOP and the module for accumulating Cupree propagation weights to design a four-part storage format, the Cupree algorithm can be implemented through hardware logic, separating the Cupree algorithm from the firmware, improving the efficiency of Cupree algorithm execution, reducing first frame latency, significantly reducing scheduling and solving data blocking problems, and also providing some data required by the code control algorithm (RC) for the firmware, further reducing the firmware pressure and improving the efficiency of the video encoding preprocessing part.

[0095] At least one embodiment of this disclosure also provides an electronic device, including the video encoding and decoding system provided in any of the above embodiments of this disclosure.

[0096] Figure 11 A schematic diagram of an electronic device provided by at least one embodiment of the present disclosure is shown. For example, such as Figure 11 As shown, the electronic device 200 includes a video encoding / decoding system 100. For example, the video encoding / decoding system 100 may employ... Figure 1The structure shown is implemented as described. For example, the electronic device 200 can be any electronic device with computing capabilities, such as a mobile phone, digital camera, laptop, tablet, desktop computer, network server, etc., which can load and execute the video processing method. The embodiments of this disclosure do not limit this. For example, the electronic device may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, storage units, etc. The electronic device is also equipped with an operating system, application programming interfaces (e.g., OpenGL (Open Graphics Library), Metal, etc.), etc., and implements video encoding and decoding by running code or instructions. For example, the electronic device may also include an output component, such as a display component, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a quantum dot light-emitting diode (QLED) display, etc. The embodiments of this disclosure do not limit this.

[0097] It should be noted that, for clarity and brevity, this disclosure does not show all the constituent units of the electronic device 200. To achieve the necessary functions of the electronic device 200, those skilled in the art can provide and set other constituent units (not shown) according to specific needs, and this disclosure does not limit this.

[0098] The following is for reference. Figure 12 This document illustrates a specific structural diagram of an electronic device (e.g., a terminal device or server) 600 suitable for implementing a video encoding / decoding system according to embodiments of the present disclosure. The terminal device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 12 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0099] like Figure 12As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 606 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0100] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 606 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 12 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0101] Specifically, according to embodiments of this disclosure, the processes described above in the reference flowchart of the control state machine core or the video encoding / decoding method can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 606, or installed from ROM 602. When the computer program is executed by processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0102] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0103] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0104] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0105] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire at least two Internet Protocol (IP) addresses; send a node evaluation request including the at least two IP addresses to a node evaluation device, wherein the node evaluation device selects an IP address from the at least two IP addresses and returns it; and receive the IP address returned by the node evaluation device; wherein the acquired IP address indicates an edge node in a content delivery network.

[0106] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a node evaluation request including at least two Internet Protocol (IP) addresses; select an IP address from the at least two IP addresses; and return the selected IP address; wherein the received IP address indicates an edge node in the content delivery network.

[0107] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0110] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0111] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0112] According to one or more embodiments of this disclosure, Example 1 provides a video encoding / decoding system, including: a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit; wherein,

[0113] The first processing unit is configured to generate first intermediate data based on the original image data in the first storage unit;

[0114] The second processing unit is configured to generate second intermediate data based on the original data in the first storage unit;

[0115] The first storage unit is configured to store the first intermediate data and the second intermediate data;

[0116] The second storage unit is configured to store control flow data;

[0117] The data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as the first output data based on the first intermediate data and the second intermediate data.

[0118] According to one or more embodiments of this disclosure, Example 2 provides a video encoding and decoding system of Example 1, wherein the data preprocessing unit is further configured to read the control flow data from the second storage unit, and generate frame-level prediction loss as second output data based on the control flow data and output it to the second storage unit.

[0119] According to one or more embodiments of this disclosure, Example 3 provides the video encoding and decoding system of Example 1, wherein the data preprocessing unit includes a control register module, at least one control state machine core, a data acquisition module, a reference weight calculation module, a reference weight block storage module, a quantization parameter offset calculation module, and a quantization parameter offset result output module; wherein,

[0120] The control register module is connected to the second storage unit and is configured to broadcast information to the at least one control state machine core, wherein the information includes process control parameters;

[0121] The at least one control state machine core is configured to control the processing flow of each module in the data preprocessing unit to be consistent according to the process control parameters.

[0122] The data acquisition module is configured to send a data request and arrange the returned data.

[0123] The reference weight calculation module is configured to obtain corresponding pre-data from the data acquisition module, quantify the importance of the current codec unit based on the pre-data to obtain weight parameters, calculate the weight parameters through motion vectors and send them to the reference block of the current codec unit. The pre-data includes intra-frame and inter-frame prediction loss and the reference weight of the current codec unit.

[0124] The reference weight block storage module includes storage spaces corresponding to multiple adjacent codec units of the reference frame, and is configured to store the weight parameters calculated by motion vectors into the corresponding storage spaces.

[0125] The quantization parameter offset calculation module is configured to obtain quantization parameters based on the weight parameters, wherein the quantization parameters include quantization parameter offsets;

[0126] The quantization parameter offset result output module is configured to determine whether the quantization parameter needs to be output, and to calculate the address offset corresponding to the quantization parameter and output it to the first storage unit.

[0127] According to one or more embodiments of this disclosure, Example 4 provides a video encoding / decoding system of Example 3, wherein the storage space includes the radiation range of an inter-frame reference.

[0128] According to one or more embodiments of this disclosure, Example 5 provides a video encoding and decoding system of Example 3, wherein the data preprocessing unit includes two reference weight calculation modules, and the two reference weight calculation modules are further configured to perform bidirectional propagation to access the reference weight block storage module respectively.

[0129] According to one or more embodiments of this disclosure, Example 6 provides a video encoding and decoding system of Example 3, wherein the data preprocessing unit further includes a reference weight result output module configured to output the calculation result of the weight parameters stored in the reference weight block storage module to the first storage unit.

[0130] According to one or more embodiments of this disclosure, Example 7 provides a video encoding and decoding system of Example 3, wherein the data preprocessing unit further includes a frame-level prediction loss calculation module configured to calculate frame-level prediction loss as second output data based on the quantization offset and the control flow data.

[0131] According to one or more embodiments of this disclosure, Example 8 provides the video encoding and decoding system of Example 7, wherein the frame-level prediction loss is returned to the second storage unit via the control register module.

[0132] According to one or more embodiments of this disclosure, Example 9 provides a video encoding / decoding system of Example 3, wherein the second storage unit is further configured to configure a look-ahead window required by the control state machine core, wherein the look-ahead window includes a plurality of frame sets (GOPs).

[0133] According to one or more embodiments of this disclosure, Example 10 provides a video encoding / decoding system of Example 9, wherein the second storage unit is further configured to describe a reference architecture of the look-ahead window and set a fast mode for the control state machine core based on the reference architecture of the look-ahead window to reduce the computation of duplicate frame sets in adjacent look-ahead windows.

[0134] According to one or more embodiments of this disclosure, Example 11 provides the video encoding / decoding system of Example 9, wherein the control state machine core is further configured as follows:

[0135] Reset the storage module within the reference weight block;

[0136] After calculating and writing out the weight parameters propagating from the B-frames, determine whether all B-frames have been processed.

[0137] If not, switch to the next B-frame to continue the aforementioned calculation and write operations;

[0138] If so, then the calculation and write operations are performed on the weight parameters propagated from the reference frame REF1, and it is determined whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window.

[0139] If not, continue performing the computation and write operations on the next GOP in the lookout window.

[0140] If so, then the calculation and write operation are performed on the weight parameters of the reference frame REF0 in the last GOP.

[0141] According to one or more embodiments of this disclosure, Example 12 provides the video encoding / decoding system of Example 9, wherein the control state machine core is further configured as follows:

[0142] Reset the storage module within the reference weight block;

[0143] After calculating and writing out the weight parameters of the current codec unit line propagating from the B-frame, it is determined whether all lines in all B-frames have been processed.

[0144] If not, switch to the next codec unit line of the B frame to continue the calculation and write operations;

[0145] If so, then perform the aforementioned calculation and write operation on the weight parameters of the current codec unit line propagating from reference frame REF1, and determine whether all lines of reference frame REF1 have been processed.

[0146] If not, then switch to the next encoding / decoding unit line of the reference frame REF1 to continue the calculation and write operation;

[0147] If so, determine whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window.

[0148] If not, continue performing the computation and write operations on the next GOP in the lookout window.

[0149] If so, then the calculation and write operation are performed on the weight parameters of the current codec unit line of the reference frame REF0 in the last GOP, and it is determined whether all lines of the reference frame REF0 have been processed.

[0150] If not, then switch to the next encoding / decoding unit line of the reference frame REF0 to continue the calculation and write operations.

[0151] According to one or more embodiments of this disclosure, Example 13 provides a video encoding / decoding system of Example 9, wherein the control state machine core is further configured as follows:

[0152] Obtain the frame set information of the current GOP from the second storage unit, and determine whether the architecture of the current GOP is a new GOP architecture based on the frame set information;

[0153] If so, calculate the complexity of the current GOP, call the processing feature information of the current GOP based on the complexity of the current GOP, and determine whether the weight parameters corresponding to the current GOP need to be preloaded based on the processing feature information of the current GOP.

[0154] If necessary, the weight parameters corresponding to the current GOP are preloaded, and calculation and writing operations are performed on all codec unit lines of the B-frame, reference frame REF1, and reference frame REF0 based on the preloaded weight parameters corresponding to the current GOP; if not necessary, the storage module within the reference weight block is reset, and calculation and writing operations are performed on all codec unit lines of the B-frame, the reference frame REF1, and the reference frame REF0.

[0155] If not, then directly jump to the calculation and write-out operation for all codec unit lines of the reference frame REF1 and the reference frame REF0.

[0156] According to one or more embodiments of this disclosure, Example 14 provides a video encoding / decoding system as described in any of Examples 3-13, wherein each module of the data preprocessing unit instantiates one of the control state machine cores.

[0157] According to one or more embodiments of this disclosure, Example 15 provides an electronic device including the video encoding / decoding system provided in Examples 1-14.

[0158] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0159] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0160] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A video encoding and decoding system, comprising: The system comprises a first processing unit, a second processing unit, a data preprocessing unit, a first storage unit, and a second storage unit; wherein... The first processing unit is configured to generate first intermediate data based on the original image data in the first storage unit; The second processing unit is configured to generate second intermediate data based on the original data in the first storage unit; The first storage unit is configured to store the first intermediate data and the second intermediate data; The second storage unit is configured to store control flow data; The data preprocessing unit is configured to read the first intermediate data and the second intermediate data from the first storage unit, and generate quantization parameters as the first output data based on the first intermediate data and the second intermediate data.

2. The video encoding and decoding system according to claim 1, wherein, The data preprocessing unit is further configured to read the control flow data from the second storage unit, and generate frame-level prediction loss as second output data based on the control flow data and output it to the second storage unit.

3. The video encoding and decoding system according to claim 1, wherein, The data preprocessing unit includes a control register module, at least one control state machine core, a data acquisition module, a reference weight calculation module, a reference weight block storage module, a quantization parameter offset calculation module, and a quantization parameter offset result output module; wherein... The control register module is connected to the second storage unit and is configured to broadcast information to the at least one control state machine core, wherein the information includes process control parameters; The at least one control state machine core is configured to control the processing flow of each module in the data preprocessing unit to be consistent according to the process control parameters. The data acquisition module is configured to send a data request and arrange the returned data. The reference weight calculation module is configured to obtain corresponding pre-data from the data acquisition module, quantify the importance of the current codec unit based on the pre-data to obtain weight parameters, and transmit the weight parameters to the reference block of the current codec unit through motion vectors. The pre-data includes intra-frame and inter-frame prediction loss and the reference weight of the current codec unit. The reference weight block storage module includes storage spaces corresponding to multiple adjacent codec units of the reference frame, and is configured to store the weight parameters transmitted through motion vectors into the corresponding storage spaces. The quantization parameter offset calculation module is configured to obtain quantization parameters based on the weight parameters passed by the motion vector, wherein the quantization parameters include quantization parameter offsets. The quantization parameter offset result output module is configured to determine whether the quantization parameter needs to be output, and to calculate the address offset corresponding to the quantization parameter and output it to the first storage unit.

4. The video encoding and decoding system according to claim 3, wherein, The storage space includes the radiation range of the inter-frame reference.

5. The video encoding and decoding system according to claim 3, wherein, The data preprocessing unit includes two reference weight calculation modules, which are further configured to perform bidirectional propagation to access the storage modules within the reference weight blocks respectively.

6. The video encoding and decoding system according to claim 3, wherein, The data preprocessing unit further includes a reference weight result output module, configured to output the calculation results of the weight parameters stored in the reference weight block storage module to the first storage unit.

7. The video encoding and decoding system according to claim 3, wherein, The data preprocessing unit further includes a frame-level prediction loss calculation module, configured to calculate the frame-level prediction loss as the second output data based on the quantization offset and the control flow data.

8. The video encoding and decoding system according to claim 7, wherein, The frame-level prediction loss is returned to the second storage unit through the control register module.

9. The video encoding and decoding system according to claim 3, wherein, The second storage unit is also configured to configure a look-ahead window for the control state machine core, wherein the look-ahead window includes multiple frame sets (GOPs).

10. The video encoding and decoding system according to claim 9, wherein, The second storage unit is also configured to describe the reference architecture of the look-ahead window and set a fast mode for the control state machine core based on the reference architecture of the look-ahead window to reduce the computation of duplicate frame sets in adjacent look-ahead windows.

11. The video encoding and decoding system according to claim 9, wherein, The core of the control state machine is also configured as follows: Reset the storage module within the reference weight block; After calculating and writing out the weight parameters propagating from the B-frames, determine whether all B-frames have been processed. If not, switch to the next B-frame to continue the aforementioned calculation and write operations; If so, then the calculation and write operations are performed on the weight parameters propagated from the reference frame REF1, and it is determined whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window. If not, continue performing the computation and write operations on the next GOP in the lookout window. If so, then the calculation and write operation are performed on the weight parameters of the reference frame REF0 in the last GOP.

12. The video encoding and decoding system according to claim 9, wherein, The core of the control state machine is also configured as follows: Reset the storage module within the reference weight block; After calculating and writing out the weight parameters of the current codec unit line propagating from the B-frame, it is determined whether all lines in all B-frames have been processed. If not, switch to the next codec unit line of the B frame to continue the calculation and write operations; If so, then perform the aforementioned calculation and write operation on the weight parameters of the current codec unit line propagating from reference frame REF1, and determine whether all lines of reference frame REF1 have been processed. If not, then switch to the next encoding / decoding unit line of the reference frame REF1 to continue the calculation and write operation; If so, determine whether the current GOP containing the reference frame REF1 is the last GOP in the lookahead window. If not, continue performing the computation and write operations on the next GOP in the lookout window. If so, then the calculation and write operation are performed on the weight parameters of the current codec unit line of the reference frame REF0 in the last GOP, and it is determined whether all lines of the reference frame REF0 have been processed. If not, then switch to the next encoding / decoding unit line of the reference frame REF0 to continue the calculation and write operations.

13. The video encoding and decoding system according to claim 9, wherein, The core of the control state machine is also configured as follows: Obtain the frame set information of the current GOP from the second storage unit, and determine whether the architecture of the current GOP is a new GOP architecture based on the frame set information; If so, calculate the complexity of the current GOP, call the processing feature information of the current GOP based on the complexity of the current GOP, and determine whether the weight parameters corresponding to the current GOP need to be preloaded based on the processing feature information of the current GOP. If necessary, the weight parameters corresponding to the current GOP are preloaded, and calculation and writing operations are performed on all codec unit lines of B-frame, reference frame REF1, and reference frame REF0 based on the preloaded weight parameters corresponding to the current GOP. If not required, the storage module within the reference weight block is reset, and all codec unit rows of the B frame, the reference frame REF1, and the reference frame REF0 are calculated and written out. If not, then directly jump to the calculation and write-out operation for all codec unit lines of the reference frame REF1 and the reference frame REF0.

14. The video encoding and decoding system according to any one of claims 3-13, wherein, The data preprocessing unit includes a data acquisition module, a reference weight calculation module, a quantization parameter offset calculation module, and a quantization parameter offset result output module, all of which instantiate one of the control state machine cores.

15. An electronic device comprising the video encoding / decoding system as described in any one of claims 1-14.

Images