Image frame encoding method, object search method, computer device, storage medium

Abstract

Embodiments of the present application disclose an image frame encoding method, an object search method, a computer device and a storage medium. The image frame encoding method comprises the following steps: dividing a target image frame into a plurality of first data blocks; inputting the plurality of first data blocks into a plurality of encoding processing units which are executed in sequence in batches; synchronously executing processing of at least two encoding processing units in a partial time period in which the plurality of encoding processing units execute processing; and encoding the target image frame according to encoding parameters corresponding to the plurality of first data blocks respectively. Compared with a mode of sequentially executing in units of video frames, the time consumed for determining encoding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of data blocks is reduced. Since the mode of parallel operation of multiple encoding units reuses resources of hardware units, the chip area can be reduced, the hardware cost can be reduced, and hardware implementation is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to an image frame encoding method, an image search method, an object search method, a computer device and a computer readable storage medium. BACKGROUND

[0002] The current video technology develops rapidly and is widely applied. When a video is transmitted over a network, the video image is encoded for various video application scenarios based on bandwidth limitation and traffic saving purposes.

[0003] Video encoding refers to converting a file in an original video format into another video format file through discrete cosine transform, quantization and entropy encoding of video image frames. The purpose is to improve the compression rate of the video as much as possible and reduce the size of the video file under the premise of ensuring the encoding quality.

[0004] Therefore, video pre-analysis plays an important role in selecting appropriate methods to improve video quality and control video compression rate. SUMMARY

[0005] In view of the above problems, the present application is proposed to provide an image processing method, an image search method, an object search method, a computer device and a computer readable storage medium which overcome the above problems or at least partially solve the above problems.

[0006] According to an aspect of the present application, an image frame encoding method is provided, comprising:

[0007] dividing a target image frame into a plurality of first data blocks;

[0008] batching the plurality of first data blocks into a plurality of encoding processing units which are executed in sequence; after the encoding processing units complete the processing of a previous batch of first data blocks, the processing of a next batch of first data blocks is executed; the encoding processing units correspond to one or more steps in the determination process of the encoding parameters of the target image frame; at least two encoding processing units synchronously execute processing within a partial time period in which the plurality of encoding processing units execute processing;

[0009] encoding the target image frame according to the encoding parameters corresponding to the plurality of first data blocks.

[0010] According to an aspect of the present application, a video encoding method is provided, comprising:

[0011] dividing a target image frame in a video into a plurality of first data blocks;

[0012] inputting the first data blocks into a plurality of encoding processing units in batches in sequence; the encoding processing units perform processing of a next batch of first data blocks after processing of a previous batch of first data blocks, the encoding processing units correspondingly perform one or more steps in a process of determining encoding parameters of the image frame, at least two of the encoding processing units perform processing in synchronization within a partial time period in which the plurality of encoding processing units perform processing;

[0013] encoding the target image frame according to the encoding parameters corresponding to the first data blocks;

[0014] obtaining an encoding result of the video according to the encoding result of the image frame.

[0015] According to an aspect of the present application, a video encoding method is provided, comprising:

[0016] dividing a specific region of a target image frame in a video into a plurality of first data blocks, the target region being a region in which content of the target image frame changes relative to a reference image frame;

[0017] inputting the first data blocks into four encoding processing units in batches in sequence; the encoding processing units perform processing of a next batch of first data blocks after processing of a previous batch of first data blocks, the four encoding processing units respectively perform a quantization parameter calculation step in a process of determining encoding parameters of the image frame, the quantization parameter calculation step comprises a reading step, an information quantity obtaining step, a quantization parameter and information quantity calculation step, and a quantization parameter and information quantity writing step, at least two of the four encoding processing units perform processing in synchronization within a partial time period in which the four encoding processing units perform processing;

[0018] encoding the target image frame according to the encoding parameters corresponding to the first data blocks;

[0019] obtaining an encoding result of the video according to the encoding result of the image frame.

[0020] According to an aspect of the present application, a video encoding method is provided, comprising:

[0021] dividing a specific region of a target image frame in a video into a plurality of first data blocks, the target region being a region in which content of the target image frame changes relative to a reference image frame;

[0022] inputting the first data blocks into a plurality of encoding processing units in batches in sequence; the encoding processing units perform processing of a next batch of first data blocks after processing of a previous batch of first data blocks, the encoding processing units correspondingly perform one or more steps in a process of determining encoding parameters of the image frame, at least two of the encoding processing units perform processing in synchronization within a partial time period in which the plurality of encoding processing units perform processing.

[0023] According to an aspect of the present application, a method for processing image frames is provided, which is applied to a software service platform, comprising:

[0024] obtaining at least one target video submitted by a software service client;

[0025] dividing image frames of the target video into a plurality of first data blocks;

[0026] inputting the plurality of first data blocks into a plurality of encoding processing units in batches in sequence; after the encoding processing units complete processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed, the encoding processing units correspondingly perform one or more steps in a determination process of encoding parameters of the target image frames, and at least two encoding processing units synchronously perform processing in a partial time period in which the plurality of encoding processing units perform processing;

[0027] encoding the target image frames according to the encoding parameters corresponding to the plurality of first data blocks respectively;

[0028] obtaining an encoding result of the target video according to encoding results of the plurality of image frames;

[0029] providing the encoding result of the target video based on the software service client.

[0030] According to an aspect of the present application, an image frame encoding system is provided, comprising a frame division module, a plurality of encoding processing units executed in sequence, and an encoding module;

[0031] The frame division unit is configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches.

[0032] The encoding processing unit is configured to correspondingly perform one or more steps in a determination process of encoding parameters of the target image frames, and after completing processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed, and at least two encoding processing units synchronously perform processing in a partial time period in which the plurality of encoding processing units perform processing.

[0033] The encoding unit is configured to encode the target image frames according to the encoding parameters corresponding to the plurality of first data blocks respectively.

[0034] According to an aspect of the present application, a chip based on a field programmable logic gate array is provided, comprising a frame division module, a plurality of encoding processing units executed in sequence, and an encoding module;

[0035] The frame division unit is configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches.

[0036] The encoding processing units are configured to perform one or more steps in a determination process of encoding parameters corresponding to the target image frame, and perform processing of a next batch of first data blocks after processing of a previous batch of first data blocks is completed, and at least two of the encoding processing units perform processing synchronously in a partial time period in which the multiple encoding processing units perform processing.

[0037] The encoding units are configured to encode the target image frame according to the encoding parameters corresponding to the multiple first data blocks respectively.

[0038] According to an aspect of the present application, an electronic device is provided, comprising: a processor; and

[0039] A memory having executable code stored thereon, which, when executed, causes the processor to perform the method according to any one of the preceding aspects.

[0040] According to an aspect of the present application, one or more machine readable media having executable code stored thereon, which, when executed, causes a processor to perform the method according to any one of the preceding aspects.

[0041] According to the embodiments of the present application, multiple encoding processing units are used to perform the determination process of encoding parameters, and a single encoding processing unit performs one or more steps in the determination process of encoding parameters, and when the encoding parameters corresponding to the image frame are calculated, the image frame is divided into data blocks, and the multiple encoding processing units performing processing in sequence are input in batches, the encoding parameters can be obtained, and then the image frame is encoded by using the encoding parameters. Since the encoding processing unit can perform processing of a next batch of data blocks after processing of a previous batch of data blocks is completed, at least two of the multiple encoding processing units perform processing synchronously in a partial time period in which the multiple encoding processing units perform processing, so that processing of multiple batches of data blocks can be performed simultaneously, compared with a sequential execution mode in which a video frame is used as a unit, the time consumed for determining the encoding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel running mode of the multiple encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0042] In addition, the multiple encoding processing unit processing scheme of the embodiments of the present application has strong versatility, can be copied to different encoding systems, and the encoding processing units can be configured to be compatible with other processing processes, so that the optimization processing of multiple processing processes is realized, and the processing efficiency of the system is improved.

[0043] The above description is only a summary of the technical solutions of the present application. In order to enable one skilled in the art to better understand the technical means of the present application and to implement the same according to the contents of the specification, and in order to enable the above and other purposes, characteristics and advantages of the present application to be more apparent and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0044] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Moreover, the same reference numerals in the attached drawings indicate the same or similar components. In the drawings:

[0045] Figure 1 A schematic diagram showing the implementation of a video encoding process based on FPGA according to an embodiment of the present application is shown;

[0046] Figure 2 A flow chart showing an embodiment of an image frame encoding method according to an embodiment of the present application is shown;

[0047] Figure 3 A flow chart showing an embodiment of an image frame encoding method according to an embodiment of the present application is shown;

[0048] Figure 4 A flow chart showing an embodiment of an image frame encoding method according to an embodiment of the present application is shown;

[0049] Figure 5 A flow chart showing an embodiment of a video encoding method according to an embodiment of the present application is shown;

[0050] Figure 6 A flow chart showing an embodiment of a video encoding method according to an embodiment of the present application is shown;

[0051] Figure 7 A flow chart showing an embodiment of an image frame processing method according to an embodiment of the present application is shown;

[0052] Figure 8 A block diagram showing an embodiment of an image frame encoding device according to an embodiment of the present application is shown;

[0053] Figure 9 A block diagram showing an embodiment of a video encoding device according to an embodiment of the present application is shown;

[0054] Figure 10 A block diagram showing an embodiment of a video encoding device according to an embodiment of the present application is shown;

[0055] Figure 11A structural block diagram of an embodiment of an image frame processing device according to Embodiment Ten of the present application is shown;

[0056] Figure 12 A structural block diagram of an embodiment of an image frame encoding system according to Embodiment Eleven of the present application is shown;

[0057] Figure 13 A structural block diagram of an embodiment of a chip based on a field programmable logic gate array according to Embodiment Twelve of the present application is shown;

[0058] Figure 14 A schematic diagram of an image frame encoding flow is shown;

[0059] Figure 15 An exemplary system that can be used to implement various embodiments described herein is shown. DETAILED DESCRIPTION

[0060] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0061] To better understand the scheme of the present application, the concepts involved in the present application are explained as follows:

[0062] The encoding parameters are used to encode the video data, and can specifically include frame types, quantization parameters (QP), and the like. The quantization parameter has an important influence on the video quality. For each encoding object, the smaller the quantization parameter, the higher the quantization precision, and the better the video quality. However, the bandwidth cost of the encoder will also increase. Therefore, by reasonably allocating the quantization parameter, the video quality is ensured while the bandwidth cost is saved. For example, a smaller quantization parameter can be allocated to an important region in the video, and a larger quantization parameter can be allocated to an unimportant region.

[0063] Based on the importance of the encoding parameters, by pre-analyzing the video to be encoded, a suitable set of encoding parameters is selected for the encoder, which plays an important role in improving the video quality and controlling the compression rate of the encoding.

[0064] The CUTREE algorithm is used in the embodiments of the present application to calculate the encoding parameters. The basic principle of the CUTREE algorithm is to analyze the relative information amount of each block of the current image frame with other image frames, and to allocate a reasonable encoding parameter to each block. The CUTREE algorithm greatly improves the video quality.

[0065] Generally, when using the CUTREE algorithm, the relative relationship between the current image frame and the reference image frame needs to be calculated, which results in too many iterations, too large computation, and slow processing speed when calculating each image frame, and cannot process high-resolution video in real time, and causes large resource consumption and high cost when implemented in hardware.

[0066] To solve the above problems, the embodiments of the present application adopt multiple encoding processing units to perform the determination process of the encoding parameters in the processing process of the CUTREE algorithm, and a single encoding processing unit performs one or more steps in the determination process of the encoding parameters. The encoding processing unit is a unit subject that processes different steps, and can be understood as a pipeline that performs the functions corresponding to each step from the application level, and the determination process of the encoding parameters is distributed to multiple pipelines with a sequence to perform.

[0067] In an optional embodiment, the determination process of the encoding parameters can be divided into four steps: a parameter reading step, an information quantity obtaining step, a data calculation step, and a data interaction step. The parameter reading step and the information quantity obtaining step are used to determine the data used in the calculation of the encoding parameters, the data calculation step completes the calculation of the encoding parameters, and the data interaction step saves the encoding parameters for subsequent encoding. In a specific implementation, each of the above steps is performed by an encoding processing unit, for example, in an implementation, four encoding processing units can be set to perform the above four steps. It can be understood that in a specific implementation, the determination process of the encoding parameters can have different step division methods, and the number of encoding processing units can also be set according to actual needs.

[0068] When calculating the encoding parameters for the image frame to be processed, the image frame is divided into data blocks. Since the data blocks corresponding to the current image frame to be processed and the reference image frame are involved, the current image frame to be processed is corresponded to a first data block, and the reference image frame is corresponded to a second data block. The first data block is input into multiple encoding processing units that are executed in sequence in batches, and the encoding parameters corresponding to the data block can be obtained after the multiple encoding processing units are processed, and then the image frame is encoded using the encoding parameters.

[0069] In the embodiment of the present application, the encoding processing units process the same batch of data blocks at the same time. Before a data block is processed by multiple encoding processing units, the next data block is started to be processed by the encoding processing units. Therefore, the processing of multiple batches of data blocks can be performed simultaneously. To improve the efficiency of the simultaneous processing of multiple batches of data blocks, the encoding processing units can perform the processing of the next batch of data blocks after the processing of the previous batch of data blocks is completed. During the time period when the multiple encoding processing units perform the processing, at least two encoding processing units perform the processing simultaneously. It can be understood that, in this processing mode, the processing time of the multiple data blocks corresponding to the target image frame to be processed is the sum of the processing time of the first data block in the order in the all encoding processing units and the processing time of the first data block in the last encoding processing unit. Compared with the frame-by-frame processing mode, the processing efficiency is greatly improved.

[0070] Therefore, compared with the sequential execution mode in units of video frames, the embodiment of the present application can greatly shorten the time consumed for determining the encoding parameters, improve the efficiency of video encoding (compared with the prior art, the efficiency is improved by several times), and reduce the processing delay of data blocks. Because the parallel operation of multiple encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware cost can be reduced, which is convenient for hardware implementation.

[0071] Because there is motion correlation between the image frames in a video, that is, there is part of the changed content and part of the same content between the image frames, the target image frame can be represented as the image content difference (which can be represented as the amount of transmitted information) between the target image frame and a certain image frame as a reference image frame. Conversely, the target image frame can be obtained according to the image content difference and the reference image frame. Therefore, when the target image frame is divided into data blocks, a specific region of the target image frame can be divided into multiple first data blocks, wherein the target region is a region of the target image frame that changes in content relative to the reference image frame.

[0072] The above-mentioned batch input of the first data blocks into the encoding processing units can be multiple data blocks as a batch or one data block as a batch, and both can achieve the effect of simultaneous processing of different batches of data blocks. When multiple data blocks enter the encoding processing units for processing, the data blocks processed earlier need to wait for the data blocks processed later, thereby causing time waste. Therefore, using one data block as a batch and inputting multiple first data blocks into the data processing units for data processing one by one can better improve the processing efficiency.

[0073] Compared with the scheme without using the CUTREE technology, the video quality can be significantly improved, and the encoding code rate can be reduced. According to statistics, the encoding system using the structure can save 10%-15% of the encoding code rate on average under the premise of the same video quality.

[0074] The determination process of the encoding parameters is described above, and the total information amount of the reference image frame needs to be updated in real time in the process of video encoding, so as to ensure that the information amount of the reference image frame used by each encoding unit is the latest accurate data. Correspondingly, the processing process of the CUTREE algorithm is divided into two parts. One is to determine the encoding parameters (mainly the quantization parameter), and the quantization parameter of the first data block is calculated according to the information amount and related information of the current first data block. The second is to superimpose the information amount, and the transfer information amount of the first data block needs to be determined, and the transfer information amount of the first data block is superimposed to the initial information amount of the second data block corresponding to the reference image frame according to the motion vector, so as to obtain the total information amount of the second data block, which can also be called the CUTREE information amount. Different from the information amount in the traditional sense, if a block is superimposed more times, the information amount is larger, and the corresponding determined quantization parameter is smaller, which represents that the importance of the block is high, and high-quality encoding needs to be used.

[0075] Therefore, the above-mentioned encoding processing unit can also correspondingly perform one or more steps in the total information amount determination process of the reference image frame of the target image frame. The parameter reading step and the information amount obtaining step also correspondingly read the reference image frame information determination data used by the data interaction step to save the reference image frame information amount for calling during encoding.

[0076] The calculation of the total information amount of the reference frame can be performed before the determination of the encoding parameters, and the execution order can also be set according to actual needs.

[0077] In a specific implementation, other calculation processes can also be implemented using the above-mentioned multi-encoding processing unit execution mode of the present application. The specific implementation process is similar to the above-mentioned execution process, which will not be described here.

[0078] The embodiment of the present application can be deployed on a processing chip, for example, an FPGA (Field Programmable Gate Array, programmable logic device) chip. The chip can be further deployed on a video encoding device. The device has a certain computing capability, and the device can be a user-side terminal, a conventional server, a cloud server, a cloud host, a virtual center, etc. The video encoded by the chip can be further transmitted over a network and used for terminal playback, video processing, etc.

[0079] The following will be described in combination with a specific application example.Figure 1 The diagram illustrates a video encoding process implemented using an FPGA according to an embodiment of this application. The FPGA chip is equipped with four encoding processing units: a first encoding processing unit, a second encoding processing unit, a third encoding processing unit, and a fourth encoding processing unit, which respectively perform the parameter reading step, information acquisition step, data calculation step, and data interaction step in the process of determining encoding parameters.

[0080] The target image frame input to the FPGA chip is first divided into multiple first data blocks. As shown in the figure, the video frame is divided into five first data blocks: CU0, CU1, CU2, CU3, and CU4. These five first data blocks are then input into multiple encoding processing units in batches. After the previous batch of first data blocks is processed, the next batch is processed. For example... Figure 1 As shown, the first data block CU0, which is at the top of the sorted list, enters the first encoding processing unit to perform the parameter reading step. During this time, other encoding processing units have no processing tasks. After CU0 is processed by the first encoding processing unit, it continues to the second encoding processing unit for processing. The idle first encoding processing unit then continues to process CU1. For each CU, the processing process is to enter each encoding processing unit sequentially for processing. For each encoding processing unit, the processing process is to process each CU sequentially, and to continue processing the next CU after processing the previous one. After all encoding processing units have completed processing, each CU obtains its corresponding encoding parameters. Based on the encoding results of each CU according to the encoding parameters, the encoding results of the image frame can be further obtained.

[0081] During a portion of the time period in which multiple encoding processing units perform processing, at least two encoding processing units perform processing synchronously. Figure 1 It can be seen that, excluding the processing of CU0 in the first coding unit and CU4 in the fourth coding unit, the processing of different data blocks in each coding unit is performed simultaneously. For example, while CU4 is processed in the first coding unit, CU3 is processed in the second coding unit, CU2 in the third coding unit, and CU1 in the fourth coding unit. The total processing time is the sum of the processing time of CU0 in the four coding units and the processing time of CU1, CU2, CU3, and CU4 in the fourth coding unit. This saves processing time compared to processing each data block sequentially. Assuming that the processing time of each data block in the coding unit is the same, which is X time, then this example takes 8X time, while frame-by-frame processing takes 20X time.

[0082] It should be noted that the purpose of providing the above application examples is to facilitate understanding of the methods provided in the embodiments of this application, and is not intended to limit the methods.

[0083] Various embodiments of the present application are described next.

[0084] Referring to Figure 2 , a flow chart of an embodiment of a method for encoding an image frame according to an embodiment of the present application is shown, which can specifically include the following steps:

[0085] Step 101, divide the target image frame into a plurality of first data blocks.

[0086] The data block CU is a smaller coding unit divided from the image frame. The division effect can be set according to actual needs, including the size, shape, area of each data block, and the number of data blocks, etc. For example, it can be divided into 8x8 or 16x16 data blocks. Larger data blocks can also be divided first, and then the larger data blocks are further divided into smaller data blocks. The specific division method can be selected according to actual needs, for example, a quadtree partitioning method can be used.

[0087] In order to better adapt to image details, the image area with complex details can be divided into smaller data block size, and the image area with simple part is divided into larger data block size.

[0088] In the process of dividing data blocks, rate-distortion optimization can also be referred to for decision. A larger data block can be divided into smaller data blocks, and in this process, the rate-distortion cost of the larger data block and the sum of the rate-distortion costs of the smaller data blocks are calculated. If the latter is smaller, the division of the smaller data block will be performed.

[0089] Step 102, input the plurality of first data blocks into a plurality of encoding processing units in batches in sequence; after the encoding processing units complete the processing of the previous batch of first data blocks, the processing of the next batch of first data blocks is performed, the encoding processing units correspond to one or more steps in the determination process of the encoding parameters of the target image frame, and at least two encoding processing units synchronously perform processing within a partial time period when the plurality of encoding processing units perform processing.

[0090] The encoding processing units process the data blocks entering in the same batch at the same time, and the processing of the next data block in the encoding processing unit is started before a data block is processed by multiple encoding processing units. That is, the processing of multiple batches of data blocks can be performed at the same time.

[0091] The first data blocks are input into the encoding processing units in batches, which can be multiple data blocks as a batch, or one data block as a batch.

[0092] In order to achieve higher efficiency in the simultaneous processing of multiple batches of data blocks, the encoding processing units are designed to perform the processing of the next batch of data blocks after completing the processing of the previous batch of data blocks. Compared with the frame-by-frame processing method, the processing efficiency is greatly improved.

[0093] In specific implementation, the control signal can be used to ensure that the coding processing units are independent and work uninterruptedly.

[0094] At step 103, the target image frame is encoded according to the coding parameters corresponding to the plurality of first data blocks.

[0095] The coding parameters can be used to encode the first data blocks, and the coding results of the target image frame can be obtained according to the coding results of the data blocks.

[0096] According to the embodiments of the present application, the determination of the coding parameters is performed by using a plurality of coding processing units, and one or more steps in the determination of the coding parameters are performed by a single coding processing unit. When the coding parameters are calculated for the image frame, the image frame is divided into data blocks, and the plurality of coding processing units are input in batches and executed in sequence. The coding parameters can be obtained, and the image frame is encoded by using the coding parameters. Since the coding processing units can perform the processing of the next batch of data blocks after the processing of the previous batch of data blocks, at least two coding processing units perform the processing synchronously in the time period in which the plurality of coding processing units perform the processing. Therefore, the processing of the plurality of batches of data blocks can be performed simultaneously. Compared with the sequential execution in the unit of video frame, the time consumed for the determination of the coding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel operation of the plurality of coding units can reuse the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0097] In addition, the processing scheme of the plurality of coding processing units in the embodiments of the present application has strong versatility, can be copied to different encoding systems, and the coding processing units can be configured to be compatible with other processing processes. Therefore, the optimization processing of the plurality of processing processes is implemented, and the processing efficiency of the system is improved.

[0098] Referring to Figure 3 FIG. 2 shows a flowchart of an embodiment of an image frame encoding method according to Embodiment Two of the present application. The method can specifically include the following steps:

[0099] At step 201, a specific region of the target image frame is divided into a plurality of first data blocks. The target region is a region in which the content of the target image frame changes relative to a reference image frame.

[0100] Based on the motion correlation between the image frames in a video, the target image frame can be represented as the image content difference between the target image frame and a certain image frame as a reference image frame. Accordingly, when the target image frame is divided into data blocks, the region in which the content of the target image frame changes relative to the reference image frame can be determined, and the changed region is further divided into a plurality of first data blocks.

[0101] In step 202, the plurality of first data blocks are input into the data processing unit one by one for data processing; after the encoding processing unit finishes processing the previous batch of first data blocks, the encoding processing unit performs processing on the next batch of first data blocks, and the encoding processing unit corresponds to performing one or more steps in the determination process of the encoding parameters of the target image frame; and at least two encoding processing units perform processing synchronously in the partial time period in which the plurality of encoding processing units perform processing.

[0102] Using one data block as one batch can better improve the processing efficiency than using multiple data blocks as one batch.

[0103] In step 203, the target image frame is encoded according to the encoding parameters corresponding to the plurality of first data blocks.

[0104] According to the embodiments of the present application, the determination process of the encoding parameters is performed by using a plurality of encoding processing units, one or more steps in the determination process of the encoding parameters are performed by a single encoding processing unit, the image frame is divided into data blocks when the encoding parameters of the image frame are calculated, the plurality of encoding processing units are input in batches and executed in sequence, the encoding parameters are obtained, and then the image frame is encoded by using the encoding parameters. Since the encoding processing unit can perform processing on the next batch of data blocks after finishing processing on the previous batch of data blocks, at least two encoding processing units perform processing synchronously in the partial time period in which the plurality of encoding processing units perform processing, so that the processing of multiple batches of data blocks can be performed simultaneously, the time consumed for determining the encoding parameters can be greatly shortened compared with the sequential execution mode in units of video frames, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel running mode of the multiple encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0105] In addition, the multiple encoding processing unit processing scheme of the embodiments of the present application has strong versatility, can be copied to different encoding systems, and can be configured to be compatible with other processing processes, so that the optimization processing of multiple processing processes is realized, and the processing efficiency of the system is improved.

[0106] Reference Figure 4 Fig. 3 shows a flowchart of an image frame encoding method according to an embodiment of the present application, which can specifically include the following steps:

[0107] In step 301, the target image frame is divided into a plurality of first data blocks.

[0108] Step 302, inputting the plurality of first data blocks into a plurality of encoding processing units in batches in sequence; the encoding processing units execute the processing of the next batch of first data blocks after the processing of the previous batch of first data blocks is completed, and the data processing units correspondingly execute the parameter reading step, the information quantity obtaining step, the data calculation step, and the data interaction step in the determination process of the encoding parameters of the target image frame; at least two encoding processing units synchronously execute processing in the time period in which the four encoding processing units execute processing.

[0109] Step 303, encoding the target image frame according to the encoding parameters corresponding to the plurality of first data blocks respectively.

[0110] In the embodiment, the processing content of each encoding processing unit is specifically determined, and the determination process of the encoding parameters is divided into a parameter reading step, an information quantity obtaining step, a data calculation step, and a data interaction step. The number of encoding processing units and the specific processing content executed can be set according to actual needs, and one encoding processing unit can process one or more steps. For example, three encoding processing units are designed, the first encoding processing unit processes the parameter reading step and the information quantity obtaining step, the second encoding processing unit processes the data calculation step, and the third encoding processing unit processes the data interaction step.

[0111] In an optional embodiment of the present application, the parameter reading step includes reading prediction parameters of the first data block, and the prediction parameters include a prediction direction and a prediction cost in the encoding process of the target image frame. The prediction direction can be intra prediction or inter prediction, or intra prediction combined with inter prediction, and the prediction cost measures the deviation data caused in the encoding prediction process.

[0112] Prediction of the image frame is a compression process adopted in the encoding process, which can be divided into intra prediction and inter prediction, and can also be called inter encoding and intra encoding. Intra prediction is based on the strong correlation between pixels in the same image, and can utilize the correlation between pixels for compression encoding. For example, taking a pixel as a reference pixel, the difference between other pixels and the reference pixel is used to represent other pixels, so as to compress other pixels. For example, when transmitting data, the actual pixel X (current value) and the reference pixel (predicted value) are subtracted, and the difference value is transmitted. The receiving end obtains the actual pixel according to the sum of the difference value and the reference pixel. Inter prediction is based on the strong correlation between two image frames in a video, for example, a reference image frame can be an image frame closest to the target image frame in a certain direction, and the difference between the target image frame and the reference image frame is used to express the target image frame, so as to compress the image frame.

[0113] In an optional embodiment of the present application, the information amount obtaining step comprises: obtaining the initial information amount of the first data block from the corresponding memory according to the information amount storage address of the first data block; obtaining the intra-frame information amount of the first data block according to the prediction cost and the encoding frame rate of the first data block; and determining the total information amount of the first data block according to the intra-frame information amount and the initial information amount. For example, the intra-frame information amount of the first data block can be obtained by multiplying the prediction cost by the encoding frame rate, and the total information amount of the first data block can be obtained by adding the intra-frame information amount to the initial information amount.

[0114] The initial information amount of the image frame is used to calculate the total information amount, and other frames that refer to the current image frame are superimposed on the information amount of the current image frame. Therefore, the information amount of each frame is continuously increased in the process of CUTREE calculation, until the superimposition process is completed, and the total information amount is obtained, which is used to derive the quantization parameter.

[0115] The initial information amount of the image frame can be calculated in advance by the number of pixels of the image and the number of distinguishable colors.

[0116] In an optional embodiment of the present application, the data calculating step comprises: determining the encoding parameter corresponding to the first data block according to the total information amount, the prediction direction and the prediction cost of the first data block. Taking the prediction direction as intra-frame prediction and the encoding parameter as the quantization parameter as an example, the quantization parameter can be obtained by multiplying the total information amount of the first data block by the encoding frame rate coefficient, dividing the result by the intra-frame prediction cost coefficient of the first data block, and taking the log of the result.

[0117] In an optional embodiment of the present application, the parameter reading step comprises: obtaining the information amount storage address of the first data block according to the position of the first data block in the image. First, the position of the data block in the image is determined, and the information amount storage address of the first data block can be determined according to the mapping relationship between the position and the information amount storage address.

[0118] Since the process of video encoding not only uses the encoding parameter, but also needs to be based on the information amount of the reference image frame, the determination process of the encoding parameter can also be implemented by multiple encoding processing units.

[0119] Therefore, the above-mentioned encoding processing unit can also correspondingly perform one or more steps in the total information amount determination process of the reference image frame of the target image frame, and the parameter reading step and the information amount obtaining step also correspondingly read the reference data used to determine the information amount of the reference image frame. The data interaction step is also used to save the information amount of the reference image frame for calling during encoding.

[0120] Correspondingly, the information amount obtaining step can further comprise: determining the transfer information amount between the target image frame and the reference image frame according to the total information amount of the first data block. The information amount obtaining step can also obtain one of the bases for calculating the total information amount of the reference frame, i.e., the total information amount of the first data block of the target image frame, and further determine the transfer information amount between the target image frame and the reference image frame according to the total information amount of the first data block, i.e., the difference information amount between the two.

[0121] Correspondingly, the data calculating step further comprises: obtaining the initial information amount of the second data block in the reference image frame; and superimposing the transfer information amount of the first data block on the initial information amount of the second data block in the reference image frame to obtain the total information amount of the second data block in the reference image frame. Since the transfer information amount is also the difference information amount between the target image frame and the reference image frame, the transfer information amount can be superimposed on the initial information amount of the reference image frame to obtain the total information amount of the second data block in the reference image frame.

[0122] In an optional embodiment of the present application, the parameter reading step comprises: reading a prediction cost coefficient. The prediction cost coefficient is a coefficient for the prediction cost, i.e., the prediction deviation or the encoding deviation, which can be preset or determined according to the numerical range of the prediction cost.

[0123] Correspondingly, in an optional embodiment of the present application, the determination of the transfer information amount between the target image frame and the reference image frame according to the initial information amount of the first data block comprises: obtaining the intra-frame information amount of the first data block according to the prediction cost and the encoding frame rate of the first data block; determining the total information amount of the first data block according to the intra-frame information amount and the initial information amount; and determining the transfer information amount of the first data block according to the total information amount and the prediction cost coefficient. The specific calculation formula can be set according to actual requirements, and the data calculation process is a known technology in the art, which will not be described here.

[0124] In an optional embodiment of the present application, the parameter reading step further comprises: determining the position of the second data block in the reference image frame to which the information amount is to be superimposed according to the position of the first data block in the target image frame and the motion vector. The motion vector (MV) herein represents the relative displacement between the first data block and the second data block with the highest matching degree in the image of the reference image frame in the inter-frame encoding process, and therefore, the position of the second data block corresponding to the first data block in the reference image frame can be determined according to the position of the first data block in the target image and the relative displacement represented by the motion vector.

[0125] In an optional embodiment of the present application, the data interaction step comprises: storing the encoding parameters into corresponding storage units for subsequent calling during encoding.

[0126] In an optional embodiment of the present application, the data interaction step comprises: transferring the total information amount of the second data block in the reference image frame to other encoding processing units associated with use, and updating the data used by other encoding processing units in time to ensure that the data used by other encoding processing units is the latest accurate number, thereby avoiding the situation that when accessing the total information amount of the second data block in the reference image frame from the storage location, the memory data is not updated in time, resulting in the encoding processing unit being paused.

[0127] According to the embodiments of the present application, a plurality of encoding processing units are used to perform the encoding parameter determination process, and a single encoding processing unit performs one or more steps in the encoding parameter determination process. When calculating the encoding parameters for the image frame, the image frame is divided into data blocks, and the plurality of encoding processing units are input in batches and executed in sequence. The encoding parameters can be obtained, and then the image frame is encoded using the encoding parameters. Since the encoding processing unit can perform the processing of the next batch of data blocks after processing the previous batch of data blocks, at least two encoding processing units perform processing synchronously during the time period when the plurality of encoding processing units perform processing, so that the processing of multiple batches of data blocks can be performed simultaneously. Compared with the sequential execution mode in units of video frames, the time consumed for determining the encoding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel running mode of the multiple encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0128] Moreover, the multiple encoding processing unit processing scheme of the embodiments of the present application has strong versatility, can be copied to different encoding systems, and can be configured to be compatible with other processing processes, thereby realizing the optimized processing of multiple processing processes and improving the processing efficiency of the system.

[0129] Reference Figure 5 is a flowchart of an embodiment of a video encoding method according to Embodiment Four of the present application. The method can specifically include the following steps:

[0130] Step 401: dividing a target image frame in a video into a plurality of first data blocks.

[0131] Step 402: inputting the plurality of first data blocks in batches into a plurality of encoding processing units executed in sequence; after the encoding processing unit completes the processing of the previous batch of first data blocks, the processing of the next batch of first data blocks is performed, and the encoding processing unit corresponds to one or more steps in the determination process of the encoding parameters of the image frame.

[0132] Step 403, encoding the target image frame according to the encoding parameters corresponding to the plurality of first data blocks respectively.

[0133] Step 404, obtaining the encoding result of the video according to the encoding result of the target image frame.

[0134] According to the embodiments of the present application, a plurality of encoding processing units are used to perform the encoding parameter determination process, and a single encoding processing unit performs one or more steps in the encoding parameter determination process. When calculating the encoding parameters for an image frame, the image frame is divided into data blocks, and the plurality of encoding processing units are input in batches and executed in sequence. The encoding parameters can be obtained, and then the image frame is encoded using the encoding parameters. Since the encoding processing unit can perform the processing of the next batch of data blocks after processing the previous batch of data blocks, at least two encoding processing units perform processing synchronously during the time period when the plurality of encoding processing units perform processing. Therefore, the processing of multiple batches of data blocks can be performed simultaneously. Compared with the sequential execution mode in units of video frames, the time consumed for determining the encoding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel operation of the multiple encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0135] Furthermore, the multiple encoding processing unit processing scheme of the embodiments of the present application has strong versatility and can be copied to different encoding systems. In addition, the encoding processing units can be configured to be compatible with other processing processes, so that the optimization processing of multiple processing processes is realized, and the processing efficiency of the system is improved.

[0136] Reference Figure 6 , a flowchart of a video encoding method embodiment according to Embodiment Five of the present application is shown. The method can specifically include the following steps:

[0137] Step 501, dividing a specific region of a target image frame in a video into a plurality of first data blocks, the target region being a region of the target image frame that has content changes relative to a reference image frame.

[0138] Step 502, inputting the plurality of first data blocks in batches into four encoding processing units executed in sequence; after the four encoding processing units complete the processing of the previous batch of first data blocks, the processing of the next batch of first data blocks is performed, and the four encoding processing units respectively perform the quantization parameter calculation in the encoding parameter determination process of the image frame according to the reading step, the information amount acquisition step, the quantization parameter and information amount calculation step, and the quantization parameter and information amount writing step.

[0139] Step 503, encoding the target image frame according to the encoding parameters corresponding to the four first data blocks respectively.

[0140] Step 504, obtaining the encoding result of the video according to the encoding result of the target image frame.

[0141] According to the embodiments of the present application, the determination process of the encoding parameters is performed by using multiple encoding processing units, a single encoding processing unit performs one or more steps in the determination process of the encoding parameters, when the encoding parameters are calculated for the image frame, the image frame is divided into data blocks, and the multiple encoding processing units performing the processing in sequence are input in batches, the encoding parameters can be obtained, and then the image frame is encoded by using the encoding parameters. Since the encoding processing unit can perform the processing of the next batch of data blocks after the processing of the previous batch of data blocks is completed, at least two encoding processing units perform the processing synchronously in the time period in which the multiple encoding processing units perform the processing, so that the processing of the multiple batches of data blocks can be performed simultaneously, compared with the sequential execution mode in the unit of the video frame, the time consumed for determining the encoding parameters can be greatly shortened, the efficiency of the video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel running mode of the multiple encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0142] In addition, the multiple encoding processing unit processing scheme of the embodiments of the present application has strong versatility, can be copied to different encoding systems, and the encoding processing unit can be configured to be compatible with other processing processes, so that the optimization processing of the multiple processing processes is realized, and the processing efficiency of the system is improved.

[0143] Referring to Figure 7 , a flow chart of an embodiment of an image frame processing method according to Embodiment Six of the present application is shown, the method can specifically include the following steps:

[0144] Step 601, dividing a target image frame into multiple first data blocks.

[0145] Step 602, inputting the multiple first data blocks into multiple processing units performing the processing in sequence in batches. After the processing of the previous batch of first data blocks is completed, the processing of the next batch of first data blocks is performed, and the processing unit corresponds to performing one or more steps in the processing process of the target image frame.

[0146] According to the embodiment of the present application, a plurality of processing units are used to perform the processing procedure of the target image frame, a single processing unit performs one or more steps in the processing procedure, the image frame is divided into data blocks, and the plurality of processing units performing in sequence are input in batches, and the processing result can be obtained. Since the processing unit can perform the processing of the next batch of data blocks after completing the processing of the previous batch of data blocks, at least two encoding processing units perform processing synchronously in the partial time period when the plurality of encoding processing units perform processing, so that the processing of the plurality of batches of data blocks can be performed simultaneously, compared with the sequential execution mode in units of video frames, the time consumed by the processing procedure can be greatly reduced, and the processing efficiency of the video frame is improved. Since the parallel operation mode of the plurality of processing units reuses the resources of the hardware unit, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0147] In addition, the multi-processing unit processing scheme of the embodiment of the present application has strong universality, can be copied to different processing systems for use, and the processing units can be configured to be compatible with other processing procedures, so that the optimized processing of the plurality of processing procedures is realized, and the processing efficiency of the system is improved.

[0148] The image frame processing method of the embodiment of the present application can be applied to video processing, and further can be applied to a software service SaaS (Software-as-a-Service, software as a service) platform to provide a video processing service for various software.

[0149] The SaaS provides software services through a network, and a platform vendor uniformly deploys application software on its own server. A platform customer (for example, a service provider providing software) can order the required software service from the platform through the Internet according to actual needs, and further obtain the service provided by the platform through the Internet.

[0150] The specific implementation process can include:

[0151] Step 1, obtaining at least one target video submitted through a software service client.

[0152] Step 2, dividing the image frames of the target video into a plurality of first data blocks.

[0153] Step 3, inputting the plurality of first data blocks in batches into a plurality of encoding processing units performing in sequence. The encoding processing unit performs the processing of the next batch of first data blocks after completing the processing of the previous batch of first data blocks, and the encoding processing unit corresponds to performing one or more steps in the determination procedure of the encoding parameters of the target image frame. At least two encoding processing units perform processing synchronously in the partial time period when the plurality of encoding processing units perform processing.

[0154] Step 4, encoding the target image frame according to the encoding parameters corresponding to the plurality of first data blocks respectively.

[0155] Step 5, obtaining the encoding result of the target video according to the encoding results of the plurality of image frames.

[0156] Step 6, providing the encoding result of the target video based on the software service client.

[0157] According to the embodiments of the present application, a plurality of processing units are used to perform the processing procedure of the target image frame, and a single processing unit performs one or more steps in the processing procedure. The image frame is divided into data blocks, and the plurality of processing units are input in batches and executed in sequence to obtain the processing result. Since the processing unit can perform the processing of the next batch of data blocks after completing the processing of the previous batch of data blocks, at least two encoding processing units perform processing synchronously in the partial time period when the plurality of encoding processing units perform processing, so that the processing of the plurality of batches of data blocks can be performed simultaneously. Compared with the sequential execution mode in units of video frames, the processing time consumed by the processing procedure can be greatly reduced, and the processing efficiency of the video frame is improved. Since the parallel operation mode of the plurality of processing units reuses the resources of the hardware unit, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0158] In addition, the multi-processing unit processing scheme of the embodiments of the present application has strong universality, can be copied to different processing systems, and can be configured to be compatible with other processing procedures, so that the optimization processing of the plurality of processing procedures is realized, and the processing efficiency of the system is improved.

[0159] In order for those skilled in the art to better understand the present application, a specific example of an image frame encoding method of the present application is described below, in which each stage of the pipeline is an encoding processing unit executed in sequence. Referring to Figure 14 , a schematic diagram of an image frame encoding process is shown, which specifically includes the following steps:

[0160] 1. The CUTREE control unit starts the first stage pipeline, reads the information of the current CU from the CU parameter storage according to the current CU position of the target image frame, including the motion vector, the prediction type, the intra prediction cost, the inter prediction cost coefficient, etc.

[0161] 2. The first stage pipeline calculates the information storage address of the current CU in the memory according to the position of the current CU in the image; calculates the information storage address of the CU corresponding to the current CU in each reference image frame according to the position of the current CU in the image and the motion vector of the current CU.

[0162] 3. After the operation of the first stage pipeline is completed, the next CU is processed, and the control unit starts the second stage pipeline.

[0163] 4. The second pipeline stores the address of the information amount of each CU in each reference frame, reads the initial information amount of the reference image frame from the information amount storage of the reference image frame, and reads the information amount of the current CU from the information amount storage of the current CU according to the information amount storage address of the current CU.

[0164] 5. The second pipeline multiplies the intra-frame prediction cost of the CU by the encoding frame rate to obtain the intra-frame information amount of the CU, and adds the intra-frame information amount of the CU to the information amount of the current CU read from the information amount storage of the current frame to obtain the total information amount of the current CU.

[0165] 6. The second pipeline multiplies the total information amount of the current CU by the inter-frame cost coefficient of the CU to obtain the transfer information amount of the current CU, and controls the third pipeline to start after the calculation is completed.

[0166] 7. The third pipeline obtains the initial information amount of four CUs in each reference frame from the reference frame information amount storage, and superimposes the transfer information amount of the current CU calculated by the second pipeline on the initial information amount of the four CUs in each reference frame.

[0167] 8. The third pipeline divides the total information amount of the current CU by the intra-frame prediction cost of the current CU, takes the log of the result, and obtains the quantization parameter.

[0168] 9. The fourth pipeline writes the quantization parameter QP into the quantization parameter QP storage, and writes the total information amount of the CU in each reference frame after being updated by the third pipeline into the reference frame information amount storage.

[0169] 10. If the total information amount of the CU in the reference frame required by the third pipeline is sourced from the fourth pipeline, the fourth pipeline can feed back the total information amount of the CU in the reference frame to the third pipeline, so as to ensure that the information amount of the CU required by the third pipeline is the latest result, thereby avoiding pipeline stall caused by the delay of updating the memory data when different pipelines access the same address.

[0170] A specific calculation example is provided as follows:

[0171] Suppose there are 10 image frames, F0...F9, and each data block in each image frame corresponds to a CU TREE information amount. The initial information amount of each block CUTREE (which can be set according to actual needs) is stored in the storage.

[0172] In the process of determining the quantization parameter, when calculating F1, for each data block in F1, read the initial information amount A (set to 0) of the block in the memory, and calculate the intra-frame information amount B according to the intra-frame prediction cost and the encoding frame rate of the block, then add A and B to obtain the total information amount of the data block.

[0173] Multiply the total information amount by the inter-frame cost coefficient C to obtain the result (A+B)*C, which is the total information amount of the current block.

[0174] Suppose the reference frame of F1 is F0, find the four blocks in F0 according to the motion vector, read the information amounts of the four blocks from the memory, such as K0, K1, K2, K3, divide (A+B)*C into four parts, and add them to K0, K1, K2, K3 to obtain the total information amounts of the four data blocks in F0 respectively, and then update them to the memory of F0 frame.

[0175] Then start calculating F2, if F2 refers to F0 and F1 at the same time, when F2 is calculated, the information amounts of each block in F0 and F1 will also be updated. Similarly, when F0-F9 are all calculated.

[0176] After calculating the total information amount of each data block, start calculating the quantization parameter of each block in F0 to F9.

[0177] For example, start calculating the quantization parameter of a block in F0, first read the total information amount (after F1...F9 superposition) from the memory, multiply the total information amount by the encoding frame rate coefficient, divide by the intra-frame cost coefficient of the block, and finally take the logarithm LOG to obtain the quantization parameter.

[0178] Referring to Figure 8 , a structure block diagram of an embodiment of an image frame encoding device according to Embodiment Seven of the present application is shown, which can specifically include:

[0179] The data block segmentation unit 701 is configured to divide the target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches for sequential execution;

[0180] The encoding processing unit 702 is configured to correspondingly execute one or more steps in the process of determining the encoding parameter of the target image frame, and execute the processing of the next batch of first data blocks after the processing of the previous batch of first data blocks is completed;

[0181] The encoding unit 703 is configured to encode the target image frame according to the encoding parameters corresponding to the plurality of first data blocks respectively.

[0182] In an optional embodiment of the present application, the data block splitting unit is specifically configured to divide a specific region of the target image frame into a plurality of first data blocks, and the target region is a region of the target image frame that has content changes relative to a reference image frame.

[0183] In an optional embodiment of the present application, the data block splitting unit is specifically configured to input the plurality of first data blocks into a data processing unit one by one for data processing.

[0184] In an optional embodiment of the present application, the plurality of data processing units include a parameter reading unit, an information amount obtaining unit, a data calculation unit, and a data interaction unit.

[0185] In an optional embodiment of the present application, the parameter reading unit is specifically configured to read prediction parameters of the first data block, and the prediction parameters include a prediction direction and a prediction cost in a target image frame encoding process.

[0186] In an optional embodiment of the present application, the information amount obtaining unit is specifically configured to obtain an initial information amount of the first data block from a corresponding memory according to an information amount storage address of the first data block, obtain an intra-frame information amount of the first data block according to the prediction cost and an encoding frame rate of the first data block, and determine a total information amount of the first data block according to the intra-frame information amount and the initial information amount.

[0187] In an optional embodiment of the present application, the data calculation unit is specifically configured to determine an encoding parameter for the first data block according to the total information amount, the prediction direction, and the prediction cost of the first data block.

[0188] In an optional embodiment of the present application, the parameter obtaining unit is specifically configured to obtain the information amount storage address of the first data block according to a position of the first data block in an image.

[0189] In an optional embodiment of the present application, the encoding processing unit is further configured to correspondingly execute one or more steps in a total information amount determination process of a reference image frame of the target image frame, and the information amount obtaining unit is further configured to determine a transfer information amount between the target image frame and the reference image frame according to the total information amount of the first data block.

[0190] The data calculation unit is further configured to obtain an initial information amount of a second data block in the reference image frame, superimpose the transfer information amount of the first data block on the initial information amount of the second data block in the reference image frame to obtain a total information amount of the second data block in the reference image frame.

[0191] In an optional embodiment of the present application, the parameter reading unit is configured to read a prediction cost coefficient.

[0192] The delivery information amount determination sub-unit is configured to determine the delivery information amount of the first data block according to the total information amount of the first data block and the prediction cost coefficient.

[0193] In an optional embodiment of the present application, the parameter reading unit is further configured to determine the position of a second data block of the reference image frame to be superimposed according to the position of the first data block in the target image frame and the motion vector.

[0194] In an optional embodiment of the present application, the data interaction unit is configured to store the encoding parameter to a corresponding storage unit for subsequent calling during encoding.

[0195] In an optional embodiment of the present application, the data interaction unit is configured to deliver the total information amount of the second data block in the updated reference image frame to other encoding processing units associated for use.

[0196] According to the embodiments of the present application, a plurality of encoding processing units are adopted to perform the determination process of the encoding parameter, and a single encoding processing unit performs one or more steps in the determination process of the encoding parameter. When the encoding parameter is calculated for an image frame, the image frame is divided into data blocks, and the plurality of encoding processing units are input in batches and executed in sequence. The encoding parameter can be obtained, and the image frame is encoded by using the encoding parameter. Since the encoding processing unit can perform the processing of the next batch of data blocks after processing the previous batch of data blocks, at least two encoding processing units perform processing synchronously in a part of time period in which the plurality of encoding processing units perform processing. Therefore, the processing of the plurality of batches of data blocks can be performed simultaneously. Compared with the sequential execution mode in units of video frames, the time consumed for determining the encoding parameter can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel operation mode of the plurality of encoding units reuses the resources of the hardware unit, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0197] In addition, the processing scheme of the plurality of encoding processing units in the embodiments of the present application is highly versatile, can be copied to different encoding systems, and can be configured to be compatible with other processing procedures. Therefore, the optimization processing of the plurality of processing procedures is realized, and the processing efficiency of the system is improved.

[0198] Reference Figure 9 FIG. 8 shows a structure block diagram of a video encoding device according to an embodiment of the present application, which can specifically include:

[0199] The data block splitting unit 801 is configured to divide a target image frame in a video into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches for sequentially performing encoding processing.

[0200] The encoding processing unit 802 is configured to perform one or more steps in a determination process of an encoding parameter corresponding to the target image frame, and perform processing of a next batch of first data blocks after processing of a previous batch of first data blocks is completed.

[0201] The image frame encoding unit 803 is configured to encode the target image frame according to the encoding parameters corresponding to the plurality of first data blocks, respectively.

[0202] The video encoding unit 804 is configured to obtain an encoding result of the video according to an encoding result of the target image frame.

[0203] According to the embodiments of the present application, the determination process of the encoding parameter is performed by a plurality of encoding processing units, and one or more steps in the determination process of the encoding parameter are performed by a single encoding processing unit. When the encoding parameter is calculated for the image frame, the image frame is divided into data blocks, and the plurality of encoding processing units are input in batches for sequentially performing processing. The encoding parameter can be obtained, and then the image frame is encoded by using the encoding parameter. Since the processing of the next batch of data blocks can be performed by the encoding processing unit after the processing of the previous batch of data blocks is completed, at least two encoding processing units perform processing synchronously in a partial time period in which the plurality of encoding processing units perform processing. Therefore, the processing of the plurality of batches of data blocks can be performed simultaneously. Compared with the sequential execution mode in units of video frames, the time consumed for determining the encoding parameter can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel operation mode of the plurality of encoding units reuses the resources of the hardware unit, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0204] In addition, the processing scheme of the plurality of encoding processing units in the embodiments of the present application has strong versatility, can be copied to different encoding systems, and the encoding processing units can be configured to be compatible with other processing processes. Therefore, the optimization processing of the plurality of processing processes is realized, and the processing efficiency of the system is improved.

[0205] Reference Figure 10 FIG. 9 shows a structure block diagram of a video encoding device according to an embodiment of the present application, which can specifically include:

[0206] The data block splitting unit 901 is configured to divide a target region of a target image frame in a video into a plurality of first data blocks, and input the plurality of first data blocks into four encoding processing units in batches for sequentially performing processing. The target region is a region in which content of the target image frame changes relative to a reference image frame.

[0207] four encoding processing units 902, for performing quantization parameter calculation in the determination process of the encoding parameters of the image frame according to the reading step, the information quantity obtaining step, the quantization parameter and information quantity calculation step, and the quantization parameter and information quantity writing step, after the encoding processing units complete the processing of the previous batch of first data blocks, the encoding processing units perform the processing of the next batch of first data blocks;

[0208] an image frame encoding unit 903, for encoding the target image frame according to the encoding parameters corresponding to the four first data blocks respectively;

[0209] a video encoding unit 904, for obtaining the encoding result of the video according to the encoding result of the target image frame.

[0210] According to the embodiments of the present application, the determination process of the encoding parameters is performed by using multiple encoding processing units, a single encoding processing unit performs one or more steps in the determination process of the encoding parameters, when the encoding parameters of the image frame are calculated, the image frame is divided into data blocks, and the multiple encoding processing units performing processing in sequence are input in batches, the encoding parameters can be obtained, and then the image frame is encoded by using the encoding parameters. Since the encoding processing unit can perform the processing of the next batch of data blocks after the processing of the previous batch of data blocks is completed, at least two encoding processing units perform processing synchronously in the time period in which the four encoding processing units perform processing, so that the processing of multiple batches of data blocks can be performed simultaneously, compared with the sequential execution mode in which the video frame is used as a unit, the time consumed for determining the encoding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel running mode of the multiple encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0211] In addition, the multiple encoding processing unit processing scheme of the embodiments of the present application has strong versatility, can be copied to different encoding systems, and the encoding processing units can be configured to be compatible with other processing processes, so that the optimization processing of multiple processing processes is realized, and the processing efficiency of the system is improved.

[0212] Reference Figure 11 , a structural block diagram of an image frame processing device embodiment according to Embodiment Ten of the present application is shown, which can specifically include:

[0213] a data block segmentation unit 1001, for dividing a target image frame into multiple first data blocks, and inputting the multiple first data blocks into multiple processing units performing processing in sequence in batches;

[0214] The processing unit 1002 is configured to execute one or more steps in a processing procedure of the target image frame and obtain a processing result. After the processing unit finishes processing a previous batch of first data blocks, the processing unit executes processing of a next batch of first data blocks.

[0215] According to the embodiment of the present application, a plurality of processing units are used to execute the processing procedure of the target image frame, a single processing unit executes one or more steps in the processing procedure, the image frame is divided into data blocks, and the plurality of processing units are input in batches and executed in sequence, so that the processing result can be obtained. Since the processing unit can execute processing of a next batch of data blocks after processing of a previous batch of data blocks, at least two encoding processing units execute processing synchronously in a partial time period in which the plurality of encoding processing units execute processing, so that processing of a plurality of batches of data blocks can be performed simultaneously. Compared with a sequential execution mode in units of video frames, the processing procedure can be greatly shortened, and the processing efficiency of the video frame is improved. Since the parallel operation mode of the plurality of processing units reuses resources of the hardware unit, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0216] In addition, the multi-processing unit processing scheme of the embodiment of the present application is highly versatile, can be copied to different processing systems, and can be configured to be compatible with other processing procedures, so that the processing efficiency of the system is improved.

[0217] Referring to Figure 12 , a structure block diagram of an image frame encoding system embodiment according to the eleventh embodiment of the present application is shown, which includes a frame division unit, a plurality of encoding processing units executed in sequence, and an encoding unit;

[0218] The frame division unit 1101 is configured to divide the target image frame into a plurality of first data blocks, and input the plurality of first data blocks in batches to the plurality of encoding processing units;

[0219] The encoding processing unit 1102 is configured to execute one or more steps in a determination procedure of an encoding parameter of the target image frame, and execute processing of a next batch of first data blocks after finishing processing of a previous batch of first data blocks;

[0220] The image frame encoding unit 1103 is configured to encode the target image frame according to the encoding parameters corresponding to the plurality of first data blocks, respectively.

[0221] According to the embodiment of the present application, a plurality of encoding processing units are used to perform the determination process of the encoding parameters, and each encoding processing unit performs one or more steps in the determination process of the encoding parameters. When the encoding parameters are calculated for the image frame, the image frame is divided into data blocks, and the plurality of encoding processing units are input in batches and sequentially executed. The encoding parameters can be obtained, and then the image frame is encoded by using the encoding parameters. Since the encoding processing unit can perform the processing of the next batch of data blocks after the processing of the previous batch of data blocks is completed, at least two encoding processing units perform the processing synchronously in the time period when the plurality of encoding processing units perform the processing. Therefore, the processing of the plurality of batches of data blocks can be performed simultaneously. Compared with the sequential execution mode in units of video frames, the time consumed for determining the encoding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel operation mode of the plurality of encoding units reuses the resources of the hardware units, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0222] In addition, the multi-encoding processing unit processing scheme of the embodiment of the present application has strong versatility, can be copied to different encoding systems, and can be configured to be compatible with other processing processes. Therefore, the optimization processing of the plurality of processing processes is realized, and the processing efficiency of the system is improved.

[0223] Reference Figure 13 FIG. 12 shows a structure block diagram of a chip embodiment based on a field programmable logic gate array according to the twelfth embodiment of the present application, which includes a frame division unit, a plurality of encoding processing units executed in sequence, and an encoding unit;

[0224] The frame division unit 1201 is configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into the plurality of encoding processing units in batches.

[0225] The encoding processing unit 1202 is configured to correspondingly perform one or more steps in the determination process of the encoding parameters of the target image frame, and perform the processing of the next batch of first data blocks after the processing of the previous batch of first data blocks is completed.

[0226] The image frame encoding unit 1203 is configured to encode the target image frame according to the encoding parameters corresponding to the plurality of first data blocks, respectively.

[0227] According to the embodiment of the present application, a plurality of encoding processing units are used to perform the determination process of the encoding parameters, and each encoding processing unit performs one or more steps in the determination process of the encoding parameters. When the encoding parameters are calculated for an image frame, the image frame is divided into data blocks, and the plurality of encoding processing units are input in batches to perform the encoding processing in sequence. The encoding parameters can be obtained, and then the image frame is encoded by using the encoding parameters. Since the encoding processing unit can perform the processing of the next batch of data blocks after the processing of the previous batch of data blocks is completed, at least two encoding processing units perform the processing synchronously in a part of the time period in which the plurality of encoding processing units perform the processing. Therefore, the processing of the plurality of batches of data blocks can be performed simultaneously. Compared with the sequential execution mode in which the video frame is used as a unit, the time consumed for determining the encoding parameters can be greatly shortened, the efficiency of video encoding is improved, and the processing delay of the data blocks is reduced. Since the parallel operation mode of the plurality of encoding units reuses the resources of the hardware unit, the chip area can be reduced, the hardware cost can be reduced, and the hardware implementation is facilitated.

[0228] In the FPGA implementation scheme using the multiplexing structure, the memory and the calculation unit are multiplexed in a fine granularity, and the chip area occupied is only 1800 LUTs, which is convenient for hardware implementation.

[0229] In addition, the processing scheme of the plurality of encoding processing units in the embodiment of the present application has strong versatility, can be copied to different encoding systems, and the encoding processing units can be configured to be compatible with other processing processes, so that the optimization processing of the plurality of processing processes is implemented, and the processing efficiency of the system is improved.

[0230] For the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts are described in the part of the method embodiments.

[0231] Embodiments of the present disclosure can be implemented as a system configured in any appropriate hardware, firmware, software, or any combination thereof. Figure 15 An exemplary system (or device) 1300 that can be used to implement various embodiments described in this disclosure is schematically shown.

[0232] For one embodiment, Figure 15 An exemplary system 1300 is shown having one or more processors 1302, a system control module (chipset) 1304 coupled to at least one of the processors 1302, a system memory 1306 coupled to the system control module 1304, a non-volatile memory (NVM) / storage device 1308 coupled to the system control module 1304, one or more input / output devices 1310 coupled to the system control module 1304, and a network interface device 1312 coupled to the system control module 1306.

[0233] Processor 1302 can include one or more single-core or multi-core processors, which can include general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.) in any combination. In some embodiments, system 1300 can be capable of acting as a browser as described in embodiments of the present application.

[0234] In some embodiments, system 1300 can include one or more computer- readable media (e.g., system memory 1306 or NVM / storage 1308) having instructions and one or more processors 1302 that, in combination with the one or more computer-readable media, are configured to execute the instructions to implement modules to perform the actions described in the present disclosure.

[0235] For one embodiment, system control module 1304 can include a memory controller module to provide an interface to system memory 1306. The memory controller module can be a hardware module, a software module, and / or a firmware module.

[0236] System control module 1304 can include a memory controller module to provide an interface to system memory 1306. The memory controller module can be a hardware module, a software module, and / or a firmware module.

[0237] System memory 1306 can be used to, for example, load and store data and / or instructions for system 1300. For one embodiment, system memory 1306 can include any suitable volatile memory, such as suitable DRAM. In some embodiments, system memory 1306 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0238] For one embodiment, system control module 1304 can include one or more input / output controllers to provide an interface to NVM / storage 1308 and input / output device(s) 1310.

[0239] For example, NVM / storage 1308 can be used to store data and / or instructions. NVM / storage 1308 can include any suitable non-volatile memory (e.g., flash memory) and / or can include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0240] The NVM / storage 1308 can include a storage resource that is physically part of the device on which the system 1300 is installed or it can be accessible by the device, without being physically part of the device. For example, the NVM / storage 1308 can be accessed over a network via the input / output device(s) 1310.

[0241] The input / output device(s) 1310 can provide interfaces for the system 1300 to communicate with any other suitable device, including communication components, audio components, sensor components, and the like. The network interface 1312 can provide an interface for the system 1300 to communicate over one or more networks, and the system 1300 can communicate wirelessly with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as to access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, or 5G, or combinations thereof.

[0242] For one embodiment, at least one of the processor(s) 1302 can be packaged together with logic for one or more controllers of the system control module 1304, such as a memory controller module. For one embodiment, at least one of the processor(s) 1302 can be packaged together with logic for one or more controllers of the system control module 1304 to form a system in a package (SiP). For one embodiment, at least one of the processor(s) 1302 can be fabricated together with logic for one or more controllers of the system control module 1304 on the same die. For one embodiment, at least one of the processor(s) 1302 can be fabricated together with logic for one or more controllers of the system control module 1304 on the same die to form a system on a chip (SoC).

[0243] In various embodiments, the system 1300 can be, but is not limited to, a browser, a workstation, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet, a netbook, etc.). In various embodiments, the system 1300 can have more or less components, and / or different architectures. For example, in some embodiments, the system 1300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including touch screen displays), non- volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0244] If the display includes a touch panel, the display screen can be implemented as a touch screen display to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions, but also the duration and pressure associated with the touch or swipe operations.

[0245] The embodiments of the present application further provide a non-volatile readable storage medium, which stores one or more programs. When the one or more programs are applied to a terminal device, the terminal device can execute instructions of the steps of the methods in the embodiments of the present application.

[0246] In one example, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the processor implements the method in the embodiments of the present application.

[0247] In one example, a computer readable storage medium is also provided, which stores a computer program. When the processor executes the program, the processor implements the method in one or more embodiments of the present application.

[0248] According to an aspect of the embodiments of the present application, an image frame encoding method is provided. Example 1 includes:

[0249] The target image frame is divided into a plurality of first data blocks;

[0250] The plurality of first data blocks are input into a plurality of encoding processing units in batches and sequentially executed. After the encoding processing units execute the processing of a previous batch of first data blocks, the processing of a next batch of first data blocks is executed. The encoding processing units correspond to one or more steps in a determination process of an encoding parameter of the target image frame. At least two encoding processing units synchronously execute processing in a partial time period in which the plurality of encoding processing units execute processing.

[0251] The target image frame is encoded according to the encoding parameters corresponding to the plurality of first data blocks.

[0252] Example 2 includes the method of example 1, and the dividing of the target image frame into a plurality of first data blocks includes:

[0253] A specific region of the target image frame is divided into a plurality of first data blocks. The target region is a region of the target image frame in which content changes relative to a reference image frame.

[0254] Example 3 includes the method of example 1, wherein the inputting the plurality of first data blocks into the data processing unit in batches for data processing comprises:

[0255] inputting the plurality of first data blocks into the data processing unit one by one for data processing.

[0256] Example 4 includes the method of example 1, wherein the data processing unit corresponds to performing a parameter reading step, an information amount obtaining step, a data calculation step, and a data interaction step in a determination process of an encoding parameter of the target image frame.

[0257] Example 5 includes the method of example 1, wherein the information amount obtaining step comprises:

[0258] obtaining an initial information amount of the first data block from a corresponding memory according to an information amount storage address of the first data block;

[0259] obtaining an intra-frame information amount of the first data block according to a prediction cost and an encoding frame rate of the first data block;

[0260] determining a total information amount of the first data block according to the intra-frame information amount and the initial information amount.

[0261] Example 6 includes the method of example 5, wherein the parameter reading step comprises:

[0262] reading a prediction parameter of the first data block; the prediction parameter comprises a prediction direction and a prediction cost in an encoding process of a target image frame.

[0263] Example 7 includes the method of example 6, wherein the data calculation step comprises:

[0264] determining an encoding parameter for the first data block according to the total information amount, the prediction direction, and the prediction cost of the first data block.

[0265] Example 8 includes the method of example 5, wherein the parameter reading step comprises:

[0266] obtaining the information amount storage address of the first data block according to a position of the first data block in an image.

[0267] Example 9 includes the method of example 6, wherein the encoding processing unit further corresponds to performing one or more steps in a total information amount determination process of a reference image frame of the target image frame;

[0268] the information amount obtaining step further comprises:

[0269] determining a transfer information amount between the target image frame and the reference image frame according to the total information amount of the first data block;

[0270] The data calculating step further comprises:

[0271] obtaining an initial information amount of the second data block in the reference image frame;

[0272] adding the transferred information amount of the first data block to the initial information amount of the second data block in the reference image frame to obtain a total information amount of the second data block in the reference image frame.

[0273] Example 10 comprises the method of example 9, wherein the parameter reading step further comprises:

[0274] reading a prediction cost coefficient;

[0275] The determining the transferred information amount between the target image frame and the reference image frame according to the total information amount of the first data block comprises:

[0276] determining the transferred information amount of the first data block according to the total information amount of the first data block and the prediction cost coefficient.

[0277] Example 11 comprises the method of example 9, wherein the parameter reading step further comprises:

[0278] determining a position of a second data block in the reference image frame to which the information amount is to be added according to the position of the first data block in the target image frame and the motion vector.

[0279] Example 12 comprises the method of example 4, wherein the data interaction step comprises:

[0280] storing the encoding parameters to corresponding storage units for subsequent calling when encoding.

[0281] Example 13 comprises the method of example 9, wherein the data interaction step comprises:

[0282] transferring the total information amount of the second data block in the reference image frame to other encoding processing units associated for use.

[0283] According to an aspect of the embodiments of the present application, a video encoding method is provided, and example 14 comprises:

[0284] dividing a target image frame in a video into a plurality of first data blocks;

[0285] inputting the plurality of first data blocks into a plurality of encoding processing units in batches for sequential execution; the encoding processing units execute the processing of the next batch of first data blocks after the processing of the previous batch of first data blocks is completed, and the encoding processing units correspond to one or more steps in the determination process of the encoding parameters of the image frame, and at least two encoding processing units synchronously execute the processing in a partial time period in which the plurality of encoding processing units execute the processing.

[0286] encoding the target image frame according to the encoding parameters corresponding to the plurality of first data blocks;

[0287] obtaining an encoding result of the video according to the encoding result of the image frame.

[0288] According to an aspect of some embodiments of the present application, a video encoding method is provided, and example 15 includes:

[0289] dividing a specific region of a target image frame in a video into a plurality of first data blocks, the target region being a region of the target image frame that has content changes relative to a reference image frame;

[0290] inputting the plurality of first data blocks into four encoding processing units in batches for sequential execution; after the encoding processing units complete processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed, and the four encoding processing units respectively perform quantization parameter calculation in a determination process of an encoding parameter of an image frame according to reading steps, information quantity acquisition steps, quantization parameter and information quantity calculation steps, and quantization parameter and information quantity writing steps, and at least two of the four encoding processing units perform processing synchronously during a partial time period in which the four encoding processing units perform processing.

[0291] encoding the target image frame according to the encoding parameters corresponding to the plurality of first data blocks;

[0292] obtaining an encoding result of the video according to the encoding result of the image frame.

[0293] According to an aspect of some embodiments of the present application, a video encoding method is provided, and example 15 includes:

[0294] dividing a specific region of a target image frame in a video into a plurality of first data blocks, the target region being a region of the target image frame that has content changes relative to a reference image frame;

[0295] inputting the plurality of first data blocks into four encoding processing units in batches for sequential execution; after the encoding processing units complete processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed, and the four encoding processing units respectively perform quantization parameter calculation in a determination process of an encoding parameter of an image frame according to reading steps, information quantity acquisition steps, quantization parameter and information quantity calculation steps, and quantization parameter and information quantity writing steps, and at least two of the four encoding processing units perform processing synchronously during a partial time period in which the four encoding processing units perform processing.

[0296] According to an aspect of some embodiments of the present application, a video encoding method is provided, and example 15 includes:

[0297] obtaining at least one target video submitted through a software service client;

[0298] dividing image frames of the target video into a plurality of first data blocks;

[0299] The plurality of first data blocks are input into a plurality of encoding processing units in batches in sequence; after the encoding processing units complete processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed, the encoding processing units correspondingly perform one or more steps in a determination process of encoding parameters of the target image frame, and at least two encoding processing units synchronously perform processing in a partial time period in which the plurality of encoding processing units perform processing;

[0300] The target image frame is encoded according to the encoding parameters corresponding to the plurality of first data blocks respectively.

[0301] An encoding result of the target video is obtained according to encoding results of the plurality of image frames.

[0302] The software service client provides the encoding result of the target video.

[0303] According to an aspect of an embodiment of the present application, an image frame encoding system is provided, and example 18 includes a frame division module, a plurality of encoding processing units executed in sequence, and an encoding module.

[0304] The frame division unit is configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches.

[0305] The encoding processing unit is configured to correspondingly perform one or more steps in a determination process of encoding parameters of the target image frame, and after completing processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed, and at least two encoding processing units synchronously perform processing in a partial time period in which the plurality of encoding processing units perform processing.

[0306] The encoding unit is configured to encode the target image frame according to the encoding parameters corresponding to the plurality of first data blocks respectively.

[0307] According to an aspect of an embodiment of the present application, a chip based on a field programmable logic gate array is provided, and example 19 includes a frame division module, a plurality of encoding processing units executed in sequence, and an encoding module.

[0308] The frame division unit is configured to divide a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into a plurality of encoding processing units in batches.

[0309] The encoding processing units are configured to perform one or more steps in a determination process of encoding parameters corresponding to the target image frame, and perform processing of a next batch of first data blocks after processing of a previous batch of first data blocks is completed, and at least two of the encoding processing units perform processing synchronously during a partial time period in which the multiple encoding processing units perform processing.

[0310] The encoding units are configured to encode the target image frame according to the encoding parameters corresponding to the multiple first data blocks respectively.

[0311] According to an aspect of an embodiment of the present application, an electronic device is provided, and example 20 includes: a processor; and

[0312] A memory having stored thereon executable code that, when executed, causes the processor to perform the method of any of examples 1-17.

[0313] According to an aspect of an embodiment of the present application, one or more machine readable media having stored thereon executable code that, when executed, causes a processor to perform the method of any of examples 1-17.

[0314] Although certain embodiments are shown and described in some detail, various substitutions, modifications and / or equivalents, or implementations of the embodiments shown and described, or of the embodiments disclosed, are possible without departing from the scope of the present application. The present application is intended to cover any and all modifications or variations of the embodiments discussed herein. It is therefore evident that the embodiments described herein have a number of modifications and alterations in structure and function. However, it is expressly intended that such modifications and alterations are within the scope and spirit of the application. Accordingly, the embodiments described herein are to be considered as illustrative only of the principles of the application.

Claims

1. An image frame coding method characterized by, The method comprises the following steps: dividing a specific region of a target image frame into a plurality of first data blocks, the specific region being a region of the target image frame that has content changes relative to a reference image frame; wherein the specific region comprises a complex image region and a simple image region, and the size of a first data block corresponding to the complex image region is smaller than the size of a first data block corresponding to the simple image region; batching the plurality of first data blocks and inputting them into a plurality of encoding processing units that are executed in sequence; after the encoding processing units complete the processing of a previous batch of first data blocks, the processing of a next batch of first data blocks is performed; the encoding processing units correspondingly perform parameter reading steps, information quantity obtaining steps, data calculation steps, and data interaction steps in the determination process of the encoding parameters of the target image frame; at least two encoding processing units synchronously perform processing within a partial time period in which the plurality of encoding processing units perform processing, so that the processing of multiple batches of first data blocks is performed simultaneously; wherein one batch comprises one first data block, and the plurality of encoding processing units are parallelly operated; encoding the target image frame according to the encoding parameters corresponding to the plurality of first data blocks; wherein the information quantity obtaining step comprises: obtaining the initial information quantity of the first data block from a corresponding memory according to the information quantity storage address of the first data block; obtaining the intra-frame information quantity of the first data block according to the prediction cost and the encoding frame rate of the first data block; determining the total information quantity of the first data block according to the intra-frame information quantity and the initial information quantity; wherein the encoding processing units also correspondingly perform one or more steps in the total information quantity determination process of the reference image frame of the target image frame; the information quantity obtaining step further comprises: determining the transmission information quantity between the target image frame and the reference image frame according to the total information quantity of the first data block; the data calculation step further comprises: obtaining the initial information quantity of a second data block in the reference image frame; superimposing the transmission information quantity of the first data block on the initial information quantity of the second data block in the reference image frame to obtain the total information quantity of the second data block in the reference image frame.

2. The method of claim 1, wherein, the step of batching the plurality of first data blocks and inputting them into the plurality of encoding processing units that are executed in sequence comprises: inputting the plurality of first data blocks into the encoding processing units one by one for data processing.

3. The method of claim 1, wherein, the parameter reading step comprises: reading the prediction parameters of the first data block; the prediction parameters include the prediction direction and the prediction cost in the encoding process of the target image frame.

4. The method of claim 3, wherein, the data calculation step comprises: determining the encoding parameters corresponding to the first data block according to the total information quantity, the prediction direction, and the prediction cost of the first data block.

5. The method of claim 3, wherein, the parameter reading step comprises: obtaining the information quantity storage address of the first data block according to the position of the first data block in the image.

6. The method of claim 1, wherein, the parameter reading step further comprises: reading the prediction cost coefficient. the step of determining the transmission information quantity between the target image frame and the reference image frame according to the total information quantity of the first data block comprises: determining the transmission information quantity of the first data block according to the total information quantity and the prediction cost coefficient of the first data block.

7. The method of claim 1, wherein, The parameter reading step further comprises: According to the position of the first data block in the target image frame and the motion vector, the position of a second data block of the to-be-overlaid information in the reference image frame is determined.

8. The method of claim 3, wherein, The data interaction step comprises: The encoding parameters are stored in corresponding storage units for subsequent calling during encoding.

9. The method of claim 1, wherein, The data interaction step comprises: The total information amount of the second data block in the reference image frame is transmitted to other encoding processing units associated with use.

10. A method of video coding, the method comprising: Comprise: A specific region of a target image frame in a video is divided into a plurality of first data blocks, the specific region being a region of the target image frame that has content changes relative to a reference image frame; wherein the specific region comprises a complex image region and a simple image region, the size of the first data block corresponding to the complex image region being smaller than the size of the first data block corresponding to the simple image region; The plurality of first data blocks are input into a plurality of encoding processing units in batches for sequential execution; after the encoding processing units execute the processing of a previous batch of first data blocks, the processing of a next batch of first data blocks is executed, and the encoding processing units correspondingly execute a parameter reading step, an information amount obtaining step, a data calculation step, and a data interaction step in a determination process of encoding parameters of the target image frame; at least two encoding processing units synchronously execute processing in a partial time period in which the plurality of encoding processing units execute processing, so that the processing of multiple batches of first data blocks is performed simultaneously; wherein one batch comprises one first data block, and the plurality of encoding processing units are in parallel operation; According to the encoding parameters corresponding to the plurality of first data blocks, the target image frame is encoded; According to the encoding result of the image frame, an encoding result of the video is obtained; The information amount obtaining step comprises: According to the information amount storage address of the first data block, the initial information amount of the first data block is obtained from a corresponding memory; According to the prediction cost and the encoding frame rate of the first data block, the intra-frame information amount of the first data block is obtained; According to the intra-frame information amount and the initial information amount, the total information amount of the first data block is determined. The encoding processing unit also correspondingly executes one or more steps in a total information amount determination process of a reference image frame of the target image frame; the information amount obtaining step further comprises: According to the total information amount of the first data block, the transmission information amount between the target image frame and the reference image frame is determined. The data calculation step further comprises: The initial information amount of a second data block in the reference image frame is obtained; The transmission information amount of the first data block is overlaid to the initial information amount of the second data block in the reference image frame to obtain the total information amount of the second data block in the reference image frame.

11. A method of video coding, the method comprising: Comprise: A specific region of a target image frame in a video is divided into a plurality of first data blocks, the specific region being a region of the target image frame that has content changes relative to a reference image frame; wherein the specific region comprises a complex image region and a simple image region, the size of the first data block corresponding to the complex image region being smaller than the size of the first data block corresponding to the simple image region; The plurality of first data blocks are input into four encoding processing units in batches in sequence; after the encoding processing units complete processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed; the four encoding processing units respectively perform calculation of a quantization parameter in a process of determining an encoding parameter of an image frame according to a reading step, an information quantity obtaining step, a quantization parameter and information quantity calculation step, and a quantization parameter and information quantity writing step; at least two encoding processing units synchronously perform processing in a partial time period in which the four encoding processing units perform processing, so that processing of multiple batches of first data blocks is performed simultaneously; wherein one batch includes one first data block, and the plurality of encoding processing units are in parallel operation; The target image frame is encoded according to the encoding parameters corresponding to the four first data blocks; An encoding result of the video is obtained according to an encoding result of the image frame; The information quantity obtaining step includes: An initial information quantity of the first data block is obtained from a corresponding memory according to an information quantity storage address of the first data block; An intra-frame information quantity of the first data block is obtained according to a prediction cost and an encoding frame rate of the first data block; A total information quantity of the first data block is determined according to the intra-frame information quantity and the initial information quantity; The encoding processing unit also corresponds to performing one or more steps in a process of determining a total information quantity of a reference image frame of the target image frame; the information quantity obtaining step further includes: A transfer information quantity between the target image frame and the reference image frame is determined according to the total information quantity of the first data block; The data calculation step further includes: An initial information quantity of a second data block in the reference image frame is obtained; The transfer information quantity of the first data block is added to the initial information quantity of the second data block in the reference image frame to obtain a total information quantity of the second data block in the reference image frame.

12. An image frame processing method, characterized by, The method includes: A specific region of a target image frame is divided into a plurality of first data blocks, the specific region being a region in which content of the target image frame changes relative to a reference image frame; wherein the specific region includes a complex image region and a simple image region, a size of a first data block corresponding to the complex image region being smaller than a size of a first data block corresponding to the simple image region; The plurality of first data blocks are input into a plurality of encoding processing units in batches in sequence; after the encoding processing units complete processing of a previous batch of first data blocks, processing of a next batch of first data blocks is performed; the encoding processing units correspond to performing a parameter reading step, an information quantity obtaining step, a data calculation step, and a data interaction step in a processing process of the target image frame; at least two encoding processing units synchronously perform processing in a partial time period in which the plurality of encoding processing units perform processing, so that processing of multiple batches of first data blocks is performed simultaneously; wherein one batch includes one first data block, and the plurality of encoding processing units are in parallel operation; The information quantity obtaining step includes: An initial information quantity of the first data block is obtained from a corresponding memory according to an information quantity storage address of the first data block; According to the prediction cost and the encoding frame rate of the first data block, an intra-frame information amount of the first data block is obtained; According to the intra-frame information amount and the initial information amount, a total information amount of the first data block is determined; The encoding processing unit also corresponds to perform one or more steps in the total information amount determination process of the reference image frame of the target image frame; the information amount obtaining step further comprises: According to the total information amount of the first data block, a transfer information amount between the target image frame and the reference image frame is determined; The data calculation step further comprises: An initial information amount of a second data block in the reference image frame is obtained; The transfer information amount of the first data block is superimposed on the initial information amount of the second data block in the reference image frame to obtain a total information amount of the second data block in the reference image frame.

13. An image frame processing method, characterized by, Applied to a software service platform, comprising: At least one target video submitted through a software service client is obtained; A specific region of a target image frame in the video is divided into a plurality of first data blocks, the specific region being a region in which the target image frame has content changes relative to a reference image frame; wherein the specific region includes a complex image region and a simple image region, the size of the first data block corresponding to the complex image region being smaller than the size of the first data block corresponding to the simple image region; The plurality of first data blocks are input into a plurality of encoding processing units in batches for sequential execution; after the encoding processing unit completes the processing of a previous batch of first data blocks, it executes the processing of a next batch of first data blocks, and the encoding processing unit corresponds to perform a parameter reading step, an information amount obtaining step, a data calculation step, and a data interaction step in a determination process of an encoding parameter of the target image frame, at least two encoding processing units synchronously execute processing in a partial time period in which the plurality of encoding processing units execute processing, so that the processing of multiple batches of first data blocks is performed simultaneously; wherein one batch includes one first data block, and the plurality of encoding processing units are parallelly run; According to the encoding parameters corresponding to the plurality of first data blocks, the target image frame is encoded; According to the encoding results of the plurality of image frames, an encoding result of the target video is obtained; The software service client provides the encoding result of the target video based on the software service client; The information amount obtaining step comprises: According to the information amount storage address of the first data block, the initial information amount of the first data block is obtained from the corresponding memory; According to the prediction cost and the encoding frame rate of the first data block, an intra-frame information amount of the first data block is obtained; According to the intra-frame information amount and the initial information amount, a total information amount of the first data block is determined; The encoding processing unit also corresponds to perform one or more steps in the total information amount determination process of the reference image frame of the target image frame; the information amount obtaining step further comprises: According to the total information amount of the first data block, a transfer information amount between the target image frame and the reference image frame is determined; The data calculation step further comprises: An initial information amount of a second data block in the reference image frame is obtained; The transfer information amount of the first data block is superimposed on the initial information amount of the second data block in the reference image frame to obtain a total information amount of the second data block in the reference image frame. Superimpose the transfer information amount of the first data block to the initial information amount of a second data block in a reference image frame to obtain the total information amount of the second data block in the reference image frame.

14. An image frame encoding system characterized by comprising: The frame division unit, the plurality of encoding processing units executed in sequence, and the encoding unit are included. The frame division unit is configured to divide a specific region of a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into the plurality of encoding processing units in batches; wherein the specific region is a region of the target image frame that has content changes relative to a reference image frame, the specific region includes a complex image region and a simple image region, and a size of a first data block corresponding to the complex image region is smaller than a size of a first data block corresponding to the simple image region. The encoding processing unit is configured to correspondingly execute a parameter reading step, an information amount obtaining step, a data calculation step, and a data interaction step in a determination process of an encoding parameter of the target image frame, and execute processing of a next batch of first data blocks after processing of a previous batch of first data blocks is completed, and at least two encoding processing units synchronously execute processing in a partial time period in which the plurality of encoding processing units execute processing, so that processing of multiple batches of first data blocks is simultaneously performed; wherein one batch includes one first data block, and the plurality of encoding processing units are in parallel operation. The encoding unit is configured to encode the target image frame according to the encoding parameters corresponding to the plurality of first data blocks, respectively. The information amount obtaining step includes: obtaining the initial information amount of the first data block from a corresponding memory according to an information amount storage address of the first data block; obtaining the intra-frame information amount of the first data block according to a prediction cost and an encoding frame rate of the first data block; determining the total information amount of the first data block according to the intra-frame information amount and the initial information amount; The encoding processing unit also correspondingly executes one or more steps in a total information amount determination process of a reference image frame of the target image frame; and the information amount obtaining step further includes: determining the transfer information amount between the target image frame and the reference image frame according to the total information amount of the first data block; The data calculation step further includes: obtaining the initial information amount of a second data block in a reference image frame; superimposing the transfer information amount of the first data block to the initial information amount of the second data block in the reference image frame to obtain the total information amount of the second data block in the reference image frame.

15. A field programmable gate array based chip, comprising: The frame division unit, the plurality of encoding processing units executed in sequence, and the encoding unit are included. The frame division unit is configured to divide a specific region of a target image frame into a plurality of first data blocks, and input the plurality of first data blocks into the plurality of encoding processing units in batches; wherein the specific region is a region of the target image frame that has content changes relative to a reference image frame, the specific region includes a complex image region and a simple image region, and a size of a first data block corresponding to the complex image region is smaller than a size of a first data block corresponding to the simple image region. The encoding processing unit is configured to perform parameter reading, information quantity obtaining, data calculation, and data interaction in the determination process of the encoding parameters of the target image frame, and perform processing of the next batch of first data blocks after processing of the previous batch of first data blocks is completed. At least two encoding processing units perform processing synchronously in a partial time period in which the multiple encoding processing units perform processing, so that processing of multiple batches of first data blocks is performed simultaneously. One batch includes one first data block, and the multiple encoding processing units are parallel. The encoding unit is configured to encode the target image frame according to the encoding parameters corresponding to the multiple first data blocks, respectively. The information quantity obtaining step includes: obtaining the initial information quantity of the first data block from the corresponding memory according to the information quantity storage address of the first data block; obtaining the intra-frame information quantity of the first data block according to the prediction cost and the encoding frame rate of the first data block; determining the total information quantity of the first data block according to the intra-frame information quantity and the initial information quantity; The encoding processing unit also performs one or more steps in the total information quantity determination process of the reference image frame of the target image frame. The information quantity obtaining step also includes: determining the transfer information quantity between the target image frame and the reference image frame according to the total information quantity of the first data block; The data calculation step also includes: obtaining the initial information quantity of the second data block in the reference image frame; adding the transfer information quantity of the first data block to the initial information quantity of the second data block in the reference image frame to obtain the total information quantity of the second data block in the reference image frame.

16. An electronic device, comprising: The processor and the memory having stored thereon executable code that, when executed, cause the processor to perform the method of any one of claims 1-13.

17. A machine-readable medium having stored thereon executable code that, when executed, cause a processor to perform the method of any one of claims 1-13. ​ ​ ​

Images