Video processing method, video processing device, electronic device and storage medium

By performing data block division and complexity analysis on the target frame image of the video to be encoded, the appropriate encoding mode is determined, which solves the problem of low video encoding efficiency in the prior art and improves the video encoding speed and efficiency.

CN115002452BActive Publication Date: 2024-11-26TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110228518.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-02
Publication Date
2024-11-26
Estimated Expiration
2041-03-02

AI Technical Summary

Technical Problem

It is difficult for the prior art to determine a suitable encoding mode for video encoding, which affects the efficiency of video encoding.

Method used

By dividing the data blocks to be encoded in the target frame image of the video to be encoded into N sub-data blocks, the complexity index information of each sub-data block is obtained, and the complexity parameters between the sub-data blocks are determined based on these index information. If the block encoding or complete block encoding conditions are met, the corresponding encoding mode will be used to encode the encoded data blocks.

Benefits of technology

It effectively improves video encoding speed and encoding efficiency, is suitable for video encoding in different scenarios, ensuring the optimal balance between encoding speed and code rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115002452B_ABST
    Figure CN115002452B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses a video processing method, a video processing device, an electronic device and a storage medium, and the video processing method includes: determining a data block to be encoded in a target frame image of a video to be encoded; dividing the data block to be encoded into N sub-data blocks, and obtaining complexity index information corresponding to each sub-data block; determining the complexity parameter between the N sub-data blocks according to each complexity index information; if the complexity parameter meets the block encoding condition, the block encoding mode is used as the encoding mode of the data block to be encoded, and the target sub-data block in the N sub-data blocks is determined as a new data block to be encoded according to the block encoding mode; if the complexity parameter meets the complete block encoding condition, the complete block encoding mode is used as the encoding mode of the data block to be encoded, and the image encoding processing is performed on the data block to be encoded according to the complete block encoding mode. By adopting the embodiment of the present application, a suitable encoding mode can be determined for video encoding.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field, in particular to the technical field, and specifically to a video processing method, a video processing device, an electronic device and a storage medium. Background Art

[0002] Video coding technology refers to the technology of compressing and encoding videos according to a certain coding mode; compressing and encoding videos can effectively save video storage space and improve video transmission efficiency. At present, video coding technology has been widely used in scenarios such as video conversations and video on demand; for example, using video coding technology to compress and encode conversation videos can effectively improve video conversation efficiency; using video coding technology to compress and encode on-demand videos can effectively improve video on demand efficiency.

[0003] In the process of video compression encoding, using a more appropriate encoding mode can speed up the entire video encoding process and improve video encoding efficiency. Therefore, how to determine the appropriate encoding mode for video encoding has become a hot issue in current research. Summary of the invention

[0004] The embodiments of the present application provide a video processing method, a video processing device, an electronic device and a storage medium, which can determine a suitable encoding mode for video encoding.

[0005] On the one hand, an embodiment of the present application provides a video processing method, the video processing method comprising:

[0006] Determining a data block to be encoded in a target frame image of a video to be encoded;

[0007] Divide the data block to be encoded into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block, where N is an integer greater than or equal to 2;

[0008] Determine the complexity parameters between the N sub-data blocks according to each complexity indicator information;

[0009] If the complexity parameter satisfies the block coding condition, the block coding mode is used as the coding mode of the data block to be coded, and the target sub-data block in the N sub-data blocks is determined as a new data block to be coded according to the block coding mode;

[0010] If the complexity parameter satisfies the complete block coding condition, the complete block coding mode is used as the coding mode of the data block to be coded, and image coding processing is performed on the data block to be coded according to the complete block coding mode.

[0011] On the other hand, an embodiment of the present application provides a video processing device, the video processing device comprising:

[0012] A determination unit, used to determine a data block to be encoded in a target frame image of a video to be encoded;

[0013] A processing unit, used to divide the to-be-encoded data block into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block, where N is an integer greater than or equal to 2;

[0014] The processing unit is further used to determine the complexity parameters between the N sub-data blocks according to each complexity indicator information;

[0015] The processing unit is further configured to use the block coding mode as the coding mode of the data block to be coded if the complexity parameter satisfies the block coding condition, and determine the target sub-data block in the N sub-data blocks as a new data block to be coded according to the block coding mode;

[0016] The processing unit is further configured to, if the complexity parameter satisfies a complete block coding condition, use the complete block coding mode as the coding mode of the data block to be coded, and perform image coding processing on the data block to be coded according to the complete block coding mode.

[0017] In one embodiment, the processing unit is further configured to:

[0018] If the complexity parameter satisfies the coding decision condition, then M associated data blocks associated with the data block to be coded are determined in the target frame image, where M is a positive integer;

[0019] Obtaining associated block index parameters of the M associated data blocks, and determining a precoding order of the to-be-coded data blocks according to the associated block index parameters;

[0020] The data block to be encoded is pre-encoded according to the determined pre-encoding order, and the encoding mode for the data block to be encoded is determined.

[0021] In one embodiment, the associated block indicator parameter includes: a first number of associated data blocks whose coding mode is a block coding mode among the M associated data blocks;

[0022] The processing unit is used to determine the precoding order of the to-be-coded data blocks according to the associated block indicator parameter, specifically to:

[0023] If the first number is greater than or equal to the first threshold, determining that the precoding order of the data block to be encoded is the first precoding order;

[0024] A processing unit is used to perform precoding processing on the data block to be coded according to the determined precoding order, and when determining the coding mode for the data block to be coded, specifically to:

[0025] Obtaining sub-block index parameters of N sub-data blocks according to a first precoding order;

[0026] The encoding mode for the data block to be encoded is determined according to the sub-block index parameter.

[0027] In one embodiment, the sub-block indicator parameter includes a second number of sub-data blocks whose coding mode is a block coding mode in the N sub-data blocks, and the processing unit is used to determine the coding mode for the coded data block according to the sub-block indicator parameter, specifically for:

[0028] If the second number is greater than or equal to the second threshold, the block coding mode is used as the coding mode of the data block to be encoded.

[0029] In one embodiment, when the processing unit is used to determine the encoding mode for the data block to be encoded according to the sub-block indicator parameter, it is also used to:

[0030] If the second number is less than the second threshold, image coding is performed on the data block to be coded to obtain coding information of the data block to be coded;

[0031] Obtaining coding information of N sub-data blocks, where the coding information of the N sub-data blocks is obtained by performing image coding processing on each of the N sub-data blocks;

[0032] A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

[0033] In one embodiment, the processing unit is used to determine the precoding order of the data blocks to be encoded according to the associated block indicator parameter, specifically to: if the first number is less than the first threshold, determine the precoding order of the data blocks to be encoded as the second precoding order;

[0034] A processing unit is used to perform precoding processing on the data block to be coded according to the determined precoding order, and when determining the coding mode for the data block to be coded, specifically to:

[0035] According to the second pre-coding sequence, image coding is performed on the data block to be coded, and a coding mode for the data block to be coded is determined according to coding information of the data block to be coded obtained by coding.

[0036] In one embodiment, the coding information of the data block to be coded includes a coding distortion parameter of the data block to be coded; the processing unit is used to determine the coding mode of the data block to be coded according to the coding information of the data block to be coded obtained by coding, specifically to:

[0037] The coding parameter is calculated according to the coding distortion parameter of the data block to be coded and the quantization parameter for quantizing the data block to be coded;

[0038] If the encoding parameter is less than the third threshold, the complete block encoding mode is used as the encoding mode of the data block to be encoded;

[0039] If the encoding parameter is greater than or equal to the third threshold, performing image encoding processing on each of the N sub-data blocks respectively to obtain encoding information of the N sub-data blocks;

[0040] A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

[0041] In one embodiment, the coding information of the data block to be coded includes the precoding evaluation parameters of the data block to be coded; the coding information of the N sub-data blocks includes the precoding evaluation parameters of the N sub-data blocks, and the precoding evaluation parameters of the N sub-data blocks are calculated according to the precoding evaluation parameters of each sub-data block in the N sub-data blocks; the processing unit is used to determine the data block processing mode for the data block to be coded according to the coding information of the data block to be coded and the coding information of the N sub-data blocks, specifically for:

[0042] If the precoding evaluation parameter of the data block to be encoded is greater than or equal to the precoding evaluation parameters of the N sub-data blocks, the block coding mode is used as the coding mode of the data block to be encoded;

[0043] If the precoding estimation parameter of the data block to be encoded is less than the precoding estimation parameters of the N sub-data blocks, the complete block encoding mode is used as the encoding mode of the data block to be encoded.

[0044] In one embodiment, the processing unit is further configured to:

[0045] Obtaining the complexity parameter of the data block to be encoded;

[0046] If the complexity parameter of the to-be-encoded data block meets the overall encoding condition, the complete block encoding mode is used as the encoding mode of the to-be-encoded data block, and image encoding processing is performed on the to-be-encoded data block according to the complete block encoding mode;

[0047] If the complexity parameter of the data block to be encoded does not meet the overall encoding condition, the step of dividing the data block to be encoded into N sub-data blocks is triggered.

[0048] In one embodiment, the complexity parameter of the data block to be encoded includes: any one or more of a distortion estimation parameter of the data block to be encoded, a spatial information parameter of the data block to be encoded, and a temporal information parameter of the data block to be encoded;

[0049] The complexity index information of the i-th sub-data block among N sub-data blocks includes: any one or more of the distortion estimation parameters of the i-th sub-data block, the spatial information parameters of the i-th sub-data block, and the temporal information parameters of the i-th sub-data block, where i is a positive integer less than or equal to N.

[0050] In one embodiment, after the processing unit is used to determine the target sub-data block in the N sub-data blocks as the new data block to be encoded according to the block encoding mode, it is also used to:

[0051] The target sub-data block is further divided into N sub-data blocks;

[0052] Determine the encoding mode of the target sub-data block according to the complexity parameters between the N sub-data blocks obtained by dividing the target sub-data block;

[0053] The determined encoding mode of the target sub-data block includes: a block encoding mode of further dividing the target sub-data block into N sub-data blocks, or a complete block encoding mode of directly performing image encoding processing on the target sub-data block.

[0054] Accordingly, an embodiment of the present application further provides an electronic device, the electronic device comprising: a memory and a processor;

[0055] Memory for storing computer programs;

[0056] The processor is used to execute the computer program to implement the above-mentioned video processing method.

[0057] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device implements the above-mentioned video processing method.

[0058] Accordingly, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device implements the above-mentioned video processing method.

[0059] The embodiment of the present application divides the data block to be encoded in the target frame image of the video to be encoded into N sub-data blocks, and determines the encoding mode of the data block to be encoded according to the complexity parameters between the N sub-data blocks. It is possible to determine a suitable encoding mode for the data block to be encoded in the target frame image, effectively improve the encoding speed of the video to be encoded, and further improve the encoding efficiency of the video to be encoded. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 is a schematic diagram of a recursive partitioning process of a data block provided in an embodiment of the present application;

[0062] Figure 2 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0063] Figure 3 is a schematic diagram of an associated data block provided in an embodiment of the present application;

[0064] Figure 4 is a schematic diagram of the architecture of a video processing system provided in an embodiment of the present application;

[0065] Figure 5 It is a flowchart of a video processing method provided by an embodiment of the present application;

[0066] Figure 6 is a flowchart of another video processing method provided in an embodiment of the present application;

[0067] Figure 7 is a flowchart of another video processing method provided in an embodiment of the present application;

[0068] Figure 8a is a flowchart of another video processing method provided in an embodiment of the present application;

[0069] Figure 8b is a flowchart of another video processing method provided in an embodiment of the present application;

[0070] Fig. 9 is a structural schematic diagram of a video processing device provided in an embodiment of the present application;

[0071] Fig.10 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] The embodiments of the present application involve cloud technology, and the embodiments of the present application can use the computing services provided by cloud technology to realize the video processing process. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing, and sharing of data. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the application of cloud computing business model, which can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0074] Among them, cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. As a basic capability provider of cloud computing, a cloud computing resource pool (referred to as cloud platform) will be established, generally referred to as IaaS (Infrastructure as a Service) platform, and various types of virtual resources will be deployed in the cloud computing resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices. According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer. SaaS can also be deployed directly on IaaS. PaaS is a platform for software (such as databases, web containers, etc.) to run. SaaS refers to a variety of business software (such as web portals, SMS mass senders, etc.). Generally speaking, SaaS and PaaS are upper layers relative to IaaS. Cloud computing also refers to the delivery and use model of IT (Internet Technology) infrastructure, which means obtaining required resources through the network in an on-demand, easily scalable manner; in a broad sense, cloud computing refers to the delivery and use model of services, which means obtaining required services through the network in an on-demand, easily scalable manner. This service can be IT and software, Internet-related, or other services. Cloud computing is the product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0075] Cloud computing can be used in the field of cloud conferencing. Cloud conferencing is an efficient, convenient and low-cost conference format based on cloud computing technology. Users only need to use the Internet interface to perform simple and easy-to-use operations to quickly and efficiently share voice, data files and videos with teams and customers around the world. The cloud conferencing service provider helps users to operate complex technologies such as data transmission and processing in the conference. At present, domestic cloud conferencing mainly focuses on service content based on the SaaS model, including telephone, network, video and other service forms. Video conferencing based on cloud computing is called cloud conferencing. In the era of cloud conferencing, data transmission, processing and storage are all handled by the computer resources of video conferencing manufacturers. Users no longer need to purchase expensive hardware and install cumbersome software. They only need to open the browser and log in to the corresponding interface to conduct efficient remote conferencing. The cloud conferencing system supports dynamic cluster deployment of multiple servers and provides multiple high-performance servers, which greatly improves the stability, security and availability of the conference. In recent years, video conferencing has been welcomed by many users because it can greatly improve communication efficiency, continuously reduce communication costs, and bring about internal management upgrades. It has been widely used in various fields such as government, military, transportation, finance, operators, education, and enterprises. There is no doubt that after the use of cloud computing, video conferencing will be more attractive in terms of convenience, speed and ease of use, and will surely trigger a new wave of video conferencing applications.

[0076] The embodiments of the present application relate to video encoding technology. Video encoding technology can also be called video compression technology, which refers to the technology of compressing and encoding the video to be encoded according to a certain encoding mode. The video to be encoded is a frame image sequence composed of continuous frame images. Due to the visual persistence effect of the human eye, when the frame image sequence is played at a certain rate, what we see is a video with continuous action; since the similarity between continuous frame images is extremely high, there is a large amount of redundant information within each frame image and between continuous frame images. Therefore, before storing or transmitting the video, it is often necessary to use video encoding technology to compress and encode the video to be encoded, and remove the redundant information of the video to be encoded in the space, time and other dimensions, so as to save storage space and improve video transmission efficiency.

[0077] In the actual encoding process of the video to be encoded, each frame image in the video to be encoded is often divided into multiple data blocks to be encoded. The encoding mode refers to the processing method of the data block to be encoded, that is, each data block to be encoded in the frame image is processed according to the determined encoding mode, so as to achieve compression encoding of the video to be encoded. For example, the encoding mode of the data block to be encoded is to directly perform image encoding processing on the data block to be encoded; another example is that the encoding mode of the data block to be encoded is to recursively divide the data block to be encoded, so-called recursive division means dividing the data block to be encoded into N sub-data blocks, and then taking each sub-data block in the N sub-data blocks as a new data block to be encoded, re-determining the encoding mode of the new data block to be encoded, and then processing the new data block to be encoded according to the determined encoding mode, and N is an integer greater than or equal to 2.

[0078] Combine the following Figure 1 The recursive partitioning process involved in the embodiment of the present application is introduced. Figure 1 is a schematic diagram of a recursive partitioning process of a data block provided in an embodiment of the present application, such as Figure 1 As shown, the frame image is divided into multiple data blocks to be encoded. The data blocks to be encoded can be directly subjected to image encoding processing (for example, the encoding mode of the data block to be encoded 10 indicates that the data block to be encoded 10 is directly subjected to image encoding processing), or the data blocks to be encoded can be recursively divided. Figure 1 An exemplary recursive partitioning process is shown as follows: the data block 11 to be encoded is divided into four sub-data blocks of the same size, namely sub-data block 111, sub-data block 112, sub-data block 113 and sub-data block 114; sub-data block 111 is used as a new data block to be encoded, and the encoding mode of sub-data block 111 is re-determined, and the determined encoding mode indicates that image encoding processing is directly performed on sub-data block 111; and so on, sub-data block 112, sub-data block 113 and sub-data block 114 are processed in a manner similar to sub-data block 111. Figure 1 Another exemplary recursive partitioning process shown is as follows: the data block 12 to be encoded is divided into four sub-data blocks of the same size, namely, sub-data block 121, sub-data block 122, sub-data block 123 and sub-data block 124; sub-data block 121 is used as a new data block to be encoded, and the encoding mode for sub-data block 121 is re-determined, and the determined encoding mode indicates that the new data block to be encoded (i.e., sub-data block 121) is further divided into four sub-data blocks of the same size, namely, sub-data block 1211, sub-data block 1212, sub-data block 1213 and sub-data block 1214; and so on, sub-data block 122, sub-data block 123, sub-data block 124 and sub-data blocks 1211, sub-data block 1212, sub-data block 1213, sub-data block 1214 are processed in a manner similar to sub-data block 121.

[0079] Using a suitable coding mode to process the coded data block can greatly improve the coding speed of the coded video, thereby improving the coding efficiency of the coded video. Based on this, an embodiment of the present application provides a video processing scheme, which provides two different coding modes for the coded data blocks, namely a top-down coding mode and a bottom-up coding mode; both coding modes can determine the optimal division method for each coded data block in the frame image; the optimal division method can be understood as that after the coded data block is divided according to the data block division method, the coding quality when encoding the divided sub-data blocks is better and the coding bit rate is lower; the coding bit rate can refer to the amount of data transmitted per unit time (for example, 1 second) when the coded video data stream is transmitted. The more data transmitted per unit time, the higher the coding bit rate. The top-down coding mode and the bottom-up coding mode are introduced in detail below:

[0080] (1) Top-down coding mode

[0081] In the process of encoding the video to be encoded, each data block to be encoded in the frame image is divided from top to bottom and recursively. Specifically, the top-down and recursive division may refer to the process of dividing the data block from the maximum size of the frame image to the minimum size of the data block in the frame image until the optimal division method of the data block is found. For example, the maximum size of the data block in the frame image is 64px (P ixel, pixel) × 64px, and the minimum size of the data block in the frame image is 4px × 4px; the maximum size of the data block is 64px × 64px, and the minimum size of the data block is 4px × 4px. It is only used for example and does not constitute a limitation on the embodiments of the present application. For example, the maximum size of the data block can also be 128px × 128px, and the minimum size of the data block can also be 8px × 8px, etc.; the 64px × 64px data block can be divided into 4 32px × 32px data blocks; the rate-distortion parameter (Rate-Distortion Cost, RDCost), and calculate the sum of the RDCost of four 32px×32px data blocks (that is, calculate the first sum RDCost of four 32px×32px data blocks), and calculate the RDCost obtained by directly encoding the 64px×64px data block; if the first sum RDCost is greater than the RDCost of the 64px×64px data block, it is determined that the division size of the data block is 64px×64px, that is, there is no need to divide the 64px×64px data block; if the first sum RDCost is less than or equal to the RDCost of the 64px×64px data block, it is determined that the division size of the data block is 32px×32px, and the 32px×32px data block can be further divided into four 16px×16px data blocks for further judgment, and so on, until the optimal division method of the data block is found.

[0082] The top-down encoding mode can usually determine the optimal division method more accurately. For data blocks with low scene complexity, the optimal division size of the data block is relatively large. Therefore, for data blocks with low scene complexity, the top-down data block division strategy can quickly determine the optimal division method of the data block. For data blocks with high scene complexity, the optimal division size of the data block is relatively small. Therefore, for data blocks with high scene complexity, the process of determining the optimal division method of the data block through the top-down data block division strategy requires a lot of time cost, which in turn affects the encoding speed of the data block.

[0083] The optimal partition size of a data block can be understood as when the size of the data block obtained by partitioning is the size, the encoding quality of the data block obtained by partitioning is better and the encoding bit rate is lower; generally speaking, the optimal partition size corresponding to the data block with lower scene complexity is larger, and the optimal partition size corresponding to the data block with higher scene complexity is smaller. The scene complexity of a data block can be measured by using methods such as spatial information (SI) or temporal information. Spatial information can be used to characterize the amount of spatial details of a data block. The more elements a data block contains, the higher the value of the spatial information of the data block, and the higher the scene complexity of the data block; for example, data block A contains 5 elements (a cat, a dog, a tree, a flower and a sun), and data block B contains 2 elements (a cat and a dog). The elements contained in data block A are more than those contained in data block B, then the value of the spatial information of data block A is higher than that of the spatial information of data block B, and the scene complexity of data block A is higher than that of data block B. The time information can be used to characterize the time variation of the data block. The higher the degree of motion of the data block to be encoded in the target frame image currently being processed relative to the reference data block in the reference frame image of the target frame image, the larger the value of the time information of the data block to be encoded, and the higher the scene complexity of the data block to be encoded. The reference frame image is a frame image whose encoding order is before the target frame image in the frame image sequence, and the position of the data block to be encoded in the target frame image is the same as the position of the reference data block in the reference frame image. For example, the data block to be encoded contains an element (such as a car element), and the reference data block also contains the element. The greater the displacement of the car element in the data block to be encoded relative to the car element in the reference data block, the higher the degree of motion of the data block to be encoded in the target frame image relative to the reference data block in the reference frame image, the larger the value of the time information of the data block to be encoded, and the higher the scene complexity of the data block to be encoded.

[0084] (2) Bottom-up coding mode

[0085] In the process of encoding the video, each data block to be encoded in the frame image is divided bottom-up and recursively. Specifically, the bottom-up and recursive division can refer to the process of dividing the data block from the minimum size of the frame image to the maximum size of the data block in the frame image until the optimal division method of the data block is found. For example, the minimum size of the data block in the frame image is 4px×4px, and the maximum size of the data block in the frame image is 64px×64px; encode each of the 4 4px×4px data blocks, and then encode the 8px×8px data block composed of the 4 4px×4px data blocks, and calculate the RDCost obtained by encoding each of the 4 4px×4px data blocks and the RDCo of the 4 4px×4px data blocks. st (i.e., the second sum RDCost of 4 4px×4px data blocks is calculated); and RDCost is calculated by encoding the 8px×8px data block; if the second sum RDCost is less than or equal to the RDCost of the 8px×8px data block, it is determined that the data block is divided into 4px×4px in size; if the second sum RDCost is greater than the RDCost of the 8px×8px data block, it is determined that the data block is divided into 8px×8px in size, and further judgment can be made on the 16px×16px data block composed of 4 8px×8px data blocks, and so on, until the optimal division method of the data block is found.

[0086] Bottom-up coding mode can usually determine the optimal division method more accurately. For data blocks with high scene complexity, the optimal division size of the data block is relatively small. Therefore, for data blocks with high scene complexity, the bottom-up coding mode can quickly determine the optimal division method of the data block; for data blocks with low scene complexity, the optimal division size of the data block is relatively large. Therefore, for data blocks with low scene complexity, the process of determining the optimal division method of the data block through the bottom-up coding mode requires a lot of time cost, which in turn affects the coding speed of the data block.

[0087] It can be seen that the top-down encoding mode and the bottom-up encoding mode can usually more accurately determine the optimal division method for each data block in the frame image. The top-down encoding mode is more suitable for data blocks with lower scene complexity, and the bottom-up encoding mode is more suitable for data blocks with higher scene complexity.

[0088] On this basis, the embodiment of the present application provides a further video processing scheme, which divides the data block to be encoded in the target frame image of the video to be encoded into N sub-data blocks, and determines the encoding mode of the data block to be encoded according to the complexity parameters between the N sub-data blocks; it can be seen that the embodiment of the present application can determine a suitable encoding mode for the data block to be encoded according to the complexity parameters between the N sub-data blocks obtained by dividing the data block to be encoded, and the suitable encoding mode can greatly speed up the encoding speed of the data block to be encoded, and effectively improve the encoding efficiency of the video to be encoded. In addition, the complexity parameters between the N sub-data blocks are determined according to the complexity index information corresponding to each sub-data block in the N sub-data blocks, and the complexity index information of each sub-data block is obtained by performing scene complexity analysis on the sub-data block; the video processing scheme can formulate an encoding mode suitable for the data block to be encoded according to the scene complexity of the N sub-data blocks. The video processing scheme is more universal and applicable to any video encoding scene. It can determine the encoding mode for the data block with lower scene complexity and the encoding mode for the data block with higher scene complexity.

[0089] Based on the above description, an embodiment of the present application provides an encoder, which can be used to implement the above video processing solution. The video encoder can be an AV1 (Alliance for Open Media Video 1) standard encoder. Figure 2 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application, such as Figure 2 As shown, the encoder 20 includes a scene complexity analysis module 201, an adaptive partition decision module 202, an early partition termination module 203 and an encoding module 204. Among them:

[0090] (1) Scene complexity analysis module 201

[0091] The scene complexity analysis module 201 is used to divide the data block to be encoded in the target frame image of the video to be encoded into N sub-data blocks, and perform scene complexity analysis on each of the N sub-data blocks to obtain complexity index information of each sub-data block, where N is an integer greater than or equal to 2; in actual encoding scenarios, the value of N is generally 4, and the embodiments of the present application do not limit this; that is, dividing the data block to be encoded into N sub-data blocks may refer to dividing the data block to be encoded into 4 sub-data blocks of the same size.

[0092] The scene complexity analysis module 201 is further used to determine the complexity parameters between the N sub-data blocks according to the complexity index information corresponding to each sub-data block in the N sub-data blocks.

[0093] (2) Adaptive partition decision module 202

[0094] The adaptive partition decision module 202 can be used to determine the encoding mode of the data block to be encoded according to the complexity parameters between the N sub-data blocks. The encoding mode may include any one of the block encoding mode and the complete block encoding mode, that is, the adaptive partition decision module 202 can determine the encoding mode of the data block to be encoded as any one of the block encoding mode and the complete block encoding mode according to the complexity parameters between the N sub-data blocks. The block encoding mode refers to dividing the data block to be encoded into N sub-data blocks, and determining the target sub-data block in the N sub-data blocks as a new data block to be encoded, so as to further determine the encoding mode of the target sub-data block, and the target sub-data block is any one of the N sub-data blocks; the complete block encoding mode refers to directly using the encoding module 204 to perform image encoding processing on the data block to be encoded.

[0095] If the adaptive partitioning decision module 202 cannot determine the encoding mode of the data block to be encoded according to the complexity parameters between the N sub-data blocks, then the adaptive partitioning decision module 202 can also be used to obtain the associated block index parameters of the M associated data blocks associated with the data block to be encoded in the target frame image, and determine the pre-coding order of the data block to be encoded according to the associated block index parameters, where M is a positive integer. In actual encoding scenarios, the value of M is generally 3, which is not limited in the embodiments of the present application. Figure 3 is a schematic diagram of an associated data block provided in an embodiment of the present application, such as Figure 3 As shown, the M associated data blocks associated with the data block to be encoded in the target frame image may refer to the associated data block 302 located at the left position of the data block to be encoded 301 in the target frame image, the associated data block 303 located at the top position of the data block to be encoded 301, and the associated data block 304 located at the upper left position of the data block to be encoded 301. The precoding order may include any one of the first precoding order or the second precoding order, that is, the adaptive partition decision module 202 may determine the precoding order of the data block to be encoded as the first precoding order or the second precoding order according to the associated block index parameter. The first pre-coding order refers to a pre-coding order that preferentially attempts to divide the data block to be encoded into N sub-data blocks, and inputs the N sub-data blocks into the encoding module 204 for image encoding processing respectively, and then attempts to directly input the data block to be encoded into the encoding module 204 for image encoding processing; the second pre-coding order is a coding order that preferentially attempts to directly input the data block to be encoded into the encoding module 204 for image encoding processing, and then attempts to divide the data block to be encoded into N sub-data blocks, and inputs the N sub-data blocks into the encoding module 204 for image encoding processing respectively.

[0096] (3) Dividing the Early Termination Module 203

[0097] The early partition termination module 203 is used to set a partition termination condition when the adaptive partition decision module 202 cannot determine the coding mode for the data block to be coded, and determine the coding mode for the data block to be coded according to the set partition termination condition.

[0098] In the case where the precoding order of the data block to be encoded is the first precoding order, the early division termination module 203 obtains the sub-block index parameters of the N sub-data blocks, and the sub-block index parameters include the second number of sub-data blocks in the N sub-data blocks whose coding mode is the block coding mode; if the sub-block index parameter satisfies the first division termination condition (i.e., the second number is greater than or equal to the second threshold), the early division termination module 203 determines that the coding mode of the data block to be encoded is the block coding mode. If the sub-block index parameter does not satisfy the first division termination condition (i.e., the second number is less than the second threshold), the early division termination module 203 obtains the coding information of the data block to be encoded obtained by directly performing image coding processing on the data block to be encoded from the coding module 204, and obtains the coding information of the N sub-data blocks obtained by performing image coding processing on each sub-data block in the N sub-data blocks from the coding module 204, and determines the coding mode of the data block to be encoded according to the coding information of the data block to be encoded and the coding information of the N sub-data blocks.

[0099] In the case where the precoding order of the data block to be encoded is the second precoding order, the early division termination module 203 obtains the encoding parameters of the data block to be encoded obtained by directly performing image encoding processing on the data block to be encoded from the encoding module, and the encoding parameters are calculated according to the encoding information of the data block to be encoded; if the encoding parameters meet the second division termination condition (that is, the encoding parameters are less than the third threshold), the early division termination module 203 determines that the encoding mode of the data block to be encoded is the complete block encoding mode. If the sub-block index parameter does not meet the first division termination condition (that is, the second number is less than the second threshold), the early division termination module 203 obtains the encoding information of the data block to be encoded obtained by directly performing image encoding processing on the data block to be encoded from the encoding module 204, and obtains the encoding information of the N sub-data blocks obtained by performing image encoding processing on each of the N sub-data blocks from the encoding module 204, and determines the encoding mode of the data block to be encoded according to the encoding information of the data block to be encoded and the encoding information of the N sub-data blocks.

[0100] (4) Encoding module 204

[0101] The coding module 204 is used to perform image coding processing on the data block to be coded, and calculate the coding information of the data block to be coded obtained by performing image coding processing on the data block to be coded. The coding module 204 is also used to perform image coding processing on the N sub-data blocks respectively, and calculate the coding information of the N sub-data blocks obtained by performing image coding processing on the N sub-data blocks.

[0102] Among them, image coding processing refers to performing image coding processing on the data block or sub-data block to be encoded as an image; for example, the data block or sub-data block to be encoded may include multiple pixels, and image coding processing may be performed on the data block or sub-data block to be encoded according to the correlation between pixel information of each pixel; or, performing coding processing on the data block or sub-data block to be encoded may refer to performing intra-frame prediction coding processing on the data block or sub-data block to be encoded, or may refer to performing inter-frame prediction coding processing on the data block or sub-data block to be encoded.

[0103] It should be noted that if the encoder 20 determines through four modules (i.e., the scene complexity analysis module 201, the adaptive partition decision module 202, the early partition termination module 203, and the encoding module 204) that the encoding mode for the data block to be encoded is the block encoding mode, then the data block to be encoded is divided into N sub-data blocks according to the block encoding mode, and each of the N sub-data blocks is used as a new data block to be encoded, and is re-input into the encoder 20 for processing to determine the encoding mode of each sub-data block in the N sub-data blocks. If the encoder 20 determines through four modules that the encoding mode for the data block to be encoded is the complete block encoding mode, the encoder directly inputs the data block to be encoded into the encoding module 204 for image encoding processing.

[0104] Figure 2 The encoder 20 shown can determine the encoding mode suitable for the data block to be encoded according to the scene complexity of the N sub-data blocks obtained by dividing the data block to be encoded, which effectively improves the encoding speed of the data block to be encoded, thereby improving the video encoding speed. In addition, the early division termination module 203 in the encoder 20 can determine the encoding mode of the data block to be encoded when the division termination condition is met, and further improve the encoding speed of the data block to be encoded on the premise of achieving a balance between the encoding speed of the data block to be encoded and the encoding bit rate, thereby further improving the video encoding efficiency. Figure 4 The video processing system shown in the figure is a video processing solution provided by the embodiment of the present application and Figure 2 The specific application scenario of the encoder 20 provided in the illustrated embodiment is introduced.

[0105] Figure 4 is a schematic diagram of the architecture of a video processing system provided by an embodiment of the present application, such as Figure 4As shown, the video processing system includes P terminals (such as a first terminal 401, a second terminal 402, etc.) and a server 403, where P is an integer greater than 1. Among them, any one of the P terminals can be a device with a camera function, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted device, a smart speaker, a smart watch, a smart wearable device, etc., but is not limited to this. Any one of the P terminals can support the installation and operation of various applications, and the applications here can include but are not limited to social applications (such as instant messaging applications, audio conversation applications, video conversation applications, etc.), audio and video applications (such as audio and video on-demand applications, audio and video players, etc.), game applications, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services, which is not limited in this embodiment of the present application. The P terminals can be directly or indirectly connected to the server 403 via wired communication or wireless communication, which is not limited in this application. Taking the video conversation scenario as an example, the video processing solution provided in the embodiment of the present application is introduced below:

[0106] (1) The encoders 20 are deployed in P terminals respectively, and the P terminals perform video processing.

[0107] P users participate in a video conversation using video conversation applications running in P terminals respectively, for example, user 1 participates in a video conversation using a video conversation application running in a first terminal 401, user 2 participates in a video conversation using a video conversation application running in a second terminal 402, and so on; server 403 is used to transmit target videos generated by the P terminals during the video conversation. The target video may include conversation videos generated by the P terminals during the video conversation. The first terminal 401 is any one of the P terminals, and the processing of the target video by the first terminal 401 is used as an example for explanation. The processing process of the target video by other terminals among the P terminals except the first terminal 401 is the same as the processing process of the target video by the first terminal 401.

[0108] The target video is composed of a plurality of continuous frame images, and each frame image contained in the target video includes a plurality of data blocks to be encoded. The encoder 20 is deployed in the first terminal 401, and the first terminal 401 calls the encoder 20 to process each data block to be encoded in each frame image contained in the target video, determines the encoding mode for each data block to be encoded, and processes each data block according to the determined encoding mode, and finally obtains the encoded target video. The first terminal 401 can send the encoded target video to the server 403, and the server 403 transmits the encoded target video to other terminals participating in the video session; or, the first terminal 401 can also directly transmit the encoded target video to other terminals participating in the video session; other terminals receive the encoded target video, parse and play the target video, so as to realize the video session participated by P terminals.

[0109] (2) The encoder 20 is deployed in the server 403, and the server 403 performs video processing.

[0110] P users participate in a video session using video session applications running in P terminals respectively, for example, user 1 participates in a video session using a video session application running in a first terminal 401, user 2 participates in a video session using a video session application running in a second terminal 402, and so on; server 403 is used to process and transmit target videos generated by the P terminals during the video session. The target videos may include session videos generated by the P terminals during the video session.

[0111] The target video may be a session video generated by the first terminal 401 in the process of participating in a video session. The first terminal 401 transmits the target video to the server 403. The target video is composed of a plurality of continuous frame images, and each frame image contained in the target video includes a plurality of data blocks to be encoded. The encoder 20 is deployed in the server 403. The server 403 calls the encoder 20 to process each data block to be encoded in each frame image contained in the target video, determines the encoding mode for each data block to be encoded, and processes each data block according to the determined encoding mode, and finally obtains the encoded target video. The server 403 transmits the encoded target video to other terminals participating in the video session; other terminals receive the encoded target video, parse and play the target video, so as to realize a video session in which P terminals participate.

[0112] In an embodiment of the present application, each terminal or server participating in a video session can call an encoder to encode a target video generated during the video session, and the encoding mode of each terminal or server for each data block to be encoded contained in the target video is determined according to the scene complexity of the N sub-data blocks obtained by dividing each data block to be encoded, thereby effectively improving the encoding speed of the data block to be encoded, thereby improving the encoding speed of the target video, and greatly accelerating the encoding speed of the target video while ensuring the quality of the target video, thereby improving the fluency of the target video, improving the session quality of the video session to a certain extent, optimizing the video session effect, and improving the user experience.

[0113] It should be noted that the video processing system described in the embodiment of the present application is for the purpose of more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0114] See also Figure 5 , Figure 5 is a flow chart of a video processing method provided in an embodiment of the present application. The video processing method can be executed by an electronic device, which can be a smart terminal or a server. The smart terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a car device, a smart speaker, a smart watch, a smart wearable device, etc. For example, the electronic device can be Figure 4 Any terminal or server in the illustrated embodiment. The video processing method may include the following steps S501 to S505:

[0115] S501, determining a data block to be encoded in a target frame image of a video to be encoded.

[0116] The video to be encoded is composed of multiple continuous frame images. Determining the data block to be encoded in the target frame image of the video to be encoded may include: obtaining the target frame image from the video to be encoded, the target frame image is any frame image to be encoded in the video to be encoded; the target frame image contains multiple data blocks, the multiple data blocks may contain encoded data blocks and data blocks to be encoded, and determining any data block to be encoded from the target frame image.

[0117] S502: Divide the data block to be encoded into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block.

[0118] The data block to be encoded is divided into N sub-data blocks, where N is an integer greater than or equal to 2; complexity index information corresponding to each sub-data block in the N sub-data blocks is obtained, and the complexity index information corresponding to each sub-data block is obtained by performing scene complexity analysis on the sub-data block.

[0119] The complexity index information of the ith sub-data block among the N sub-data blocks may include, but is not limited to: any one or more of the distortion estimation parameters of the ith sub-data block, the spatial information parameters of the ith sub-data block, and the temporal information parameters of the ith sub-data block, where i is a positive integer less than or equal to N. The distortion estimation parameter of the ith sub-data block is obtained by estimating the degree of distortion during the intra-frame prediction coding process or the inter-frame prediction coding process of the ith sub-data block; the distortion estimation parameter of the ith sub-data block can be used to measure the degree of distortion of the reconstructed ith sub-data block compared with the original ith sub-data block; the original ith sub-data block refers to the ith sub-data block that has not been input into the coding module 204 for image coding, and the reconstructed ith sub-data block refers to the coded ith sub-data block obtained after the ith sub-data block is input into the coding module 204 for intra-frame prediction coding process or inter-frame prediction coding process. The spatial information parameter of the ith sub-data block may refer to the value of the spatial information calculated for the ith sub-data block; the temporal information parameter of the ith sub-data block may refer to the value of the temporal information calculated for the ith sub-data block.

[0120] Before step S502, that is, before dividing the data block to be encoded into N sub-data blocks, the complexity parameters of the data block to be encoded can be obtained, and the complexity parameters of the data block to be encoded are obtained by performing scene complexity analysis on the data block to be encoded; the complexity parameters of the data block to be encoded may include but are not limited to: any one or more of the distortion estimation parameters of the data block to be encoded, the spatial information parameters of the data block to be encoded, and the time information parameters of the data block to be encoded; the distortion estimation parameters of the data block to be encoded are obtained by estimating the degree of distortion during the intra-frame prediction coding process or the inter-frame prediction coding process of the data block to be encoded; the distortion estimation parameters of the data block to be encoded can be used to measure the degree of distortion of the reconstructed data block to be encoded compared with the original data block to be encoded; the original data block to be encoded refers to the data block to be encoded that has not been input into the encoding module 204 for image coding processing, and the reconstructed data block to be encoded refers to the encoded data block to be encoded obtained after the data block to be encoded is input into the encoding module 204 and subjected to intra-frame prediction coding or inter-frame prediction coding. The spatial information parameter of the data block to be encoded may refer to the value of the spatial information obtained by calculating the data block to be encoded; the temporal information parameter of the data block to be encoded may refer to the value of the temporal information obtained by calculating the data block to be encoded. If the complexity parameter of the data block to be encoded meets the overall coding condition, the complete block coding mode is used as the coding mode of the data block to be encoded, and the data block to be encoded is subjected to image coding processing according to the complete block coding mode; the complexity parameter of the data block to be encoded meeting the overall coding condition may include any one or more of the following four situations: ① the distortion estimation parameter of the data block to be encoded is less than the distortion estimation threshold; ② the spatial information parameter of the data block to be encoded is less than the spatial information threshold; ③ the time information parameter of the data block to be encoded is less than the time information threshold; ④ the complexity index parameter of the data block to be encoded is less than the complexity index threshold; the complexity index parameter of the data block to be encoded is calculated based on any one or more of the distortion estimation parameter of the data block to be encoded, the spatial information parameter of the data block to be encoded, and the time information parameter of the data block to be encoded; for example, the complexity index parameter of the data block to be encoded may be the sum of the distortion estimation parameter of the data block to be encoded, the spatial information parameter of the data block to be encoded, and the time information parameter of the data block to be encoded. If the complexity parameters of the data block to be encoded do not meet the overall encoding conditions, step S502 is triggered to be executed, that is, the step of dividing the data block to be encoded into N sub-data blocks is triggered; the complexity parameters of the data block to be encoded do not meet the overall encoding conditions and may include any one or more of the following four situations: ① The distortion estimation parameter of the data block to be encoded is greater than or equal to the distortion estimation threshold; ② The spatial information parameter of the data block to be encoded is greater than or equal to the spatial information threshold; ③ The time information parameter of the data block to be encoded is greater than or equal to the time information threshold; ④ The complexity index parameter of the data block to be encoded is greater than or equal to the complexity index threshold.It should be noted that the distortion estimation threshold, the spatial information threshold, the temporal information threshold, and the complexity index threshold can be set according to empirical values. In this way, when the complexity parameter of the data block to be encoded meets the overall encoding condition, the complete block encoding mode can be used as the encoding mode of the data block to be encoded, and the image encoding processing is performed on the data block to be encoded according to the complete block encoding mode, without dividing the data block to be encoded into N sub-data blocks for processing, which can effectively improve the encoding speed of the data block to be encoded, and thus improve the encoding efficiency of the video to be encoded.

[0121] S503: Determine the complexity parameters between the N sub-data blocks according to each complexity indicator information.

[0122] Determining the complexity parameters between the N sub-data blocks according to the complexity index information may include: determining the complexity parameters between the N sub-data blocks according to the difference between the complexity index information corresponding to the sub-data blocks. Specifically, the difference between the complexity index information corresponding to any two sub-data blocks in each sub-data block may be calculated, and then the complexity parameter may be determined according to the sum of the calculated differences; for example, the complexity parameter is the sum of the differences, or the complexity parameter is the weighted sum of the differences, etc.

[0123] Since the complexity indicator information corresponding to the sub-data block may include one or more parameters (i.e., the distortion estimation parameters of the sub-data block, the time information parameters of the sub-data block, the spatial information parameters of the sub-data block, etc.), the difference between the complexity indicator information corresponding to any two sub-data blocks may be the difference between a single parameter; for example, the difference between the complexity indicator information corresponding to the first sub-data block and the complexity indicator information corresponding to the second sub-data block is the difference between the distortion estimation parameters of the first sub-data block and the distortion estimation parameters of the second sub-data block. The difference between the complexity indicator information corresponding to any two sub-data blocks may also be the sum of the differences between multiple parameters; for example, the difference between the complexity indicator information corresponding to the first sub-data block and the complexity indicator information corresponding to the second sub-data block is the sum of the first difference, the second difference, and the third difference, the first difference being the difference between the distortion estimation parameters of the first sub-data block and the distortion estimation parameters of the second sub-data block, the second difference being the difference between the spatial information parameters of the first sub-data block and the spatial information parameters of the second sub-data block, and the third difference being the difference between the time information parameters of the first sub-data block and the time information parameters of the second sub-data block. In this way, there are multiple combinations of complexity parameters between the N sub-data blocks, and a suitable combination can be selected as the complexity parameter between the N sub-data blocks as needed, so that the encoding mode of the data block to be encoded can be accurately determined.

[0124] It should be noted that the complexity parameter between the N sub-data blocks can be used to indicate the complexity correlation between the N sub-data blocks. The so-called complexity correlation between the N sub-data blocks refers to the degree of association between the N sub-data blocks; the larger the complexity parameter between the N sub-data blocks, the lower the complexity correlation between the N sub-data blocks, the weaker the degree of association between the N sub-data blocks, and the more inclined to divide the data block to be encoded into N sub-data blocks for processing; the smaller the complexity parameter between the N sub-data blocks, the higher the complexity correlation between the N sub-data blocks, the stronger the degree of association between the N sub-data blocks, and the more inclined to perform image encoding processing on the data block to be encoded.

[0125] S504: If the complexity parameter meets the block coding condition, the block coding mode is used as the coding mode of the data block to be coded, and the target sub-data block in the N sub-data blocks is determined as a new data block to be coded according to the block coding mode.

[0126] S505: If the complexity parameter satisfies the complete block coding condition, the complete block coding mode is used as the coding mode of the data block to be coded, and image coding processing is performed on the data block to be coded according to the complete block coding mode.

[0127] In step S504 to step S505, the execution order of step S504 and step S505 is in no particular order. If the complexity parameter meets the block coding condition, the block coding mode is used as the coding mode of the data block to be encoded, and the target sub-data block in the N sub-data blocks is determined as a new data block to be encoded according to the block coding mode, so as to further determine the coding mode of the target sub-data block. Determining the coding mode of the target sub-data block may include: further dividing the target sub-data block into N sub-data blocks; determining the coding mode of the target sub-data block according to the complexity parameters between the N sub-data blocks obtained by dividing the target sub-data block; wherein the determined coding mode of the target sub-data block includes: a block coding mode for dividing the N sub-data blocks obtained by further dividing the target sub-data block, or a complete block coding mode for directly performing image coding processing on the target sub-data block. The complexity parameter between N sub-data blocks meets the block coding condition, which means that the complexity parameter between N sub-data blocks is greater than the block coding threshold. The complexity parameter between N sub-data blocks meets the block coding condition, indicating that the complexity correlation between N sub-data blocks is low, the degree of association between N sub-data blocks is weak, and the coding mode that is more suitable for the coded data block is the block coding mode. If the complexity parameter meets the complete block coding condition, the complete block coding mode is used as the coding mode of the data block to be coded, and the image coding processing is performed on the coded data block according to the complete block coding mode; the complexity parameter between N sub-data blocks meets the complete block coding condition, which means that the complexity parameter between N sub-data blocks is less than the complete block coding threshold. The complexity parameter between N sub-data blocks meets the complete block coding condition, indicating that the complexity correlation between N sub-data blocks is high, the degree of association between N sub-data blocks is strong, and the coding mode that is more suitable for the coded data block is the complete block coding mode. It should be noted that the block coding threshold and the complete block coding threshold here can be set according to empirical values.

[0128] In one embodiment, the complexity parameters of the data block to be encoded and the complexity index information corresponding to the N sub-data blocks are also input into the joint statistical model, and the complexity parameters of the data block to be encoded and the complexity index information corresponding to the N sub-data blocks are calculated using the joint statistical model; the output value obtained after the complexity parameters of the data block to be encoded and the complexity index information corresponding to the N sub-data blocks are calculated by the joint statistical model is obtained, and the encoding mode for the data block to be encoded is determined according to the output value of the joint statistical model. The output value of the joint statistical model can be used to indicate the correlation between the N sub-data blocks; the larger the output value, the weaker the correlation between the N sub-data blocks, and the more inclined to divide the data block to be encoded into N sub-data blocks for processing; the smaller the output value, the stronger the correlation between the N sub-data blocks, and the more inclined to perform image encoding processing on the data block to be encoded.

[0129] Determining the coding mode for the data block to be coded according to the output value of the joint statistical model may include: if the output value is greater than the first partition threshold, the block coding mode is used as the coding mode of the data block to be coded, and the target sub-data block in the N sub-data blocks is determined as the new data block to be coded according to the block coding mode, so as to further determine the coding mode of the target sub-data block; the output value is greater than the first partition threshold, indicating that the correlation between the N sub-data blocks is weak, and the coding mode that is more suitable for the coded data block is the block coding mode. If the output value is less than the second partition threshold, the complete block coding mode is used as the coding mode of the data block to be coded, and the image coding processing is performed on the coded data block according to the complete block coding mode; the output value is less than the second partition threshold, indicating that the correlation between the N sub-data blocks is strong, and the coding mode that is more suitable for the coded data block is the complete block coding mode. It should be noted that the joint statistical model can be obtained by training using the complexity parameters of the data block with the determined coding mode, and the complexity index information corresponding to the N sub-data blocks obtained by dividing the data block with the determined coding mode; the first partition threshold and the second partition threshold here can be obtained by training during the training process of the joint statistical model.

[0130] In an embodiment of the present application, the data block to be encoded is divided into N sub-data blocks, and the encoding mode of the data block to be encoded is determined according to the complexity parameters between the N sub-data blocks; the complexity parameters between the N sub-data blocks are obtained by performing scene complexity analysis on the N sub-data blocks, and the complexity parameters between the N sub-data blocks can be used to indicate the complexity correlation between the N sub-data blocks; by performing scene complexity analysis on the N sub-data blocks, the video processing solution provided in the embodiment of the present application is applicable to any video encoding scenario, and can adaptively adjust the encoding mode of the data block to be encoded according to the scene complexity of the N sub-data blocks obtained by dividing the data block to be encoded in any video, thereby determining the encoding mode suitable for the data block to be encoded, effectively improving the encoding speed of the data block to be encoded, thereby improving the video encoding speed, and can obtain the best balance between the video encoding speed and the video encoding bit rate.

[0131] See also Figure 6 , Figure 6 is a flow chart of another video processing method provided in an embodiment of the present application. The video processing method can be executed by an electronic device, which can be a smart terminal or a server. The smart terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a car device, a smart speaker, a smart watch, a smart wearable device, etc. For example, the electronic device can be Figure 4 Any terminal or server in the illustrated embodiment. The video processing method may include the following steps S601 to S608:

[0132] S601, determining a data block to be encoded in a target frame image of a video to be encoded.

[0133] S602: Divide the data block to be encoded into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block.

[0134] S603: Determine the complexity parameters between the N sub-data blocks according to each complexity indicator information.

[0135] S604, if the complexity parameter meets the block coding condition, the block coding mode is used as the coding mode of the data block to be encoded, and the target sub-data block among the N sub-data blocks is determined as the new data block to be encoded according to the block coding mode, so as to further determine the coding mode of the target sub-data block.

[0136] S605: If the complexity parameter satisfies the complete block coding condition, the complete block coding mode is used as the coding mode of the data block to be coded, and image coding processing is performed on the data block to be coded according to the complete block coding mode.

[0137] In the embodiment of the present application, the specific execution process of step S601 to step S605 can be found in Figure 5 The description of the illustrated embodiment will not be repeated here.

[0138] S606: If the complexity parameter satisfies the coding decision condition, M associated data blocks associated with the data block to be coded are determined in the target frame image.

[0139] In one embodiment, the complexity parameter between the N sub-data blocks satisfies the coding decision condition, which means that the complexity parameter between the N sub-data blocks is less than or equal to the block coding threshold, and the complexity parameter between the N sub-data blocks is greater than or equal to the complete block coding threshold; the complexity parameter between the N sub-data blocks satisfies the coding decision condition, which indicates that the complexity correlation between the N sub-data blocks is between low and high, the degree of association between the N sub-data blocks is between weak and strong, and there is no obvious tendency to divide the data block to be encoded into N sub-data blocks for processing, nor is there an obvious tendency to perform image coding processing on the data block to be encoded. In this case, it is necessary to determine M associated data blocks associated with the data block to be encoded in the target frame image, where M is a positive integer.

[0140] In another embodiment, if the output value of the joint statistical model is less than or equal to the first partition threshold, and the output value of the joint statistical model is greater than or equal to the second partition threshold, it indicates that the correlation between the N sub-data blocks is between strong and weak, and there is no obvious tendency to divide the data block to be encoded into N sub-data blocks for processing, and there is no obvious tendency to perform image encoding processing on the data block to be encoded. In this case, it is necessary to determine M associated data blocks associated with the data block to be encoded in the target frame image, where M is a positive integer.

[0141] In actual coding scenarios, the value of M is generally 3, which is not limited in the embodiments of the present application; the M associated data blocks associated with the data block to be encoded in the target frame image may refer to the associated data block located at the left position of the data block to be encoded, the associated data block located at the top position of the data block to be encoded, and the associated data block located at the upper left position of the data block to be encoded in the target frame image, which can be seen from the above Figure 3 Description of the illustrated embodiment. It should be noted that the execution order of step S604, step S605 and step S606 is not particular.

[0142] S607, obtaining associated block index parameters of the M associated data blocks, and determining the precoding order of the to-be-coded data blocks according to the associated block index parameters.

[0143] S608: Perform precoding processing on the data block to be coded according to the determined precoding order, and determine a coding mode for the data block to be coded.

[0144] In step S607 to step S608, the associated block index parameters of the M associated data blocks are obtained. The associated block index parameters are obtained by performing scene complexity analysis on the M associated data blocks. The associated block index parameters include the first number of associated data blocks whose coding mode is the block coding mode in the M associated data blocks, that is, the associated block index parameters include the first number of associated data blocks in the M associated data blocks, in which the associated data blocks are divided into N sub-data blocks for further processing. The associated block index parameters can be used to indicate the scene complexity of the M associated data blocks; the larger the associated block index parameters, the higher the scene complexity of the M associated data blocks; the smaller the associated block index parameters, the smaller the scene complexity of the M associated data blocks. Thus, the precoding order of the data blocks to be coded can be determined according to the associated block index parameters, and the data blocks to be coded are precoded according to the determined precoding order, and the coding mode of the data blocks to be coded is determined. The precoding order can include any one of the first precoding order or the second precoding order, that is, the precoding order of the data blocks to be coded can be determined as the first precoding order or the second precoding order according to the associated block index parameters. The first pre-coding order refers to a pre-coding order that preferentially attempts to divide the data block to be encoded into N sub-data blocks, and inputs the N sub-data blocks into the encoding module 204 for image encoding processing respectively, and then attempts to directly input the data block to be encoded into the encoding module 204 for image encoding processing; the second pre-coding order is a coding order that preferentially attempts to directly input the data block to be encoded into the encoding module 204 for image encoding processing, and then attempts to divide the data block to be encoded into N sub-data blocks, and inputs the N sub-data blocks into the encoding module 204 for image encoding processing respectively.

[0145] In one embodiment, if the first number is greater than or equal to the first threshold, it indicates that the scene complexity of the M associated data blocks is relatively high, and it is more inclined to preferentially try to divide the data block to be encoded into N sub-data blocks, respectively input the N sub-data blocks into the encoding module 204 for image encoding processing, and then try to directly input the data block to be encoded into the encoding module 204 for image encoding processing. The pre-coding sequence, that is, the pre-coding sequence of the data block to be encoded is determined as the first pre-coding sequence. It should be noted that the first threshold can be set according to an empirical value. For example, if the number of associated data blocks related to the data block to be encoded is 3, the first threshold can be set to 2; Figure 3For example, the three associated data blocks related to the data block 301 to be encoded are associated data block 302, associated data block 303 and associated data block 304, the associated data block 302 and the associated data block 303 are divided into multiple sub-data blocks for further processing, and the associated data blocks 304304 are directly processed for image encoding, that is, the first number of associated data blocks whose encoding mode is the block encoding mode among the three associated data blocks is 2, and the first number is equal to the second threshold value. Then, it can be determined that the scene complexity of the three associated data blocks related to the associated data block 301 is relatively high, and the precoding order of the data blocks to be encoded can be determined as the first precoding order.

[0146] Precoding the data block to be coded according to the first precoding order, and determining the coding mode of the precoded data block may include: obtaining subblock index parameters of N sub-data blocks according to the first precoding order; and determining the coding mode of the data block to be coded according to the subblock index parameters. The subblock index parameters include the second number of sub-data blocks in the N sub-data blocks whose coding mode is the block coding mode, that is, the sub-block index parameters include the second number of sub-data blocks in the N sub-data blocks, which are used as new data blocks to be coded and the new data blocks to be coded are divided into N sub-data blocks for further processing.

[0147] If the second number is greater than or equal to the second threshold, the block coding mode is used as the coding mode of the data block to be coded, and the target sub-data block in the N sub-data blocks is determined as a new data block to be coded according to the block coding mode, so as to further determine the coding mode of the target sub-data block. If the second number is less than the second threshold, the data block to be coded is subjected to image coding processing to obtain coding information of the data block to be coded; the coding information of the N sub-data blocks is obtained, and the coding information of the N sub-data blocks is obtained by performing image coding processing on each sub-data block in the N sub-data blocks; and the coding mode of the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks. It should be noted that the second threshold can also be set based on an empirical value; for example, if the data block to be encoded is divided into 4 sub-data blocks, the second threshold can be set to 3. If all of the 4 sub-data blocks are used as new data blocks to be encoded, and the new data block to be encoded is divided into N sub-data blocks for further processing, that is, the second number is 4, and the second number is greater than or equal to the second threshold, it can be determined that the encoding mode for the data block to be encoded is the block encoding mode.

[0148] In another embodiment, if the first number is less than the first threshold, it indicates that the scene complexity of the M associated data blocks is relatively low, and it is more inclined to give priority to trying to directly input the data block to be encoded into the encoding module 204 for image encoding processing in a pre-coding order, and then try to divide the data block to be encoded into N sub-data blocks, and respectively input the N sub-data blocks into the encoding module 204 for image encoding processing in a pre-coding order, that is, the pre-coding order of the data block to be encoded is determined as the second pre-coding order. Performing pre-coding processing on the data block to be encoded according to the second pre-coding order, and determining the encoding mode of the pre-coded data block may include: performing image coding processing on the data block to be encoded according to the second pre-coding order, and determining the encoding mode of the data block to be encoded according to the encoding information of the data block to be encoded obtained by encoding.

[0149] The coding information of the data block to be coded may include a coding distortion parameter of the data block to be coded; it should be noted that the distortion estimation parameter of the data block to be coded may refer to an estimated value of the degree of distortion during the coding process of the data block to be coded, and the coding distortion parameter of the data block to be coded may be used to measure the actual degree of distortion of the coded data block to be coded (i.e., the reconstructed data block to be coded) compared to the coded data block to be coded before coding (i.e., the original data block to be coded). Performing image coding processing on the data block to be coded, and determining the coding mode of the data block to be coded according to the coding information of the data block to be coded obtained by coding may include: calculating a coding parameter (Quatization Parameter, QP) according to the coding distortion parameter of the data block to be coded and a quantization parameter for quantizing the data block to be coded, and the calculation process of the coding parameter may refer to the following formula 1:

[0150] Code=Dist / QP 2 Formula 1

[0151] As shown in the above formula 1, Code represents a coding parameter, Dist represents a coding distortion parameter of a data block to be coded, and QP represents a quantization parameter.

[0152] If the coding parameter is less than the third threshold, the complete block coding mode is used as the coding mode of the data block to be coded, and the data block to be coded is subjected to image coding processing according to the complete block coding mode. If the coding parameter is greater than or equal to the third threshold, image coding processing is performed on each of the N sub-data blocks to obtain coding information of the N sub-data blocks; and the coding mode of the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks. It should be noted that the third threshold can be set according to an empirical value, or it can be obtained through training during the training process of the joint statistical model.

[0153] In the above two embodiments, the encoding information of the data block to be encoded may also include precoding evaluation parameters of the data block to be encoded; the encoding information of the N sub-data blocks may include precoding evaluation parameters of the N sub-data blocks, and the precoding evaluation parameters of the N sub-data blocks are calculated based on the precoding evaluation parameters of each sub-data block in the N sub-data blocks. For example, the precoding evaluation parameters of the N sub-data blocks may be the sum of the precoding evaluation parameters of each sub-data block in the N sub-data blocks. Among them, the pre-coding evaluation parameter may include a coding rate distortion loss parameter. The coding rate distortion loss parameter of the data block to be coded is calculated according to the coding rate and the coding distortion parameter according to directly performing image coding processing on the data block to be coded. For example, the coding rate distortion loss parameter of the data block to be coded may be the ratio of the coding rate to the coding distortion parameter. The coding rate distortion loss parameter of the data block to be coded may be used to measure the coding effect of directly performing image coding processing on the data block to be coded. The smaller the coding rate distortion loss parameter of the data block to be coded is, the better the effect of image coding processing on the data block to be coded is. The good coding effect of the data block to be coded may mean that the coding rate of the data block to be coded is lower while ensuring that the distortion degree of the data block to be coded after coding is lower than that of the data block to be coded before the encoder. Similarly, the coding rate distortion loss parameters of N sub-data blocks may be used to measure the coding effect of dividing the data block to be coded into N sub-data blocks and performing image coding processing on each sub-data block respectively. The smaller the coding rate distortion loss parameters of the N sub-data blocks are, the better the effect of encoding the data block to be coded is. If the coding rate distortion loss parameter of the data block to be encoded is greater than or equal to the coding rate distortion loss parameter of the N sub-data blocks, the block coding mode is used as the coding mode of the data block to be encoded, and the target sub-data block among the N sub-data blocks is used as the new data block to be encoded according to the block coding mode, so as to further determine the coding mode of the target sub-data block; if the coding rate distortion loss parameter of the data block to be encoded is less than the coding rate distortion loss parameter of the N sub-data blocks, the complete block coding mode is used as the coding mode of the data block to be encoded, and the image coding processing is performed on the data block to be encoded according to the complete block coding mode.

[0154] In the embodiment of the present application, the scene complexity of the associated data block associated with the data block to be encoded can also reflect the scene complexity of the data block to be encoded to a certain extent. Therefore, the embodiment of the present application jointly determines the encoding mode of the data block to be encoded according to the scene complexity of the data block to be encoded (i.e., the output value of the joint statistical model or the complexity parameter between N sub-data blocks) and the scene complexity of the associated data block associated with the data block to be encoded (i.e., the associated block indication parameter). The video processing scheme provided by the embodiment of the present application is universal, and can adaptively adjust the encoding mode of the data block to be encoded according to the scene complexity of the data block to be encoded in any video, and according to the scene complexity of the surrounding blocks of the data block to be encoded (i.e., M associated data blocks), so as to determine the encoding mode suitable for the scene complexity of the data block to be encoded, effectively improve the encoding speed of the data block to be encoded, and then improve the video encoding speed, and can effectively improve the video encoding speed without losing the video encoding efficiency, and obtain the best balance between the video encoding speed and the video encoding bit rate.

[0155] Figure 5 and Figure 6 The video processing method described in the illustrated embodiment can be summarized as follows: Figure 7 , Figure 8a and Figure 8b The flowchart shown, Figure 7 , Figure 8a and Figure 8b The process shown can be executed by an electronic device, which can be a smart terminal or a server. The smart terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a car device, a smart speaker, a smart watch, a smart wearable device, etc. For example, the electronic device can be Figure 4 Any terminal or server in the illustrated embodiments.

[0156] See also Figure 7 , Figure 7 is a flow chart of another video processing method provided by an embodiment of the present application, such as Figure 7 As shown, first determine the encoding mode of the data block to be encoded; then determine whether the encoding mode of the data block to be encoded is a block encoding mode; if so, take the target sub-data block among the N sub-data blocks (that is, any sub-data block among the N sub-data blocks) as a new data block to be encoded, and further determine the encoding mode of the target sub-data block; if not, perform image encoding processing on the data block to be encoded, that is, directly input the data block to be encoded into the encoding module 204 for image encoding processing.

[0157] See also Figure 8a , Figure 8ais a flow chart of another video processing method provided in an embodiment of the present application. The video processing method may include steps S801 to S809:

[0158] S801, determining a data block to be encoded in a target frame image of a video to be encoded.

[0159] S802, dividing the data block to be encoded into N sub-data blocks, and obtaining complexity index information corresponding to each sub-data block, where N is an integer greater than or equal to 2.

[0160] S803: Determine the complexity parameters between the N sub-data blocks according to each complexity indicator information.

[0161] S804, determining whether the complexity parameter satisfies the block coding condition.

[0162] If the complexity parameter meets the block coding condition, step S805 is executed. If the complexity parameter does not meet the block coding condition, step S806 is executed.

[0163] S805, taking the block coding mode as the coding mode of the data block to be coded. According to the block coding mode, the target sub-data block in the N sub-data blocks is taken as a new data block to be coded, so as to further determine the coding mode of the target sub-data block. That is, in step S805, the target sub-data block in the N sub-data blocks is taken as a new data block to be coded, and step S802 is continued.

[0164] S806, determining whether the complexity parameter satisfies the complete block coding condition.

[0165] If the complexity parameter satisfies the complete block coding condition, step S807 is executed. If the complexity parameter does not satisfy the complete block coding condition, step S808 is executed.

[0166] S807: Use the complete block coding mode as the coding mode of the data block to be coded, and perform image coding processing on the data block to be coded according to the complete block coding mode.

[0167] S808: Determine whether the complexity parameter meets the coding decision condition.

[0168] S809: If the complexity parameter satisfies the coding decision condition, then M associated data blocks associated with the data block to be coded are determined in the target frame image, associated block index parameters of the M associated data blocks are obtained, and a coding mode for the data block to be coded is determined according to the associated block index parameters. The associated block index parameters include: a first number of associated data blocks whose coding mode is a block coding mode among the M associated data blocks.

[0169] The specific execution process of step S809 can be found in Figure 8b , Figure 8b is a flow chart of another video processing method provided in an embodiment of the present application. The video processing method may include steps S810 to S820.

[0170] S810: Obtain a first number of associated data blocks whose coding mode is the block coding mode among M associated data blocks.

[0171] S811, determine whether the first number is greater than or equal to a first threshold.

[0172] If the first number is greater than or equal to the first threshold, step S812 is executed. If the first number is less than the first threshold, step S819 is executed.

[0173] S812: Determine that the precoding order of the data block to be encoded is the first precoding order.

[0174] S813: According to the first pre-coding order, preferentially try to perform image coding processing on each of the N sub-data blocks respectively to obtain coding information of the N sub-data blocks.

[0175] S814: Obtain a second number of sub-data blocks whose coding mode is the block coding mode among the N sub-data blocks.

[0176] S815: Determine whether the second number is greater than or equal to a second threshold.

[0177] S816: If the second number is greater than or equal to the second threshold, the block coding mode is used as the coding mode of the data block to be encoded.

[0178] S817: Determine a target sub-data block among the N sub-data blocks as a new data block to be encoded according to the block encoding mode, so as to further determine the encoding mode of the target sub-data block.

[0179] S818, if the second number is less than the second threshold, try to perform image coding on the data block to be coded to obtain coding information of the data block to be coded, and determine the coding mode of the data block to be coded according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

[0180] S819: Determine that the precoding order of the data block to be coded is the second precoding order.

[0181] S820: According to the second pre-coding order, preferentially try to perform image coding processing on the data block to be coded to obtain coding information of the data block to be coded.

[0182] S821, determining a coding parameter of the data block to be coded according to coding information of the data block to be coded.

[0183] S822, determine whether the encoding parameter is less than a third threshold.

[0184] S823: If the encoding parameter is less than a third threshold, the complete block encoding mode is used as the encoding mode of the data block to be encoded.

[0185] S824, performing image coding processing on the data block to be coded according to the complete block coding mode.

[0186] S825, if the coding parameter is greater than or equal to the third threshold value, then try to perform image coding processing on each of the N sub-data blocks respectively to obtain coding information of the N sub-data blocks, and determine the coding mode of the data block to be encoded based on the coding information of the data block to be encoded and the coding information of the N sub-data blocks.

[0187] Figures 7 to 8b In the illustrated embodiment, the complexity correlation between the N sub-data blocks obtained by dividing the data block to be encoded can reflect the scene complexity of the data block to be encoded to a certain extent, and the scene complexity of the associated data block associated with the data block to be encoded can also reflect the scene complexity of the data block to be encoded to a certain extent. Therefore, the embodiment of the present application jointly determines the encoding mode of the data block to be encoded according to the scene complexity of the data block to be encoded (i.e., the output value of the joint statistical model or the complexity parameter between the N sub-data blocks) and the scene complexity of the associated data block associated with the data block to be encoded (i.e., the associated block indication parameter). The scene complexity of the data block to be encoded is comprehensively analyzed from multiple angles, so that the encoding mode of the data block to be encoded that is finally determined has a high degree of adaptability to the scene complexity of the data block to be encoded, effectively improving the encoding speed of the data block to be encoded, and then improving the video encoding speed, and can effectively improve the video encoding speed without losing the video encoding quality and video encoding efficiency while ensuring the clarity and smoothness of the encoded video.

[0188] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0189] See also Fig. 9 , Fig. 9 It is a structural diagram of a video processing device provided in an embodiment of the present application. The video processing device 90 can be set in an intelligent terminal. The intelligent terminal may include a smart phone, a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted device, a smart speaker, a smart watch or a smart wearable device and other devices with display function.

[0190] In one embodiment, the video processing device 90 can be used to perform Figure 5 , Figure 6 , Figure 7 , Figure 8a or Figure 8b The corresponding steps in the video processing method shown in the figure, the video processing device 90 may include:

[0191] A determination unit 901 is used to determine a data block to be encoded in a target frame image of a video to be encoded;

[0192] The processing unit 902 is used to divide the to-be-encoded data block into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block, where N is an integer greater than or equal to 2;

[0193] The processing unit 902 is further configured to determine the complexity parameter between the N sub-data blocks according to each complexity indicator information;

[0194] The processing unit 902 is further configured to use the block coding mode as the coding mode of the data block to be coded if the complexity parameter satisfies the block coding condition, and determine the target sub-data block in the N sub-data blocks as a new data block to be coded according to the block coding mode;

[0195] The processing unit 902 is further configured to, if the complexity parameter satisfies a complete block coding condition, use the complete block coding mode as the coding mode of the data block to be coded, and perform image coding processing on the data block to be coded according to the complete block coding mode.

[0196] In one embodiment, the processing unit 902 is further configured to:

[0197] If the complexity parameter satisfies the coding decision condition, then M associated data blocks associated with the data block to be coded are determined in the target frame image, where M is a positive integer;

[0198] Obtaining associated block index parameters of the M associated data blocks, and determining a precoding order of the to-be-coded data blocks according to the associated block index parameters;

[0199] The data block to be encoded is pre-encoded according to the determined pre-encoding order, and the encoding mode for the data block to be encoded is determined.

[0200] In one embodiment, the associated block indicator parameter includes: a first number of associated data blocks whose coding mode is a block coding mode among the M associated data blocks;

[0201] The processing unit 902 is used to determine the precoding order of the data blocks to be coded according to the associated block indicator parameter, specifically to:

[0202] If the first number is greater than or equal to the first threshold, determining that the precoding order of the data block to be encoded is the first precoding order;

[0203] The processing unit 902 is configured to perform precoding processing on the data block to be coded according to the determined precoding order, and when determining the coding mode for the data block to be coded, specifically to:

[0204] Obtaining sub-block index parameters of N sub-data blocks according to a first precoding order;

[0205] The encoding mode for the data block to be encoded is determined according to the sub-block index parameter.

[0206] In one embodiment, the sub-block indicator parameter includes a second number of sub-data blocks whose coding mode is the block coding mode in the N sub-data blocks, and the processing unit 902 is used to determine the coding mode for the data block to be coded according to the sub-block indicator parameter, specifically to:

[0207] If the second number is greater than or equal to the second threshold, the block coding mode is used as the coding mode of the data block to be encoded.

[0208] In one embodiment, the processing unit 902, when used to determine the encoding mode for the data block to be encoded according to the sub-block indicator parameter, is further used to:

[0209] If the second number is less than the second threshold, image coding is performed on the data block to be coded to obtain coding information of the data block to be coded;

[0210] Obtaining coding information of N sub-data blocks, where the coding information of the N sub-data blocks is obtained by performing image coding processing on each of the N sub-data blocks;

[0211] A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

[0212] In one embodiment, the processing unit 902 is used to determine the precoding order of the data blocks to be encoded according to the associated block indicator parameter, specifically to: if the first number is less than the first threshold, determine the precoding order of the data blocks to be encoded as the second precoding order;

[0213] The processing unit 902 is configured to perform precoding processing on the data block to be coded according to the determined precoding order, and when determining the coding mode for the data block to be coded, specifically to:

[0214] According to the second pre-coding sequence, image coding is performed on the data block to be coded, and a coding mode for the data block to be coded is determined according to coding information of the data block to be coded obtained by coding.

[0215] In one embodiment, the coding information of the data block to be coded includes a coding distortion parameter of the data block to be coded; the processing unit 902 is used to determine the coding mode of the data block to be coded according to the coding information of the data block to be coded obtained by coding, specifically to:

[0216] The coding parameter is calculated according to the coding distortion parameter of the data block to be coded and the quantization parameter for quantizing the data block to be coded;

[0217] If the encoding parameter is less than the third threshold, the complete block encoding mode is used as the encoding mode of the data block to be encoded;

[0218] If the encoding parameter is greater than or equal to the third threshold, performing image encoding processing on each of the N sub-data blocks respectively to obtain encoding information of the N sub-data blocks;

[0219] A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

[0220] In one embodiment, the coding information of the data block to be coded includes the precoding evaluation parameters of the data block to be coded; the coding information of the N sub-data blocks includes the precoding evaluation parameters of the N sub-data blocks, and the precoding evaluation parameters of the N sub-data blocks are calculated according to the precoding evaluation parameters of each sub-data block in the N sub-data blocks; the processing unit 902 is used to determine the data block processing mode for the data block to be coded according to the coding information of the data block to be coded and the coding information of the N sub-data blocks, specifically for:

[0221] If the precoding estimation parameter of the data block to be encoded is greater than or equal to the precoding estimation parameters of the N sub-data blocks;

[0222] If the precoding estimation parameter of the data block to be encoded is less than the precoding estimation parameters of the N sub-data blocks, the complete block encoding mode is used as the encoding mode of the data block to be encoded.

[0223] In one embodiment, the processing unit 902 is further configured to:

[0224] Obtaining the complexity parameter of the data block to be encoded;

[0225] If the complexity parameter of the to-be-encoded data block meets the overall encoding condition, the complete block encoding mode is used as the encoding mode of the to-be-encoded data block, and image encoding processing is performed on the to-be-encoded data block according to the complete block encoding mode;

[0226] If the complexity parameter of the data block to be encoded does not meet the overall encoding condition, the step of dividing the data block to be encoded into N sub-data blocks is triggered.

[0227] In one embodiment, the complexity parameter of the data block to be encoded includes: any one or more of a distortion estimation parameter of the data block to be encoded, a spatial information parameter of the data block to be encoded, and a temporal information parameter of the data block to be encoded;

[0228] The complexity index information of the i-th sub-data block among N sub-data blocks includes: any one or more of the distortion estimation parameters of the i-th sub-data block, the spatial information parameters of the i-th sub-data block, and the temporal information parameters of the i-th sub-data block, where i is a positive integer less than or equal to N.

[0229] In one embodiment, the processing unit 902 is configured to, after determining the target sub-data block in the N sub-data blocks as the new data block to be encoded according to the block encoding mode, further be configured to:

[0230] The target sub-data block is further divided into N sub-data blocks;

[0231] Determine the encoding mode of the target sub-data block according to the complexity parameters between the N sub-data blocks obtained by dividing the target sub-data block;

[0232] The determined encoding mode of the target sub-data block includes: a block encoding mode of further dividing the target sub-data block into N sub-data blocks, or a complete block encoding mode of directly performing image encoding processing on the target sub-data block.

[0233] In one embodiment, Fig. 9 The various units in the video processing device 90 shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the video processing device 90 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it is possible to execute the following by running on a general-purpose computing device of a general-purpose computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 5 , Figure 6 , Figure 7 , Figure 8a or Figure 8b A computer program (including program code) for each step involved in the corresponding method shown in Fig. 9The video processing device 90 shown in the figure can be used to implement the video processing method of the embodiment of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.

[0234] The embodiment of the present application divides the data block to be encoded in the target frame image of the video to be encoded into N sub-data blocks, and determines the encoding mode of the data block to be encoded according to the complexity parameters between the N sub-data blocks. It is possible to determine a suitable encoding mode for the data block to be encoded in the target frame image, effectively improve the encoding speed of the video to be encoded, and further improve the encoding efficiency of the video to be encoded.

[0235] See also Fig.10 , Fig.10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may be a smart terminal or a server. The smart terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted device, a smart speaker, a smart watch, a smart wearable device, etc. For example, the electronic device may be Figure 4 Any terminal or server in the illustrated embodiment. The electronic device 100 at least includes a processor 1001 and a memory 1002. The processor 1001 and the memory 1002 may be connected via a bus or other means.

[0236] The processor 1001 may be a central processing unit (CPU). The processor 1001 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. The PLD may be a field-programmable gate array (FPGA), a generic array logic (GAL), etc.

[0237] The memory 1002 may include a volatile memory (Volatile Memory), such as a random-access memory (Random-Access Memory, RAM); the memory 1002 may also include a non-volatile memory (Non-Volatile Memory), such as a flash memory (Flash Memory), a solid-state drive (SSD), etc.; the memory 1002 may also include a combination of the above-mentioned types of memory.

[0238] The memory 1002 is used to store computer programs, which include computer instructions, and the processor 1001 is used to execute computer instructions. The processor 1001 (or CPU (Central Processing Unit)) is the computing core and control core of the electronic device 90, which is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement corresponding method processes or corresponding functions.

[0239] The memory 1002 stores a computer program, which is executed by the processor 1001 to implement Figure 5 , Figure 6 , Figure 7 , Figure 8a or Figure 8b Corresponding steps in the method embodiment shown; in a specific implementation, the processor 1001 runs the computer program to:

[0240] Determining a data block to be encoded in a target frame image of a video to be encoded;

[0241] Divide the data block to be encoded into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block, where N is an integer greater than or equal to 2;

[0242] Determine the complexity parameters between the N sub-data blocks according to each complexity indicator information;

[0243] If the complexity parameter satisfies the block coding condition, the block coding mode is used as the coding mode of the data block to be coded, and the target sub-data block in the N sub-data blocks is determined as a new data block to be coded according to the block coding mode;

[0244] If the complexity parameter satisfies the complete block coding condition, the complete block coding mode is used as the coding mode of the data block to be coded, and image coding processing is performed on the data block to be coded according to the complete block coding mode.

[0245] In one embodiment, the processor 1001 is configured to:

[0246] If the complexity parameter satisfies the coding decision condition, then M associated data blocks associated with the data block to be coded are determined in the target frame image, where M is a positive integer;

[0247] Obtaining associated block index parameters of the M associated data blocks, and determining a precoding order of the to-be-coded data blocks according to the associated block index parameters;

[0248] The data block to be encoded is pre-encoded according to the determined pre-encoding order, and the encoding mode for the data block to be encoded is determined.

[0249] In one embodiment, the associated block indicator parameter includes: a first number of associated data blocks whose coding mode is a block coding mode among the M associated data blocks; the processor 1001 is configured to:

[0250] If the first number is greater than or equal to the first threshold, determining that the precoding order of the to-be-encoded data block is the first precoding order;

[0251] Obtaining sub-block index parameters of N sub-data blocks according to a first precoding order;

[0252] The encoding mode for the data block to be encoded is determined according to the sub-block index parameter.

[0253] In one embodiment, the sub-block indicator parameter includes a second number of sub-data blocks whose coding mode is the block coding mode among the N sub-data blocks, and the processor 1001 is configured to:

[0254] If the second number is greater than or equal to the second threshold, the block coding mode is used as the coding mode of the data block to be encoded.

[0255] In one embodiment, the processor 1001 is configured to:

[0256] If the second number is less than the second threshold, image coding is performed on the data block to be coded to obtain coding information of the data block to be coded;

[0257] Obtaining coding information of N sub-data blocks, where the coding information of the N sub-data blocks is obtained by performing image coding processing on each of the N sub-data blocks;

[0258] A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

[0259] In one embodiment, the processor 1001 is configured to: if the first number is less than a first threshold, determine that the precoding order of the data block to be encoded is a second precoding order;

[0260] According to the second pre-coding sequence, image coding is performed on the data block to be coded, and a coding mode for the data block to be coded is determined according to coding information of the data block to be coded obtained by coding.

[0261] In one embodiment, the encoding information of the to-be-encoded data block includes an encoding distortion parameter of the to-be-encoded data block; the processor 1001 is configured to:

[0262] The coding parameter is calculated according to the coding distortion parameter of the data block to be coded and the quantization parameter for quantizing the data block to be coded;

[0263] If the encoding parameter is less than the third threshold, the complete block encoding mode is used as the encoding mode of the data block to be encoded;

[0264] If the encoding parameter is greater than or equal to the third threshold, performing image encoding processing on each of the N sub-data blocks respectively to obtain encoding information of the N sub-data blocks;

[0265] A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

[0266] In one embodiment, the coding information of the data block to be coded includes the precoding evaluation parameters of the data block to be coded; the coding information of the N sub-data blocks includes the precoding evaluation parameters of the N sub-data blocks, and the precoding evaluation parameters of the N sub-data blocks are calculated according to the precoding evaluation parameters of each sub-data block in the N sub-data blocks; the processor 1001 is used to:

[0267] If the precoding evaluation parameter of the data block to be encoded is greater than or equal to the precoding evaluation parameters of the N sub-data blocks, the block coding mode is used as the coding mode of the data block to be encoded;

[0268] If the precoding estimation parameter of the data block to be encoded is less than the precoding estimation parameters of the N sub-data blocks, the complete block encoding mode is used as the encoding mode of the data block to be encoded.

[0269] In one embodiment, the processor 1001 is configured to:

[0270] Obtaining the complexity parameter of the data block to be encoded;

[0271] If the complexity parameter of the to-be-encoded data block meets the overall encoding condition, the complete block encoding mode is used as the encoding mode of the to-be-encoded data block, and image encoding processing is performed on the to-be-encoded data block according to the complete block encoding mode;

[0272] If the complexity parameter of the data block to be encoded does not meet the overall encoding condition, the step of dividing the data block to be encoded into N sub-data blocks is triggered.

[0273] In one embodiment, the complexity parameter of the data block to be encoded includes: any one or more of a distortion estimation parameter of the data block to be encoded, a spatial information parameter of the data block to be encoded, and a temporal information parameter of the data block to be encoded;

[0274] The complexity index information of the i-th sub-data block among N sub-data blocks includes: any one or more of the distortion estimation parameters of the i-th sub-data block, the spatial information parameters of the i-th sub-data block, and the temporal information parameters of the i-th sub-data block, where i is a positive integer less than or equal to N.

[0275] In one embodiment, the processor 1001 is configured to:

[0276] The target sub-data block is further divided into N sub-data blocks;

[0277] Determine the encoding mode of the target sub-data block according to the complexity parameters between the N sub-data blocks obtained by dividing the target sub-data block;

[0278] The determined encoding mode of the target sub-data block includes: a block encoding mode of further dividing the target sub-data block into N sub-data blocks, or a complete block encoding mode of directly performing image encoding processing on the target sub-data block.

[0279] In an embodiment of the present application, by performing scene complexity analysis on the target data block to be encoded in the target video frame, analysis results for the entire target data block to be encoded and analysis results after dividing the target data block to be encoded into multiple sub-data blocks are obtained respectively, and a coding mode is selected based on these analysis results. In this way, a suitable coding mode can be more accurately determined for the current target data block to be encoded, which can effectively improve the coding speed of the target data block and improve the video coding efficiency.

[0280] In one embodiment, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the video processing method provided in the above-mentioned various optional modes.

[0281] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0282] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A video processing method, characterized in that: The method comprises: Determining a data block to be encoded in a target frame image of a video to be encoded; Divide the to-be-encoded data block into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block, where N is an integer greater than or equal to 2; Determine the complexity parameters between the N sub-data blocks according to each complexity indicator information; If the complexity parameter satisfies the block coding condition, the block coding mode is used as the coding mode of the data block to be coded, and the target sub-data block in the N sub-data blocks is determined as a new data block to be coded according to the block coding mode; If the complexity parameter satisfies the complete block coding condition, the complete block coding mode is used as the coding mode of the data block to be coded, and image coding processing is performed on the data block to be coded according to the complete block coding mode; If the complexity parameter meets the coding decision condition, then M associated data blocks associated with the data block to be encoded are determined in the target frame image, where M is a positive integer; associated block index parameters of the M associated data blocks are obtained, and a precoding order of the data block to be encoded is determined according to the associated block index parameters; precoding processing is performed on the data block to be encoded according to the determined precoding order, and a coding mode for the data block to be encoded is determined.

2. The method according to claim 1, characterized in that The associated block indicator parameter includes: a first number of associated data blocks whose coding mode is the block coding mode among the M associated data blocks; The determining the precoding order of the data block to be encoded according to the associated block indicator parameter comprises: if the first number is greater than or equal to a first threshold, determining the precoding order of the data block to be encoded to be a first precoding order; The precoding process is performed on the to-be-coded data block according to the determined precoding order to determine the coding mode for the to-be-coded data block, including: Acquire sub-block index parameters of the N sub-data blocks according to the first precoding order; The encoding mode for the to-be-encoded data block is determined according to the sub-block indicator parameter.

3. The method according to claim 2, characterized in that The sub-block indicator parameter includes: a second number of sub-data blocks whose coding mode is the block coding mode in the N sub-data blocks, and determining the coding mode for the to-be-coded data block according to the sub-block indicator parameter includes: If the second number is greater than or equal to a second threshold, the block coding mode is used as the coding mode of the data block to be encoded.

4. The method according to claim 3, characterized in that The step of determining the encoding mode of the to-be-encoded data block according to the sub-block indicator parameter further includes: If the second number is less than the second threshold, performing image coding processing on the data block to be encoded to obtain coding information of the data block to be encoded; Obtaining coding information of the N sub-data blocks, where the coding information of the N sub-data blocks is obtained by performing image coding processing on each of the N sub-data blocks; and A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

5. The method according to claim 1, characterized in that The associated block indicator parameter includes: a first number of associated data blocks whose coding mode is the block coding mode among the M associated data blocks; The determining the precoding order of the data blocks to be encoded according to the associated block indicator parameter comprises: if the first number is less than a first threshold, determining the precoding order of the data blocks to be encoded to be a second precoding order; The precoding process is performed on the to-be-coded data block according to the determined precoding order to determine the coding mode for the to-be-coded data block, including: According to the second pre-coding sequence, image coding processing is performed on the data block to be coded, and a coding mode for the data block to be coded is determined according to coding information of the data block to be coded obtained by coding.

6. The method according to claim 5, characterized in that The encoding information of the to-be-encoded data block includes an encoding distortion parameter of the to-be-encoded data block; The step of determining the encoding mode of the data block to be encoded according to the encoding information of the data block to be encoded obtained by encoding includes: Calculate the encoding parameter according to the encoding distortion parameter of the data block to be encoded and the quantization parameter for quantizing the data block to be encoded; If the encoding parameter is less than a third threshold, using the complete block encoding mode as the encoding mode of the data block to be encoded; If the encoding parameter is greater than or equal to the third threshold, performing image encoding processing on each of the N sub-data blocks respectively to obtain encoding information of the N sub-data blocks; A coding mode for the data block to be coded is determined according to the coding information of the data block to be coded and the coding information of the N sub-data blocks.

7. The method according to claim 4 or 6, characterized in that The coding information of the data block to be coded includes the precoding evaluation parameters of the data block to be coded; the coding information of the N sub-data blocks includes the precoding evaluation parameters of the N sub-data blocks, and the precoding evaluation parameters of the N sub-data blocks are calculated according to the precoding evaluation parameters of each sub-data block in the N sub-data blocks; The step of determining the encoding mode of the data block to be encoded according to the encoding information of the data block to be encoded and the encoding information of the N sub-data blocks comprises: If the precoding evaluation parameter of the data block to be encoded is greater than or equal to the precoding evaluation parameters of the N sub-data blocks, the block coding mode is used as the coding mode of the data block to be encoded; If the precoding estimation parameter of the data block to be encoded is less than the precoding estimation parameters of the N sub-data blocks, the complete block encoding mode is used as the encoding mode of the data block to be encoded.

8. The method according to any one of claims 1 to 3, characterized in that Before dividing the to-be-encoded data block into N sub-data blocks, the method further includes: Obtaining a complexity parameter of the data block to be encoded; If the complexity parameter of the data block to be encoded meets the overall encoding condition, the complete block encoding mode is used as the encoding mode of the data block to be encoded, and image encoding processing is performed on the data block to be encoded according to the complete block encoding mode; If the complexity parameter of the data block to be encoded does not satisfy the overall encoding condition, the step of dividing the data block to be encoded into N sub-data blocks is triggered.

9. The method according to claim 8, characterized in that The complexity parameter of the data block to be encoded includes: any one or more of a distortion estimation parameter of the data block to be encoded, a spatial information parameter of the data block to be encoded, and a temporal information parameter of the data block to be encoded; The complexity index information of the i-th sub-data block among the N sub-data blocks includes: any one or more of the distortion estimation parameters of the i-th sub-data block, the spatial information parameters of the i-th sub-data block, and the temporal information parameters of the i-th sub-data block, where i is a positive integer less than or equal to N.

10. The method according to claim 1, characterized in that After determining the target sub-data block in the N sub-data blocks as a new data block to be encoded according to the block encoding mode, the method further includes: Further dividing the target sub-data block into N sub-data blocks; Determining a coding mode of the target sub-data block according to a complexity parameter between the N sub-data blocks obtained by dividing the target sub-data block; The determined encoding mode of the target sub-data block includes: a block encoding mode for further dividing the target sub-data block into N sub-data blocks, or a complete block encoding mode for directly performing image encoding processing on the target sub-data block.

11. A video processing device, characterized in that: The video processing device comprises: A determination unit, used to determine a data block to be encoded in a target frame image of a video to be encoded; A processing unit, used to divide the to-be-encoded data block into N sub-data blocks, and obtain complexity index information corresponding to each sub-data block, where N is an integer greater than or equal to 2; The processing unit is further used to determine the complexity parameters between the N sub-data blocks according to each complexity indicator information; The processing unit is further configured to, if the complexity parameter satisfies a block coding condition, use a first coding mode as a coding mode of the data block to be coded, and determine a target sub-data block among the N sub-data blocks as a new data block to be coded according to the first coding mode; The processing unit is further configured to, if the complexity parameter satisfies a complete block coding condition, use the second coding mode as the coding mode of the to-be-coded data block, and perform image coding processing on the to-be-coded data block according to the second coding mode; The processing unit is further used to determine, if the complexity parameter meets the coding decision condition, M associated data blocks associated with the data block to be encoded in the target frame image, where M is a positive integer; obtain associated block index parameters of the M associated data blocks, and determine the precoding order of the data block to be encoded according to the associated block index parameters; perform precoding processing on the data block to be encoded according to the determined precoding order, and determine the coding mode for the data block to be encoded.

12. An electronic device, characterized in that: The electronic device comprises: a memory and a processor; The memory is used to store a computer program, and the processor is used to execute the computer program to implement the video processing method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is read and executed by a processor, the video processing method according to any one of claims 1 to 10 is implemented.

14. A computer program product, characterized in that The computer program product comprises computer instructions, which are stored in a computer-readable storage medium; when the computer instructions are executed by a processor, the video processing method according to any one of claims 1 to 10 is implemented.