A data processing method, device, computer equipment and storage medium

Abstract

The present application proposes a data processing method, device, computer equipment and storage medium. The method includes: obtaining feature information of a target coding unit in an image to be encoded, the feature information at least including the pixel value variance and size information of the target coding unit; determining a data search method for the target coding unit according to the feature information of the target coding unit, the data search method including at least one of integer pixel search, sub-pixel search and motion information search, the motion information including motion information in the coding unit that has completed the search and motion information in the search starting point of the target coding unit; performing data search using the determined data search method to obtain a search result of the target coding unit, the search result including one or both of integer pixel search results and sub-pixel search results; determining motion estimation information of the target coding unit according to the search results. The present application can reduce the time overhead of the encoder, thereby improving the encoding efficiency of the encoder.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a data processing method, device, computer equipment and storage medium. Background Art

[0002] With the continuous development of Internet technology and the increasing demand for video cloud computing, in the process of video transmission, in order to reduce the bandwidth and storage space occupied during transmission, the video needs to be encoded. During video encoding, each video frame in the video will be divided into image blocks, which are also called coding blocks. When encoding the coding blocks, prediction must be performed first. During prediction, the coding blocks can be divided into multiple coding units (Coding Unit, CU). Prediction is divided into intra-frame prediction and inter-frame prediction. Among them, the key technologies in inter-frame prediction include motion estimation. Motion estimation is the process of searching for the best reference frame in a certain range of reference frames, and searching for the best matching block of the coding unit in the best reference frame.

[0003] In the motion estimation of existing video encoders, each coding unit of the encoder is usually subjected to integer pixel search and sub-pixel search in the reference frame by block matching to obtain the optimal motion information. However, integer pixel search and sub-pixel search are time-consuming, which will bring a large time overhead to the encoder, resulting in low encoding efficiency of the encoder. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, computer device, and storage medium, which can reduce the time overhead of an encoder, thereby improving the encoding efficiency of the encoder.

[0005] On the one hand, an embodiment of the present application provides a data processing method, which specifically includes:

[0006] Acquire characteristic information of a target coding unit in the image to be encoded, wherein the characteristic information includes at least pixel value variance and size information of the target coding unit;

[0007] Determining a data search mode for the target coding unit according to the characteristic information of the target coding unit, the data search mode comprising at least one of an integer pixel search, a sub-pixel search, and a motion information search, the motion information comprising motion information in a coding unit that has completed the search and motion information in a search start point of the target coding unit;

[0008] Performing data search using the determined data search method to obtain a search result of the target coding unit, wherein the search result includes one or both of an integer pixel search result and a sub-pixel search result;

[0009] Motion estimation information of the target coding unit is determined according to the search result.

[0010] On one hand, the present application provides a data processing device, which has the function of implementing the above-mentioned data processing method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes:

[0011] An acquisition unit, configured to acquire feature information of a target coding unit in the image to be encoded, wherein the feature information at least includes pixel value variance and size information of the target coding unit;

[0012] a determining unit, configured to determine a data search mode of the target coding unit according to the characteristic information of the target coding unit, wherein the data search mode includes at least one of an integer pixel search, a sub-pixel search, and a motion information search, wherein the motion information includes motion information in the coding unit that has completed the search and motion information in the search starting point of the target coding unit;

[0013] A processing unit, configured to perform data search using the determined data search method to obtain a search result of the target coding unit, wherein the search result includes one or both of an integer pixel search result and a sub-pixel search result;

[0014] The determination unit is further used to determine the motion estimation information of the target coding unit according to the pixel search result.

[0015] On one hand, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the methods in the above embodiments.

[0016] On the one hand, an embodiment of the present application provides a computer storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the methods in the above embodiments are executed.

[0017] On the one hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. When the computer instructions are executed by a processor of a computer device, the methods in the above embodiments are executed.

[0018] Through the data processing method of the embodiment of the present application, by obtaining the characteristic information of the coding unit, the characteristic information includes the pixel value variance and size information of the coding unit. According to the pixel value variance and size information of the coding unit, the encoder only needs to perform integer pixel search and sub-pixel search on part of the coding units. Compared with the existing method in which the encoder needs to perform integer pixel search and sub-pixel search on each coding unit, the time overhead of the encoder motion estimation can be reduced, thereby improving the encoding efficiency of the encoder. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 It is a schematic diagram of the architecture of a data processing system provided in an embodiment of the present application;

[0021] Figure 2a It is a schematic diagram of a computer device interface provided by an embodiment of the present application;

[0022] Figure 2b is a schematic diagram of a target encoding unit provided in an embodiment of the present application;

[0023] Figure 2c is a processing schematic diagram for a target coding unit provided by an embodiment of the present application;

[0024] Figure 3 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of a target encoding unit provided in an embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of a data search process provided by an embodiment of the present application;

[0027] Figure 6 It is a flowchart of a data search provided by an embodiment of the present application;

[0028] Figure 7 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0029] Figure 8 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0032] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0033] In order to better understand the embodiments of the present application, the professional terms involved in the embodiments of the present application are introduced below:

[0034] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and is flexible and convenient. At present, the background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0035] At present, cloud technology is mainly divided into cloud basic technology and cloud application. Cloud basic technology can be further subdivided into: cloud computing, cloud storage, database and big data, etc.; cloud application can be further subdivided into: medical cloud, cloud Internet of Things, cloud security, cloud call, private cloud, public cloud, hybrid cloud, cloud gaming, cloud education, cloud conferencing, cloud social networking and artificial intelligence cloud services, etc.

[0036] From the perspective of basic technology, the data processing method of this application involves cloud computing under cloud technology; from the perspective of application, the data processing method of this application involves artificial intelligence cloud services under cloud technology.

[0037] Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, allowing various application systems to obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". From the user's perspective, the resources in the "cloud" are infinitely scalable and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0038] In the present application, a computer device determines a data search method for a target coding unit based on characteristic information of the target coding unit, and uses the data search method to perform a data search. Obtaining integer pixel search results and sub-pixel search results of the target coding unit involves large-scale calculations, which require huge computing power and storage space. Therefore, in the present application, a computer device can obtain sufficient computing power and storage space through cloud computing technology, and then execute the motion estimation information of the target coding unit determined based on the integer pixel search results and sub-pixel search results involved in the present application.

[0039] The so-called artificial intelligence cloud service is generally also called AIaaS (AIas a Service, Chinese for "AI as a Service"). This is the current mainstream service mode of artificial intelligence platforms. Specifically, the AIaaS platform will split several common AI services and provide independent or packaged services in the cloud. This service model is similar to opening an AI theme mall: all developers can access and use one or more artificial intelligence services provided by the platform through API interfaces. Some senior developers can also use the AI ​​framework and AI infrastructure provided by the platform to deploy and operate their own cloud artificial intelligence services.

[0040] The data processing method involved in this application can be encapsulated as an artificial intelligence service, and only one interface is exposed to the outside. When the function of determining the motion estimation information of the coding unit involved in this application needs to be used in a certain business scenario, the motion estimation information of the target coding unit in the image to be encoded can be determined by calling this interface.

[0041] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0042] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0043] The solution provided in the embodiment of the present application belongs to the computer vision technology under the field of artificial intelligence.

[0044] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphics processing so that the computer processes the images into images that are more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0045] It should be noted that the present application mainly involves motion estimation of image processing. The basic idea of ​​motion estimation is to divide each frame of an image sequence into many non-overlapping maximum coding units or macroblocks, and then find the predicted image block that is most similar to the maximum coding unit or macroblock according to a certain matching criterion within a specific search range of each maximum coding unit or macroblock to the reference frame. The relative displacement between the predicted image block and the current block (maximum coding unit or macroblock) is the motion vector. When compressing a video, it is only necessary to save the motion vector and the residual data (the difference between the predicted image block and the current block). At the decoding end, according to the position indicated by the motion vector, the corresponding block or macroblock is found from the adjacent decoded reference frame image. After adding the predicted image block and the residual data, the reconstructed data is obtained. Therefore, motion estimation can reduce inter-frame redundancy and reduce the number of bits of video transmission. It should be noted that motion estimation can be applied to inter-frame prediction. Inter-frame prediction uses the correlation of the video time domain and the pixels of the adjacent encoded image (i.e., the reference frame) to predict the pixels of the current image to effectively remove video redundancy.

[0046] In the present application, a computer device obtains characteristic information of a target coding unit in the image to be encoded by performing data processing on the image to be encoded; then, the computer device determines a data search method for the target coding unit according to the characteristic information of the target coding unit, and performs a data search using the data search method to obtain a search result of the target coding unit, wherein the search result includes one or both of an integer pixel search result and a sub-pixel search result; finally, the computer device determines motion estimation information of the target coding unit according to the search result.

[0047] The present application can be applied to the following scenarios: characteristic information of a target coding unit can be obtained through image processing of an image to be encoded, a data search method for the target coding unit is determined according to the characteristic information of the target coding unit, and a data search is performed using the data search method to obtain a search result of the target coding unit; and then motion estimation information between the target coding unit and a matching coding unit in a reference image of the image to be encoded is determined.

[0048] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of a data processing system architecture provided by an embodiment of the present application. The data processing system architecture diagram includes: a server 140 and a computer device cluster, wherein the computer device cluster may include: computer device 110, computer device 120, ..., computer device 130, etc. The computer device cluster and the server 140 may be directly or indirectly connected via wired or wireless communication, and the present application does not limit this.

[0049] Figure 1 The server 140 shown can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0050] Figure 1 The computer device 110, computer device 120, computer device 130, etc. shown in the figure can be a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile Internet device (MID), a vehicle, a roadside device, an aircraft, a wearable device such as a smart watch, a smart bracelet, a pedometer, etc., and other smart devices with data processing functions.

[0051] In a possible implementation, taking the computer device 110 as an example, the computer device 110 receives a video to be processed uploaded by a user, and the computer device sends the video to be processed to the server 110. The server 110 performs image processing on the video to be processed to obtain multiple image frames to be processed, that is, multiple images to be encoded. The computer device 110 obtains feature information of a target coding unit in the image to be encoded, and the feature information includes the pixel value variance and size information of the target coding unit; the computer device determines a data search method for the target coding unit according to the feature information of the target coding unit, wherein the data search method includes one or more of an integer pixel search, a sub-pixel search, and a motion information search; the computer device 110 performs a data search using the data search method to obtain a search result of the target coding unit, and the search result includes one or both of an integer pixel search result and a sub-pixel search result; the computer device 110 determines the motion estimation information of the target coding unit according to the search result.

[0052] Of course, the data search method of the target coding unit is determined according to the characteristic information of the target coding unit, and the data search method includes one or two of integer pixel search, sub-pixel search, and motion information search. The motion information search includes searching from the coding unit that has completed the search and the search starting point of the target coding unit; the data search is performed using the data search method to obtain the search result of the target coding unit, and the search result includes one or two of integer pixel search result and sub-pixel search result; the motion estimation information of the target coding unit is determined according to the search result. It can also be performed by the server 110. The server 140 can send the motion estimation information of the target coding unit to the computer device 110, and the computer device 110 can subsequently determine the relative motion vector between the image to be encoded and the reference image according to the motion estimation information of the target coding unit.

[0053] It can be understood that the system architecture diagram described in the embodiment of the present application is for more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0054] like Figure 2a As shown, Figure 2aA schematic diagram of a computer device interface provided in an embodiment of the present application, a user uploads a video to be processed to the computer device, and the computer device has a built-in image recognition system. The computer device can perform image recognition processing on the video to be processed uploaded by the user through the image recognition system, so that the image to be processed can be pre-processed, such as frame division, so as to divide the video to be processed into multiple images to be encoded, including image 1, image 2 and image 3. It should be noted that the computer device can be a user terminal or a server.

[0055] In a possible implementation, assuming that image 1 is an image to be encoded, the computer device divides the image to be encoded into multiple encoding units. Figure 2b , Figure 2b is a schematic diagram of a target coding unit provided in an embodiment of the present application. Figure 2b As shown, the shaded portion in the figure is a schematic diagram of a target coding unit provided by an embodiment of the present application, and the shape of the target coding unit is a 4×4 square. The computer device obtains feature information of the target coding unit in image 1, wherein the feature information of the target coding unit includes the pixel value variance and size information of the target coding unit, wherein the size information may include the length and width of the target coding unit, specifically, the ratio between the length and the width of the target coding unit. Assuming that the size information of the target coding unit is 4×4, the ratio between the length and the width of the target coding unit is equal to 1.

[0056] In one possible implementation, see Figure 2c , Figure 2c This is a processing schematic diagram for a target coding unit provided by an embodiment of the present application. The computer device determines a data search method for the target coding unit based on the characteristic information of the target coding unit, wherein the data search method may include one or two of an integer pixel search, a sub-pixel search, and a motion information search. Then, the computer device performs a data search using the data search method to obtain a search result of the target coding unit, and the search result includes one or two of an integer pixel search result and a sub-pixel search result; finally, the computer device determines the motion estimation information of the target coding unit based on the search result.

[0057] When the video compression processing is performed subsequently, the computer device only needs to save the motion estimation information of the target coding unit. At the decoding end, the prediction coding unit corresponding to the target coding unit is found from the adjacent decoded reference frame image according to the position indicated by the motion estimation information. The prediction coding unit and the motion estimation information are added to obtain the reconstructed data. Through the present application, for the video to be processed uploaded by the user, the image to be encoded can be obtained through image recognition. The computer device can finally determine the motion estimation information of the target coding unit in the image to be encoded through the data processing method provided by the present application. Therefore, through the data processing method of the present application, some coding units can be made not to perform integer pixel search and sub-pixel search. For the encoder in the computer device, the time overhead of the encoder can be reduced, thereby improving the encoding efficiency of the encoder.

[0058] See also Figure 3 , Figure 3 It is a flowchart of a data processing method provided in an embodiment of the present application. Figure 3 is a flow chart of a data processing method provided in an embodiment of the present application. The method is applied to a computer device, such as Figure 3 As shown, the data processing method may include steps S310 to S340. Among them:

[0059] Step S310: Acquire characteristic information of a target coding unit in the image to be encoded, where the characteristic information at least includes pixel value variance and size information of the target coding unit.

[0060] In a possible implementation, first, a user uploads a video to be processed, and a computer device obtains the video to be processed uploaded by the user. Then, the computer device performs a frame division operation on the video to be processed, that is, divides the video to be processed into multiple images to be processed. Then, the computer device obtains any one of the multiple images to be processed as an image to be encoded. Finally, the computer device divides the image to be encoded into multiple encoding units. Among them, the target encoding unit obtained by the computer device in the image to be encoded can be a encoding unit currently being encoded, or can be any encoding unit among the multiple encoding units.

[0061] For example, when a computer device encodes an image to be encoded, it will first divide the image to be encoded into Coding Tree Units (CTU) of equal size, and then encode it using the Coding Tree Unit as the basic unit. The size of the Coding Tree Unit is generally a 64×64 block. During the encoding process, the Coding Tree Unit can be further divided into coding blocks of different sizes. Furthermore, the coding block can be divided into multiple coding units, wherein the organizational structure of the coding units in the coding block is a quadtree structure, that is, four coding units are generated by dividing the coding block once. See Figure 4 , Figure 4 is a schematic diagram of a target coding unit provided in an embodiment of the present application. Figure 4 As shown, a 64×64 block represents a coding tree unit with a width of 64 pixels and a height of 64 pixels obtained by dividing the image to be encoded. It can be seen that a 64×64 coding tree unit can be encoded as a 64×64 coding block, or it can be divided into 4 equal-sized 32×32 coding blocks, and then each 32×32 coding block is encoded. According to the rate-distortion criterion, for each 64×64 coding block, the rate-distortion cost of the 64×64 coding block is compared with the sum of the rate-distortion costs of the 4 32×32 coding blocks, and the division method with the smaller rate-distortion cost is selected for division. Each 32×32 coding block can be divided into 4 equal-sized 16×16 coding blocks, that is Figure 4 The shaded part in the figure is a coding unit. That is, for each 32×32 coding block, it is necessary to compare the rate-distortion cost of a 32×32 coding block with the sum of the rate-distortion costs of four 16×16 coding blocks, and select the division method with the smaller rate-distortion cost for division.

[0062] In a possible implementation, after the computer device obtains the target coding unit in the image to be encoded, the computer device further obtains feature information of the target coding unit, wherein the feature information of the target coding unit includes the pixel value variance and size information of the target coding unit. Assuming that the target coding unit is a 16×16 coding block, the computer device can calculate the pixel value variance of the 16×16 target coding unit according to the variance formula. In addition, the size information can be the ratio between the length and width of the target coding unit. For example, if the target coding unit is a 16×16 coding block, the size information of the target coding unit is 1. For another example, if the target coding unit is a 16×8 coding block, the size information of the target coding unit is 2.

[0063] Step S320: Determine a data search method for the target coding unit based on characteristic information of the target coding unit, wherein the data search method includes at least one of an integer pixel search, a sub-pixel search, and a motion information search, and the motion information includes motion information in the coding unit that has completed the search and motion information in the search starting point of the target coding unit.

[0064] In a possible implementation, in the process of motion estimation of the image to be coded, a coding motion search needs to be performed. The motion search may include three parts: pre-search, integer pixel search and sub-pixel search. The search methods of the three search stages may adopt the full search method, the neighbor method or other fast algorithms. Assuming that the full search method is adopted for pre-search, the image to be coded is moved within a certain search range with a preset step length. Each time the movement is performed, the cost value of the predicted image corresponding to the candidate point can be calculated (the image block can be represented by a certain pixel point in the image block, and the relative position of the point in the predicted image is consistent with the relative position of the point representing the image to be coded in the image to be coded, so the movement of the image block can be regarded as the movement of the point), and then the point with the smallest cost value is selected as the best matching point. The predicted image corresponding to the best matching point is the matching image that best matches the image to be coded. Although the search methods of the three search stages are similar, the search starting point and the search range are different. According to the order of pre-search, integer pixel search and sub-pixel search, the matching point determined in each stage is the search starting point of the next stage.

[0065] In specific implementation, in the pre-search stage, the current image to be encoded is pre-searched on two reference frames respectively, where the image to be encoded can be specifically the largest coding unit (LCU) or macroblock (MB). The matching point with the minimum search cost that matches the current image to be encoded is determined on the two reference frames respectively. The matching point is the matching point in the pre-search stage, and the predicted image block corresponding to the matching point is the image block closest to the above-mentioned image to be encoded. It should be noted that the matching point in the pre-search stage is the starting point. The matching point of the integer pixel search stage is determined according to the prediction unit (PU) in the image to be encoded. The matching point is the point with the minimum cost of the candidate point within the integer pixel search range.

[0066] In a possible implementation, the integer pixel search step includes: first, the computer device takes the position pointed to by the initial motion vector as the starting point, wherein the starting point can specifically be a matching point determined according to the pre-search; then, the computer device selects a preset search template to perform an integer point search to find an integer pixel matching block with the minimum rate distortion cost within a preset search range, and determines the integer pixel matching block as the matching block of the prediction unit. The integer pixel search range is a range covering all integer pixel points, which can be set by the user or can be a default normal search range, such as within plus or minus 12 integer pixel points around the starting point. Among them, the search templates for the integer pixel search include full search, diamond search, hexagonal search, EPZS (Enhance Predictive Zonal Search) and UMH (Unsymmetrical-cross Multi-Hexagon-grid Search), etc.

[0067] For example, if Figure 5 As shown, Figure 5 This is a schematic diagram of a search process for a pixel search provided by an embodiment of the present application. The search method can be specifically an integer pixel search, and the search range is (d+m)×(d+n), where the padding block is the target coding unit, the blank block is the matching block of the target coding unit, m is the length of the target coding unit, n is the width of the target coding unit, the upper left corner coordinates of the prediction unit are (p, q), and the upper left corner coordinates of the matching block of the target coding unit are (p+u, q+v), and the motion vector of the target coding unit can be obtained as (u, v).

[0068] Full search is an exhaustive search within the search area of ​​the reference frame, that is, traversing every pixel within the search range. This method is the simplest, most direct, and has the best performance, but the amount of calculation is too large to be convenient for real-time encoding processing.

[0069] Diamond search, also known as diamond search. This search method uses diamond as the search template. First, a large diamond is selected as the template for search. When the point with the lowest rate-distortion cost falls on the center point of the large diamond template, a small diamond template is used for search to achieve accurate positioning of the matching block. This search method can largely avoid falling into local minimization, but whether it is a moving image sequence or a static image sequence, it needs to go through the search process from a large template to a small template, which will cause a large search redundancy.

[0070] Hexagonal search is similar to diamond search. This search method replaces the large template in diamond search with a hexagon with a step size of 2, and the small template with a square with a step size of 1 for searching. Its advantages and disadvantages are also similar to diamond search.

[0071] In a possible implementation, pixel search may include: half-pixel search, 1 / 4 pixel search and 1 / 8 pixel search. In the half-pixel search, the position corresponding to the upper left corner of the matching block obtained after the whole pixel search is used as the starting point, the surrounding reference pixels are interpolated, and all the half-pixel points in the eight surrounding neighborhoods are fully searched, and the half-pixel point corresponding to the motion vector with the smallest rate-distortion cost is selected, and the half-pixel point is determined as the matching block of the prediction unit. The 1 / 4 pixel search, the position corresponding to the upper left corner of the matching block obtained by the half-pixel search is used as the starting point, the surrounding reference pixels are interpolated, and all the 1 / 4 pixel points in the eight surrounding neighborhoods are fully searched, and the 1 / 4 pixel point corresponding to the motion vector with the smallest rate-distortion cost is selected, and the 1 / 4 pixel point is determined as the matching block of the prediction unit. 1 / 8 pixel search, starting from the position corresponding to the upper left corner of the matching block obtained by the 1 / 4 pixel search, interpolate the surrounding reference pixels, and search all 1 / 8 pixel points in the eight neighborhoods around the matching block, select the 1 / 8 pixel point corresponding to the motion vector with the smallest rate-distortion cost, and determine the 1 / 8 pixel point as the matching block of the prediction unit.

[0072] Step S330: performing data search using the determined data search method to obtain search results of the target coding unit, wherein the search results include one or both of integer pixel search results and sub-pixel search results.

[0073] It should be noted that when performing pixel search, it is not necessary to perform sub-pixel search, that is, stop searching after searching for whole pixels. Also, when performing sub-pixel search, it is not necessary to perform 1 / 8 pixel search, and it can be performed to stop searching for half pixels or 1 / 4 pixels. The precision of the sub-pixel search that needs to be performed can be determined according to the motion vector precision, so that the matching block of the prediction unit can be searched within the minimum precision range.

[0074] like Figure 6 As shown, Figure 6 It is a flowchart of a pixel search provided by an embodiment of the present application. In one possible implementation, a computer device obtains the ratio between the length and width of a target coding unit according to the size information, and obtains a first preset threshold value; the computer device compares the ratio with the first preset threshold value, and if the ratio is greater than the first preset threshold value, it is determined that the target coding unit skips the integer pixel search, that is, the data search method of the target coding unit includes a motion information search. It should be noted that at this time, the result obtained by the motion information search is used as the integer pixel search result of the target coding unit. Among them, a parent coding unit refers to a coding unit whose pixels include pixels of a target coding unit, and a child coding unit refers to a coding unit whose pixels are included in the target coding unit.

[0075] In specific implementation, first, the computer device can obtain respective motion information from its own search starting point, the parent coding unit that has completed the integer pixel search, and the child coding unit that has completed the integer pixel search, wherein the motion information can be specifically integer pixel motion information, or sub-pixel motion information, which is not limited in this application. Then, the computer device compares the rate-distortion cost of these motion information, and uses the motion information with the smallest rate-distortion cost as the integer pixel search result of the target coding unit. Among them, the rate-distortion cost, that is, the coding cost, is a measurement indicator for evaluating the coding performance based on the rate-distortion theory in video coding. In view of this, determining the motion information with a smaller rate-distortion cost as the integer pixel search result of the target coding unit can improve the coding performance of the target coding unit in the coding process. It should be noted that the rate-distortion cost involved in the embodiment of the present application can also be one, multiple or all of rate distortion optimization (RDO), sum of absolute difference (SAD) or sum of absolute transformed difference (SATD).

[0076] In one possible implementation, if the ratio is greater than a first preset threshold, the computer device determines that the target coding unit can skip the integer pixel search and replace the integer pixel search with a motion information search. At this time, the search method for determining the target coding unit at least includes a motion information search.

[0077] Furthermore, the computer device may also obtain a second preset threshold value, and if the ratio is greater than the second preset threshold value, it is determined that the target coding unit may also skip the pixel search, and at this time, the data search method of the target coding unit only includes the motion information search. According to the above description that the data search method includes the motion information search, the search result of the target coding unit may be determined.

[0078] In a possible implementation, if the ratio is less than or equal to the first preset threshold, the computer device obtains a third preset threshold and compares the pixel value variance of the target coding unit with the third preset threshold. If the pixel value variance is less than the third preset threshold, the computer device determines that the target coding unit can skip the integer pixel search and replace the integer pixel search with the motion information search. At this time, the search method for determining the target coding unit at least includes the motion information search.

[0079] Further, if the ratio is greater than the second preset threshold, the target coding unit can also skip the sub-pixel search. At this time, the data search method for determining the target coding unit only includes the motion information search, and the computer device can determine the integer pixel search result of the target coding unit according to the data search method. It should be noted that there is no connection between the first preset threshold, the second preset threshold and the third preset threshold, and the user can make corresponding settings according to the specific situation and actual needs.

[0080] In one possible implementation, if the ratio is less than or equal to a first preset threshold, the pixel value variance is less than a third preset threshold, and the ratio is less than or equal to a second preset threshold, then it is determined that the data search method for the target coding unit includes a motion information search and a pixel search. The computer device can determine the integer pixel search result and the pixel search result of the target coding unit according to the data search method. At this time, the result obtained by the motion information search is used as the integer pixel search result of the target coding unit.

[0081] In one possible implementation, if the ratio is less than or equal to a first preset threshold, the ratio is greater than a second preset threshold, and the pixel value variance is greater than or equal to a third preset threshold, then it is determined that the data search method for the target coding unit only includes integer pixel search, and the computer device can directly determine the integer pixel search results of the target coding unit based on the integer pixel search for the target coding unit.

[0082] Step S340: Determine motion estimation information of the target coding unit according to the search result.

[0083] In a specific implementation, after performing a pixel search on a target coding unit, a computer device may obtain a search result of the target coding unit, and then the computer device determines the motion estimation information of the target coding unit according to the search result. The search result may include one or both of an integer pixel search result and a sub-pixel search result. Specifically, the search result may include an integer pixel search result, and the integer pixel search result is used as the motion estimation information of the target coding unit. The search result may also include an integer pixel search result and a sub-pixel search result, and the integer pixel search result and the sub-pixel search result are combined as the motion estimation information of the target coding unit.

[0084] In a possible implementation manner, after the motion estimation information of the target coding unit is determined, the video may be subsequently compressed according to the motion estimation information.

[0085] Through the method provided in the embodiment of the present application, by obtaining the characteristic information of the coding unit, the characteristic information includes the pixel value variance and size information of the coding unit. According to the pixel value variance and size information of the coding unit, the encoder only needs to perform integer pixel search and sub-pixel search on part of the coding units. Compared with the existing method in which the encoder needs to perform integer pixel search and sub-pixel search on each coding unit, the time overhead of the encoder motion estimation can be reduced, thereby improving the encoding efficiency of the encoder.

[0086] See also Figure 7 , Figure 7 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Figure 7 The data processing device shown can be used to perform the above Figure 3 to Figure 6 The device may be a computer device, or a device in a computer device, or a device that can be used in conjunction with a computer device. Furthermore, the computer device may be a user terminal or a server.

[0087] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms. The data processing device may include:

[0088] An acquisition unit 710 is used to acquire feature information of a target coding unit in the image to be encoded, wherein the feature information at least includes pixel value variance and size information of the target coding unit;

[0089] a determining unit 720, configured to determine a data search mode of the target coding unit according to the characteristic information of the target coding unit, wherein the data search mode includes at least one of an integer pixel search, a sub-pixel search, and a motion information search, wherein the motion information includes motion information in the coding unit that has completed the search and motion information in the search start point of the target coding unit;

[0090] A processing unit 730 is configured to perform a data search using the determined data search method to obtain a search result of the target coding unit, wherein the search result includes one or both of an integer pixel search result and a sub-pixel search result;

[0091] The determination unit 720 is further configured to determine the motion estimation information of the target coding unit according to the pixel search result.

[0092] In a possible implementation, the size information includes the length and width of the target coding unit, and the determination unit 720 determines the data search method of the target coding unit according to the characteristic information of the target coding unit, including:

[0093] Acquire a ratio between a length and a width of the target coding unit according to the size information;

[0094] If the ratio is greater than a first preset threshold, determining that the search method of the target coding unit includes the motion information search; or,

[0095] If the ratio is greater than a first preset threshold and the ratio is greater than a second preset threshold, and the first preset threshold is different from the second preset threshold, it is determined that the search method of the target coding unit only includes the motion information search.

[0096] In a possible implementation, if the ratio is greater than the first preset threshold and the ratio is less than or equal to the second preset threshold, the determination unit 720 determines that the data search method of the target coding unit includes the motion information search and the pixel search.

[0097] In a possible implementation, if the ratio is less than or equal to the first preset threshold, and the pixel value variance is less than a third preset threshold, determining that the search method for the target coding unit includes the motion information search; or,

[0098] If the ratio is less than or equal to the first preset threshold, the pixel value variance is less than the third preset threshold, and the ratio is greater than the second preset threshold, it is determined that the data search method of the target coding unit only includes the motion information search.

[0099] In one possible implementation, if the ratio is less than or equal to the first preset threshold, the pixel value variance is less than the third preset threshold, and the ratio is less than or equal to the second preset threshold, the determination unit 720 determines that the data search method of the target coding unit includes the motion information search and the pixel search.

[0100] In one possible implementation, if the ratio is less than or equal to the first preset threshold, the ratio is greater than the second preset threshold, and the pixel value variance is greater than or equal to the third preset threshold, the determination unit 720 determines that the data search method of the target coding unit only includes the integer pixel search.

[0101] In a possible implementation, the processing unit 730 performs data search by using the determined data search mode to obtain the search result of the target coding unit, including:

[0102] If the data search method includes the motion information search, obtaining a search start point of a coding unit that has completed the search and the target coding unit, wherein the coding unit that has completed the search includes a parent coding unit and a child coding unit of the target coding unit that has completed the integer pixel search;

[0103] Obtaining motion information corresponding to the search starting points of the parent coding unit, the child coding unit, and the target coding unit;

[0104] The rate-distortion costs in the corresponding motion information are compared, and the motion information including the minimum rate-distortion cost is used as the integer-pixel search result of the target coding unit.

[0105] See also Figure 8 , Figure 8 is a schematic diagram of a computer device provided in an embodiment of the present application, the computer device is used to execute Figure 3 to Figure 6 The steps performed by the computer device in the corresponding method embodiment, the server includes: one or more processors 810; one or more input devices 820, one or more output devices 830 and a memory 840. The processor 810, the input device 820, the output device 830 and the memory 840 are connected via a bus 850. The memory 840 is used to store a computer program, the computer program includes program instructions, and the processor 810 is used to execute the program instructions stored in the memory 840 to perform the following operations:

[0106] Acquire characteristic information of a target coding unit in the image to be encoded, wherein the characteristic information includes at least pixel value variance and size information of the target coding unit;

[0107] Determining a data search mode for the target coding unit according to the characteristic information of the target coding unit, the data search mode comprising at least one of an integer pixel search, a sub-pixel search, and a motion information search, the motion information comprising motion information in a coding unit that has completed the search and motion information in a search start point of the target coding unit;

[0108] Performing data search using the determined data search method to obtain a search result of the target coding unit, wherein the search result includes one or both of an integer pixel search result and a sub-pixel search result;

[0109] Motion estimation information of the target coding unit is determined according to the search result.

[0110] In a possible implementation manner, the size information includes the length and width of the target coding unit, and the processor 810 determines the data search method of the target coding unit according to the characteristic information of the target coding unit, including:

[0111] Acquire a ratio between a length and a width of the target coding unit according to the size information;

[0112] If the ratio is greater than a first preset threshold, determining that the search method of the target coding unit includes the motion information search; or,

[0113] If the ratio is greater than a first preset threshold and the ratio is greater than a second preset threshold, and the first preset threshold is different from the second preset threshold, it is determined that the search method of the target coding unit only includes the motion information search.

[0114] In a possible implementation, if the ratio is greater than the first preset threshold and the ratio is less than or equal to the second preset threshold, the processor 810 determines that the data search method of the target coding unit includes the motion information search and the pixel search.

[0115] In a possible implementation, if the ratio is less than or equal to the first preset threshold, and the pixel value variance is less than a third preset threshold, determining that the search method for the target coding unit includes the motion information search; or,

[0116] If the ratio is less than or equal to the first preset threshold, the pixel value variance is less than the third preset threshold, and the ratio is greater than the second preset threshold, it is determined that the data search method of the target coding unit only includes the motion information search.

[0117] In one possible implementation, if the ratio is less than or equal to the first preset threshold, the pixel value variance is less than the third preset threshold, and the ratio is less than or equal to the second preset threshold, the processor 810 determines that the data search method of the target coding unit includes the motion information search and the pixel search.

[0118] In one possible implementation, if the ratio is less than or equal to the first preset threshold, the ratio is greater than the second preset threshold, and the pixel value variance is greater than or equal to the third preset threshold, the processor 810 determines that the data search method of the target coding unit only includes the integer pixel search.

[0119] In a possible implementation manner, the processor 810 performs data search by using the determined data search manner to obtain a search result of the target coding unit, including:

[0120] If the data search method includes the motion information search, obtaining a search start point of a coding unit that has completed the search and the target coding unit, wherein the coding unit that has completed the search includes a parent coding unit and a child coding unit of the target coding unit that has completed the integer pixel search;

[0121] Obtaining motion information corresponding to the search starting points of the parent coding unit, the child coding unit, and the target coding unit;

[0122] The rate-distortion costs in the corresponding motion information are compared, and the motion information including the minimum rate-distortion cost is used as the integer-pixel search result of the target coding unit.

[0123] It should be understood that the computer device described in the embodiments of the present invention can execute the above Figure 3 to Figure 6 The description of the data processing method in the corresponding embodiment can also be performed as described above. Figure 7 The description of the data processing device in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0124] In addition, it should be pointed out here that: an embodiment of the present invention further provides a computer storage medium, and the computer storage medium stores a computer program executed by the data processing device mentioned above, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the above Figure 3 to Figure 6 The method in the corresponding embodiment, therefore, will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in the present invention, please refer to the description of the method embodiment of the present invention. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or, executed on multiple computer devices distributed in multiple locations and interconnected by a communication network. Multiple computer devices distributed in multiple locations and interconnected by a communication network can constitute a blockchain system.

[0125] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device can perform the above Figures 3 to 8 The method in the corresponding embodiment will therefore not be described in detail here.

[0126] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the above-mentioned program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The above-mentioned storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0127] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire characteristic information of a target coding unit in the image to be encoded, wherein the characteristic information at least includes pixel value variance and size information of the target coding unit; the size information includes the length and width of the target coding unit; Acquire a ratio between a length and a width of the target coding unit according to the size information; If the ratio is greater than a first preset threshold, and the ratio is greater than a second preset threshold, and the first preset threshold is different from the second preset threshold, it is determined that the data search method of the target coding unit only includes motion information search; or, if the ratio is less than or equal to the first preset threshold, the pixel value variance is less than a third preset threshold, and the ratio is greater than the second preset threshold, it is determined that the data search method of the target coding unit only includes motion information search; wherein the motion information includes motion information in the coding unit that has completed the search and motion information in the search starting point of the target coding unit; or, If the ratio is greater than the first preset threshold and the ratio is less than or equal to the second preset threshold, determining that the data search method of the target coding unit includes the motion information search and the pixel search; Performing data search using the determined data search method to obtain a search result of the target coding unit, wherein the search result includes one or both of an integer pixel search result and a sub-pixel search result; Motion estimation information of the target coding unit is determined according to the search result.

2. The method according to claim 1, characterized in that The method further comprises: If the ratio is greater than a first preset threshold, determining that the search method of the target coding unit includes the motion information search; or, If the ratio is less than or equal to the first preset threshold, and the pixel value variance is less than a third preset threshold, it is determined that the search method for the target coding unit includes the motion information search.

3. The method according to claim 1, characterized in that The method further comprises: If the ratio is less than or equal to the first preset threshold, the pixel value variance is less than the third preset threshold, and the ratio is less than or equal to the second preset threshold, it is determined that the data search method of the target coding unit includes the motion information search and the pixel search.

4. The method according to claim 1, characterized in that: The method further comprises: If the ratio is less than or equal to the first preset threshold, the ratio is greater than the second preset threshold, and the pixel value variance is greater than or equal to the third preset threshold, it is determined that the data search method of the target coding unit only includes the integer pixel search.

5. The method according to any one of claims 1 to 4, characterized in that: The step of performing data search by using the determined data search method to obtain search results of the target coding unit includes: If the data search method includes the motion information search, obtaining a search start point of a coding unit that has completed the search and the target coding unit, wherein the coding unit that has completed the search includes a parent coding unit and a child coding unit of the target coding unit that has completed the integer pixel search; Obtaining motion information corresponding to the search starting points of the parent coding unit, the child coding unit, and the target coding unit; The rate-distortion costs in the corresponding motion information are compared, and the motion information including the minimum rate-distortion cost is used as the integer-pixel search result of the target coding unit.

6. A data processing device, characterized in that: include: An acquisition unit, configured to acquire feature information of a target coding unit in the image to be encoded, wherein the feature information at least includes pixel value variance and size information of the target coding unit; The size information includes the length and width of the target coding unit; a determining unit, configured to obtain a ratio between a length and a width of the target coding unit according to the size information; The determining unit is further configured to determine that the data search method of the target coding unit only includes motion information search if the ratio is greater than a first preset threshold value, and the ratio is greater than a second preset threshold value, and the first preset threshold value is different from the second preset threshold value; or, if the ratio is less than or equal to the first preset threshold value, the pixel value variance is less than a third preset threshold value, and the ratio is greater than the second preset threshold value, determine that the data search method of the target coding unit only includes motion information search; wherein the motion information includes motion information in the coding unit that has completed the search and motion information being searched in the search starting point of the target coding unit; or, If the ratio is greater than the first preset threshold and the ratio is less than or equal to the second preset threshold, determining that the data search method of the target coding unit includes the motion information search and the pixel search; A processing unit, configured to perform data search using the determined data search method to obtain a search result of the target coding unit, wherein the search result includes one or both of an integer pixel search result and a sub-pixel search result; The determination unit is further used to determine the motion estimation information of the target coding unit according to the pixel search result.

7. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a set of program codes, and the processor calls the program codes stored in the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. When the computer instructions are executed by a processor of a computer device, they are used to execute the method according to any one of claims 1 to 5.

Images