Data encoding method, device, and computer program

The data encoding method optimizes encoding speed by adaptively determining when to execute motion detection algorithms based on average vector information ratios, addressing inefficiencies in existing methods by skipping unnecessary operations during video data encoding.

JP7578378B2Active Publication Date: 2024-11-06TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023525100
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-11-13
Filing Date
2021-11-11
Publication Date
2024-11-06
Estimated Expiration
2041-11-11

AI Technical Summary

Technical Problem

Existing video data encoding methods are time-consuming due to the need to execute motion detection algorithms for each image frame, even when there is no significant movement, leading to inefficient encoding speeds.

Method used

A data encoding method that determines overall and component average vector information for image frames, allowing adaptive decision-making on whether to execute motion detection algorithms based on the ratio of component average vector information to overall average vector information, thereby optimizing encoding speed by skipping unnecessary algorithmic operations.

Benefits of technology

Improves encoding speed by selectively executing motion detection algorithms only when significant movement is detected, reducing the overall time required to encode video data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007578378000001
    Figure 0007578378000001
  • Figure 0007578378000002
    Figure 0007578378000002
  • Figure 0007578378000003
    Figure 0007578378000003
Patent Text Reader

Abstract

The present application discloses a data encoding method, an apparatus, and a computer-readable storage medium, the method relating to data transmission-related technology in cloud technology, and including the steps of: acquiring at least two coding units included in a target image frame; acquiring motion vector information of each of the at least two coding units; determining overall average vector information for the target image frame based on the motion vector information of each coding unit; determining component average vector information in the reference direction of the target image frame based on component vector information in the reference direction of the motion vector information of each coding unit; and determining a frame execution policy for a motion detection operation associated with the reference direction of the target image frame based on the overall average vector information and the component average vector information, where the frame execution policy is a frame start execution policy or a frame end execution policy. By employing the present application, the encoding speed for the target image frame can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present application relates to the technical field of data processing, and in particular to a data encoding method, apparatus, and computer-readable storage medium.

[0002] This application claims priority to a Chinese patent application filed with the China Patent Office on November 13, 2020, bearing application number 202011264969.1 and entitled "Data Encoding Method, Apparatus, and Computer-Readable Storage Medium," the entire contents of which are incorporated herein by reference. [Background technology]

[0003] Many scenes in daily life may involve the scene of compressing video data. For example, when user A transmits a piece of video data to user B, the video data may be compressed before being transmitted. Here, when compressing the video data, the motion vector information of each image frame in the video data is used. The motion vector information can characterize the relative change between adjacent image frames of the video data. Summary of the Invention [Problem to be solved by the invention]

[0004] The present application provides a method for encoding data, Obtaining at least two coding units included in a target image frame, and obtaining motion vector information of each coding unit of the at least two coding units; determining overall average motion vector information for a target image frame based on the motion vector information of each coding unit; determining component average vector information in a reference direction of a target image frame according to component vector information in a reference direction of the motion vector information of each coding unit; and determining a frame execution policy for a motion detection operation in a reference direction of the target image frame based on the overall average vector information and the component average vector information, where the frame execution policy is a frame start execution policy or a frame end execution policy.

[0005] According to one aspect of the present application, there is provided a data encoding device, comprising: A motion information acquisition module is used for acquiring at least two coding units included in a target image frame, and acquiring motion vector information of each coding unit among the at least two coding units; a global information determining module, which is used for determining global average vector information for a target image frame according to the motion vector information of each coding unit; A component information determining module is used for determining component average vector information in a reference direction of a target image frame according to component vector information in a reference direction of the motion vector information of each coding unit; and a policy determination module used for determining a frame execution policy for a motion detection operation associated with a reference direction of a target image frame based on the overall average vector information and the component average vector information, where the frame execution policy is a frame start execution policy or a frame end execution policy.

[0006] An aspect of the present application provides a computing device, comprising a memory and a processor, the memory storing a computer program, the computer program, when executed by the processor, causing the processor to perform a method in an aspect of the present application.

[0007] One aspect of the present application provides a computer-readable storage medium having a computer program stored therein, the computer program including program instructions that, when executed by a processor, cause the processor to perform a method in the one aspect.

[0008] According to one aspect of the present application, there is provided a computer program product or computer program, the computer program product or computer program including computer instructions stored in a computer readable storage medium, a processor of a computing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computing device to perform the method provided in various selectable forms, such as the one aspect.

[0009] In order to more clearly describe the technical solutions in the present application or related art, the following briefly introduces drawings that need to be used in the description of the embodiments or related art. It is obvious that the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other drawings based on these drawings without creative labor. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a structural schematic diagram of a network architecture provided by an embodiment of the present application; [Diagram 2] FIG. 1 is a schematic diagram of a data encoding scene provided by the present application; [Diagram 3] 1 is a flowchart of a data encoding method provided by the present application. [Figure 4] FIG. 2 is a schematic diagram of a scene from which motion vector information provided by the present application is obtained; [Diagram 5] FIG. 2 is a schematic diagram of a scene for acquiring component average vector information provided by the present application; [Figure 6] FIG. 2 is a schematic diagram of a scene for acquiring component average vector information provided by the present application; [Figure 7] FIG. 2 is a schematic diagram of a scene for acquiring vector information provided by the present application; [Figure 8] FIG. 2 is a schematic diagram of a scene for obtaining a unit component vector value provided by the present application; [Figure 9] FIG. 1 is a structural schematic diagram of a data encoding device provided by the present application; [Figure 10] FIG. 1 is a structural schematic diagram of a computer device provided by the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The technical solutions in this application are described below clearly and completely in conjunction with the drawings in this application. It is obvious that the described embodiments are only some of the embodiments in this application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of this application.

[0012] This application relates to cloud technology, which refers to a type of hosting technology that integrates a set of resources, such as hardware, software, and networks, within a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0013] Cloud technology is a collective term for network technology, information technology, integration technology, management platform technology, and application technology, which are applied based on the cloud computing business model, forming a resource pool that can be used as needed, flexible and convenient. Cloud computing technology has become an important support. The background service of the technology network system requires a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the advanced development and application of the Internet industry, in the future, each item may have its own unique identification mark, which needs to be transmitted to the background system for logical processing. Data of different degrees and levels will be processed separately, and various industrial data all require powerful system support, which can only be realized by cloud computing.

[0014] In this application, the cloud technology mainly refers to a technology for transmitting video data, specifically, a technology for using a cloud server to first encode and compress video data before transmitting it. See the description below.

[0015] In the related art, when compressing video data, for each image frame in the video data, Cross Search The algorithm is executed on the image frame. Cross Search By executing the algorithm, the motion direction contained in the motion vector information of the image frame can be detected. For each image frame, Cross Search Since the algorithms are usually relatively time consuming to execute, they are typically performed for each image frame in the video data. Cross Search When the algorithm is executed, it takes a long time to encode the image frames and an even longer time to encode the video data.

[0016] In the data encoding method provided by the present application, when there is no significant motion in a certain image frame in the horizontal or vertical direction, the method provided by the present application performs the following steps: Cross Search No algorithmic operations need to be performed, thus improving the coding speed for image frames and further improving the coding speed for video data.

[0017] Referring to FIG. 1, FIG. 1 is a structural schematic diagram of a network architecture provided by an embodiment of the present application. As shown in FIG. 1, the network architecture may include a server 200 and a terminal device cluster, and the terminal device cluster may include one or more terminal devices, and the number of terminal devices is not limited here. As shown in FIG. 1, the multiple terminal devices may specifically include terminal device 100a, terminal device 101a, terminal device 102a, ..., and terminal device 103a, and as shown in FIG. 1, terminal device 100a, terminal device 101a, terminal device 102a, ..., and terminal device 103a can all be network-connected with server 200, so that each terminal device can exchange data with server 200 through network connection.

[0018] The server 200 shown in Fig. 1 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device may be a smart terminal such as a smartphone, a tablet PC, a laptop computer, a desktop PC, or a smart TV. Hereinafter, a specific description of the embodiment of the present application will be given by taking the communication between the terminal device 100a and the server 200 as an example.

[0019] Also refer to FIG. 2, which is a schematic diagram of a data encoding scene provided by the present application. The image frame 100b may be an image frame obtained after framing the video data 108b, that is, the image frame 100b may be any one of the image frames included in the video data 108b. The video data 108b may be any one of the video data, and the video data 108b is obtained by the terminal device 100a. For example, the video data 108b may be conference video data recorded in the process of realizing a real-time online conference by the terminal device 100a. After obtaining the video data 108b, the terminal device 100a can transmit the video data 108b to the server 200 and request the server 200 to encode the video data 108b. Therefore, after obtaining the video data 108b, the server 108b can frame the video data 108b to obtain the image frame 100b.

[0020] Since the principle by which the server 200 encodes each image frame included in the video data 108b is the same, the process of how the server 200 encodes the image frame 100b will be taken as an example here.

[0021] As shown in Fig. 2, the server 200 can divide the image frame 100b into multiple (at least two) coding units. Here, the multiple coding units into which the image frame 100b is divided by the server 200 may include n coding units in the region 101b, and the value of n may be determined based on the actual application scene, and is not limited thereto. The n coding units may specifically include coding unit 1, coding unit 2, coding unit 3, ... and coding unit n.

[0022] Here, each of the n coding units has corresponding motion vector information, which can characterize the change (i.e., the motion situation) between the coding unit in the image frame 100b and the coding unit at the corresponding position in the image frame before the image frame 100b in the video data 108b, and the motion vector information can be used to code the image frame 100b.

[0023] Here, the server 200 may calculate and obtain overall average vector information 105b of the image frame 100b according to the motion vector information corresponding to each of the n coding units included in the image frame 100b, and the overall average vector information 105b characterizes an average value of the motion vector information corresponding to all the coding units included in the image frame 100b. For a specific process of calculating the overall average vector information 105b, please refer to the corresponding description in the embodiment corresponding to FIG. 3 below.

[0024] The server 200 may also calculate and obtain component average vector information 106b in the vertical direction (which may be understood as the y-axis direction) of the image frame 100b according to the motion vector information corresponding to each of the n coding units included in the image frame 100b, and the component average vector information 106b characterizes the average value in the vertical direction of the motion vector information corresponding to all the coding units included in the image frame 100b. For a specific process of calculating the component average vector information 106b, refer to the corresponding description in the embodiment corresponding to FIG. 3 below.

[0025] The server 200 may also calculate and obtain component average vector information 107b in the horizontal direction (which may be understood as the x-axis direction) of the image frame 100b according to the motion vector information corresponding to each of the n coding units included in the image frame 100b, and the component average vector information 107b characterizes the average value in the horizontal direction of the motion vector information corresponding to all the coding units included in the image frame 100b. For a specific process of calculating the component average vector information 107b, please refer to the corresponding description in the embodiment corresponding to FIG. 3 below.

[0026] Here, when the overall average vector information 105b, the vertical component average vector information 106b, and the horizontal component average vector information 107b include the situation described in the area 102b or the situation described in the area 103b, the server performs the following processing for the image frame 100b when encoding the image frame 100b: Cross Search In other circumstances, i.e., other than the circumstances described in area 102b and the circumstances described in area 103b, the server may run (i.e., execute) the algorithm (i.e., reach a conclusion in area 104b) for image frame 100b. Cross Search The algorithm can be terminated (i.e. not executed).

[0027] Here, Cross Search The algorithm is used to detect the motion direction of the motion vector information included in the image frame 100b during the process in which the server encodes the image frame 100b. Cross Search The coding rate for image frame 100b when the algorithm is executed is Cross SearchThe encoding speed for the image frame 100b is slower than that when the algorithm is not executed (i.e., terminated). Therefore, the overall average vector information 105b, the vertical component average vector information 106b, and the horizontal component average vector information 107b may be used to encode the image frame 100b under some circumstances (i.e., circumstances other than those described in the area 102b and the area 103b). Cross Search By deciding to terminate the algorithm, the encoding speed for image frame 100b can be increased.

[0028] Here, as the situation described in the above area 102b, when the ratio between the component average vector information 106b in the vertical direction of the image frame 100b and the overall average vector information 105b is greater than a certain ratio threshold, Cross Search The algorithm can be run: when the ratio between the component average vector information 106b and the global average vector information 105b is greater than a certain ratio threshold (which can be set by oneself), the image frame 100b is indicated as having significant motion in the vertical direction compared to the previous image frame.

[0029] Here, as the situation described in the above area 103b, when the ratio between the component average vector information 107b in the horizontal direction of the image frame 100b and the overall average vector information 105b is greater than a certain ratio threshold (which can be set by the user), Cross Search An algorithm can also be run such that when the ratio between the component average vector information 107b and the global average vector information 105b is greater than a certain ratio threshold, the image frame 100b is indicated as having significant motion in the horizontal direction compared to the previous image frame.

[0030] The server 200 receives the image frame 100b. Cross SearchAccording to the conclusion of judging whether the algorithm needs to be executed, the video data 108b can be encoded to obtain the encoded data of the video data 108b. The server 200 can transmit the encoded data to the video client that needs to play the video data 108b. After obtaining the encoded data, the video client can decode the encoded data to obtain the decoded data of the video data 108b, and the video client can play the decoded data to realize the playback of the video data 108b.

[0031] The method provided herein allows for the determination of the motion of an image frame (e.g., image frame 100b) when the image frame has significant horizontal or vertical motion, i.e., when significant horizontal or vertical motion is present, the image frame is subjected to the motion correction process. Cross Search The algorithm can be started otherwise for that image frame. Cross Search This allows for an increased coding speed for the image frame since the algorithm can be terminated.

[0032] Please refer to FIG. 3, which is a flowchart of a data encoding method provided in the present application, and as shown in FIG. 3, the method may include the following steps:

[0033] Step S101: obtain at least two coding units included in a target image frame, and obtain motion vector information of each coding unit of the at least two coding units.

[0034] Specifically, the execution subject in the embodiment of the present application may be any one computer device, or may be a device cluster composed of multiple computer devices. The computer device may be a server or a terminal device, but is not limited thereto. Here, the execution subject in the present application is taken as an example to be a server. Please refer to the description in the following content.

[0035] The server can obtain video data, which may be any one of video data, and the video data may be transmitted to the server by the terminal device. The terminal device can request the server to encode the video data by transmitting the video data to the server. For example, the video data may be video data captured in real time by a camera, such as online conference video data, and the video data may also be video data shared through a screen.

[0036] When encoding video data, the server can frame the video data, and obtain a plurality (at least two) image frames included in the video data. Therefore, the target image frame can be any one of a plurality of image frames obtained by framing the video data. The server encoding the video data means that each image frame of the video data needs to be encoded, and here, the process of the server encoding the target image frame in the video data is taken as an example. It can be understood that the principle of the server encoding each image frame in the video data is the same. Please refer to the description in the following content.

[0037] The server can divide the target image frame into multiple coding units, where each coding unit represents an image block in the target image frame. By dividing the target image frame into multiple coding units, block-based coding can be realized for the target image frame.

[0038] Optionally, the manner in which the server divides the target image frame into multiple coding units may be that the server can first divide the target image frame into multiple coding regions of equal size, and then the server can further divide each coding region to obtain multiple coding units respectively included in each coding region. All the coding units included in all the coding regions are the multiple coding units included in the target image frame.

[0039] The server may also obtain motion vector information (which may be abbreviated as motion vector and written as mv) of each coding unit of the target image frame, where the motion vector information of each coding unit of the target image frame characterizes the relative motion between the target image frame and a reference image frame. The reference image frame may be an image frame adjacent to the target image frame and preceding the target image frame in the video data, for example, the reference image frame may be an image frame preceding the target image frame in the video data. Thus, as can be understood, the motion vector information of each coding unit of the target image frame represents the change of the target image frame relative to its preceding image frame (i.e., the reference image frame), i.e., the motion vector information of each coding unit of the target image frame represents by which motion the reference image frame changes to the target image frame.

[0040] Therefore, the manner of obtaining the motion vector information of each coding unit of the target image frame is as follows: When encoding the reference image frame, the server may also divide the reference image frame into a plurality of image blocks, and the plurality of image blocks may be understood as a plurality of coding units of the reference image frame. The server may also obtain, among the plurality of image blocks into which the reference image frame is divided, image blocks having similarity with each coding unit of the target image frame as predicted image blocks corresponding to each coding unit.

[0041] Here, the reference image frame and the target image frame may have the same size, and the positions of each image block of the reference image frame and each coding unit of the target image frame may correspond one-to-one, that is, the predicted image block in the reference image frame corresponding to each coding unit of the target image frame may be the same image block as the position corresponding to each coding unit of the target image frame in the reference image frame. Image blocks and coding units at the same positions in the reference image frame and the target image frame can be considered to be similar.

[0042] Therefore, the server can detect the relative displacement between each coding unit and its corresponding predicted image block, which characterizes the motion made from the predicted image block to its corresponding coding unit. Furthermore, the server can obtain the motion vector information of each coding unit according to the relative displacement between each coding unit and its corresponding predicted image block and the target motion vector accuracy.

[0043] The target motion vector precision can be set according to the actual application scene, and represents the precision of the motion vector information of each coding unit. For example, the target motion vector precision can be 1 / 16 pixel precision, 1 / 4 pixel precision, 1 / 2 pixel precision, 1 pixel precision, 2 pixel precision, 4 pixel precision, or 8 pixel precision, etc.

[0044] In practice, the server will perform pre-analysis on the video data before encoding the video data. The main purpose of the pre-analysis is to analyze what kind of encoding should be adopted for the video data to minimize its rate-distortion cost. The rate-distortion cost simultaneously trades off the distortion degree and bit rate of encoding the video data. In the process of performing pre-analysis on the video data, the server can obtain the motion vector information of each encoding unit of each target image frame.

[0045] Referring to Fig. 4, Fig. 4 is a schematic diagram of a scene for obtaining motion vector information provided by the present application. As shown in Fig. 4, the server can frame the video data 100c to obtain a plurality of image frames in an area 101c. In other words, the video data 100c may include an image frame 102c, an image frame 103c, an image frame 104c, and an image frame 105c.

[0046] The server can select any one of the video frames among the image frame 102c, the image frame 103c, the image frame 104c, and the image frame 105c as the target video frame, for example, the image frame 104c can be selected as the target image frame here. Therefore, the reference image frame can be the image frame 103c before the target image frame 104c. The server can obtain the motion vector information 105c of each encoding unit of the target image frame based on the relative motion between each encoding unit of the target image frame and the image block at the corresponding position in the reference image frame.

[0047] Step S102: Determine global average vector information for the target image frame based on the motion vector information of each coding unit.

[0048] Specifically, the server can obtain horizontal component vector information and vertical component vector information respectively included in the motion vector information of each encoding unit of the target image frame. The horizontal component vector information included in the motion vector information of one encoding unit is a component in the horizontal direction (which can be understood as the x direction of the xy plane coordinate axis) of the motion vector information of the encoding unit. Similarly, the vertical component vector information included in the motion vector information of one encoding unit is a component in the vertical direction (which can be understood as the y direction of the xy plane coordinate axis) of the motion vector information of the encoding unit. As can be understood, the motion vector information of an encoding unit is a vector, and the horizontal component vector information and the vertical component vector information corresponding to the motion vector information of one encoding unit are both vectors.

[0049] Since the motion vector information of the coding unit is a vector, the motion vector information of the coding unit has a direction and a magnitude, and the magnitude of the motion vector information is the length of the motion vector information (i.e., the length of the vector), and the length may be called the modulus length of the motion vector information. The server can obtain the modulus length of the motion vector information of each coding unit according to the horizontal component vector information and the vertical component vector information respectively corresponding to each coding unit, and the modulus length of the motion vector information of each coding unit of the target image frame can be called the unit motion vector value respectively corresponding to each coding unit.

[0050] For example, the modulus length of the horizontal component vector information of the motion vector information of one coding unit is x1, and the modulus length of the vertical component vector information of the motion vector information of the coding unit is x2. Then, the unit motion vector value corresponding to the motion vector information of the coding unit is equal to the modulus length of the motion vector information of the coding unit, and is also equal to the value obtained by calculating the square root of the value obtained by adding the square of the modulus length x1 of the horizontal component vector information to the square of the modulus length x2 of the vertical component vector information.

[0051] Through the above process, the server can calculate and obtain unit motion vector values ​​corresponding to each coding unit, and the server can calculate the average value of unit motion vector values ​​corresponding to all coding units of the target image frame as the overall average vector information. For example, the server can calculate the sum of unit motion vector values ​​corresponding to all coding units, and divide the sum by the total number of all coding units to obtain the overall average vector information.

[0052] For example, the target image frame includes coding unit 1, coding unit 2, and coding unit 3, where the unit motion vector value corresponding to coding unit 1 is 4, the unit motion vector value corresponding to coding unit 2 is 5, and the unit motion vector value corresponding to coding unit 3 is 6, so the overall average vector information of the target image frame is equal to (4+5+6) / 3, which is equal to 5.

[0053] Step S103: Determine component average vector information in the reference direction of the target image frame according to component vector information in the reference direction of the motion vector information of each coding unit.

[0054] Specifically, the reference direction may refer to a horizontal reference direction (which may be abbreviated as horizontal direction) or a vertical reference direction (which may be abbreviated as vertical direction). Therefore, the component vector information of the motion vector information may be component information in any one of the horizontal reference direction and the vertical reference direction.

[0055] When the reference direction is a horizontal reference direction, the process of the server obtaining component average vector information in the reference direction of the target image frame includes: The server can obtain a horizontal component value corresponding to each coding unit according to the horizontal component vector information in the horizontal reference direction of the motion vector information of each coding unit (all the following coding units are coding units of the target image frame), and the horizontal component value can be the magnitude of the horizontal component vector information. In other words, the modulus length of the horizontal component vector information in the horizontal reference direction of the motion vector information of one coding unit can be called the horizontal component value corresponding to the coding unit.

[0056] The server can determine component average vector information based on the horizontal component values ​​respectively corresponding to each encoding unit and the unit number of at least two encoding units, and the server can take the average value of the horizontal component values ​​respectively corresponding to all encoding units as the component average vector information of the target image frame. That is, the server can calculate the sum of the horizontal component values ​​respectively corresponding to all encoding units, and use the sum to divide by the total number of all encoding units (which may be called the unit number) to obtain the component average vector information of the target image frame.

[0057] For example, the target image frame includes coding unit 1, coding unit 2, and coding unit 3. Here, the horizontal component value corresponding to coding unit 1 is 6, the horizontal component value corresponding to coding unit 2 is 7, and the horizontal component value corresponding to coding unit 3 is 8, then the component average vector information of the target image frame may be equal to (6+7+8) / 3, i.e., equal to 7.

[0058] Please refer to Fig. 5, which is a schematic diagram of a scene for obtaining component average vector information provided in the present application. As shown in Fig. 5, a target image frame may include coding unit 1, coding unit 2, ..., and coding unit n. The value of n is determined based on an actual application scene.

[0059] Here, the horizontal component vector information in the horizontal reference direction of the motion vector information 100d of the coding unit 1 is horizontal component vector information 101d. The horizontal component vector information in the horizontal reference direction of the motion vector information 102d of the coding unit 2 is horizontal component vector information 103d. The horizontal component vector information in the horizontal reference direction of the motion vector information 104d of the coding unit n is horizontal component vector information 105d.

[0060] Thus, the horizontal component value corresponding to coding unit 1 is the modulus length of horizontal component vector information 101d, the horizontal component value corresponding to coding unit 2 is the modulus length of horizontal component vector information 103d, and the horizontal component value corresponding to coding unit n is the modulus length of horizontal component vector information 105d. The modulus length refers to the length of the line segment remaining after removing the directional elements from the vector.

[0061] Therefore, the server can set the average value of the horizontal component value corresponding to coding unit 1, the horizontal component value corresponding to coding unit 2, ..., and the horizontal component value corresponding to coding unit n as component average vector information 106d of the target image frame.

[0062] Similarly, when the reference direction is a vertical reference direction, the process of the server obtaining the component average vector information in the reference direction of the target image frame is as follows: The server may obtain a vertical component value corresponding to each coding unit according to the vertical component vector information in the vertical reference direction of the motion vector information of each coding unit, and the vertical component value may be the magnitude of the vertical component vector information. In other words, the modulus length of the vertical component vector information in the vertical reference direction of the motion vector information of one coding unit can be called the vertical component value corresponding to the coding unit.

[0063] The server can determine component average vector information based on the vertical component values ​​respectively corresponding to each encoding unit and the unit number of at least two encoding units, and the server can take the average value of the vertical component values ​​respectively corresponding to all encoding units as the component average vector information of the target image frame. That is, the server can calculate the sum of the vertical component values ​​respectively corresponding to all encoding units, and use the sum to divide by the total number of all encoding units (which may be called the unit number) to obtain the component average vector information of the target image frame.

[0064] For example, the target image frame includes coding unit 1, coding unit 2, and coding unit 3, where the vertical component value corresponding to coding unit 1 is 10, the vertical component value corresponding to coding unit 2 is 20, and the vertical component value corresponding to coding unit 3 is 30, and the component average vector information of the target image frame may be equal to (10+20+30) / 3, i.e., equal to 20.

[0065] Please refer to Fig. 6, which is a schematic diagram of a scene for obtaining component average vector information provided in the present application. As shown in Fig. 6, a target image frame may include coding unit 1, coding unit 2, ..., and coding unit n. The value of n is determined based on an actual application scene.

[0066] Here, the vertical component vector information in the vertical reference direction of the motion vector information 100e of the coding unit 1 is vertical component vector information 101e. The vertical component vector information in the vertical reference direction of the motion vector information 102e of the coding unit 2 is vertical component vector information 103e. The vertical component vector information in the vertical reference direction of the motion vector information 104e of the coding unit n is vertical component vector information 105e.

[0067] Therefore, the vertical component value corresponding to coding unit 1 is the modulus length of vertical component vector information 101e, the vertical component value corresponding to coding unit 2 is the modulus length of vertical component vector information 103e, and the vertical component value corresponding to coding unit n is the modulus length of vertical component vector information 105e.

[0068] Therefore, the server can set the average value of the vertical component value corresponding to coding unit 1, the vertical component value corresponding to coding unit 2, ..., and the vertical component value corresponding to coding unit n as component average vector information 106e of the target image frame.

[0069] Refer to Fig. 7, Fig. 7 is a schematic diagram of a scene of obtaining vector information provided in the present application. As shown in Fig. 7, the encoding units of the target image frame may include encoding unit 1, encoding unit 2, encoding unit 3, and encoding unit 4. The motion vector information of encoding unit 1 may be motion vector information 106f, the motion vector information of encoding unit 2 may be motion vector information 107f, the motion vector information of encoding unit 3 may be motion vector information 104f, and the motion vector information of encoding unit 4 may be motion vector information 105f.

[0070] The server can set the average value of the modulus lengths of the motion vector information 104f, the motion vector information 105f, the motion vector information 106f, and the motion vector information 107f as the overall average vector information 108f of the target image frame. The server can set the average value of the modulus lengths of the components in the horizontal direction or the vertical direction of the motion vector information 104f, the motion vector information 105f, the motion vector information 106f, and the motion vector information 107f as the component average vector information 109f of the target image frame.

[0071] Step S104: Determine a frame execution policy for the motion detection operation in the reference direction of the target image frame according to the overall average vector information and the component average vector information, where the frame execution policy is a frame start execution policy or a frame end execution policy.

[0072] Specifically, the reference directions may be horizontal and vertical, so that the motion detection operations associated with the reference directions are: Cross Search It may be an operation of executing an algorithm. Cross Search The algorithm is mainly used to detect the horizontal and / or vertical motion direction of the motion vector information of an image frame or coding unit.

[0073] Here, performing a motion detection operation associated with a reference direction on a target image frame includes performing a motion detection operation associated with a reference direction on a target image frame. Cross Search It represents that the operation of the algorithm is executed, i.e., the frame execution policy for the motion detection operation of the target image frame is the frame start execution policy. Not executing the motion detection operation associated with the reference direction for the target image frame means that the frame start execution policy for the target image frame is the frame start execution policy. Cross Search It indicates that no operation of the algorithm is to be performed, ie, the frame execution policy for the motion detection operation of the target image frame is the frame end execution policy.

[0074] By performing a motion detection operation associated with a reference direction on the target image frame, the motion direction of the motion vector information included in the target image frame can be detected and obtained, and the target image frame can be further better coded by detecting and obtaining the motion direction of the motion vector information included in the target image frame.

[0075] The present application can determine whether there is significant horizontal or vertical motion in the target image frame based on the overall average vector information and the component average vector information. When there is significant horizontal or vertical motion in the target image frame, a motion detection operation associated with a reference direction is performed on the motion vector information included in the target image frame, i.e. Cross Search otherwise, no motion detection operation associated with the reference direction is performed on the target image frame, i.e. Cross Search No operation may be performed, i.e., when there is no significant motion in the horizontal or vertical direction in the target image frame, no motion detection operation associated with the reference direction may be performed on the motion vector information contained in the target image frame.

[0076] Now, as it needs to be understood, we are going to perform a motion detection operation where the target image frame is associated with a reference direction (i.e. Cross Search Regardless of whether the algorithm is performed or not, the conventional motion detection operation is performed on the target image frame. Cross Search The algorithm is mainly used to detect the motion direction in the horizontal or vertical direction of the target image frame, but the ordinary motion detection may also refer to performing motion direction detection for each pixel point in the target image frame.

[0077] As you can see, for the target image frame Cross Search The performing of the algorithmic operation is used to assist in performing a conventional motion detection operation on the target image frame when the target image frame has significant motion in the horizontal or vertical direction. Cross SearchAn algorithm can be implemented to make the detection of the motion vector information of the target image frame for the motion direction more accurate and faster. When the target image frame does not have significant motion in the horizontal or vertical direction, Cross Search It is not very useful to perform the algorithmic operations on a target image frame when the target image frame does not have significant motion in the horizontal or vertical directions. Cross Search It is possible to not perform any algorithmic operations (ie, not perform any motion detection operations associated with the reference direction).

[0078] For image frames Cross Search Since the algorithmic operations are relatively time-consuming, when encoding video data, if one of the following is done for each image frame of the video data, Cross Search If the algorithm is performed unconditionally, it will take a long time to encode the video data for the image frame, and it will take a long time to encode the video data. Therefore, the method provided in the present application provides a method for encoding the video data for an image frame (which may be the target image frame) when there is no significant motion in the horizontal or vertical direction for the image frame. Cross Search No algorithmic operations need to be performed, thus improving the coding speed for image frames and further improving the coding speed for video data.

[0079] Here, the component average vector information may be component average vector information in any one of the horizontal or vertical directions, so as can be understood, it is possible to determine whether or not the target image frame has significant movement in the horizontal or vertical direction based solely on the component average vector information in the horizontal or vertical direction.

[0080] The reason is that if the component average vector information is in the horizontal direction, when the ratio between the component average vector information in the horizontal direction of the target image frame and the global average vector information is equal to or greater than a certain threshold (which can be set by the user and may be called a first ratio threshold), it indicates that the component average vector information in the horizontal direction of the target image frame occupies most of the information in the global average vector information, and the target image frame also indicates that there is a significant movement in the horizontal direction.When the ratio between the component average vector information in the horizontal direction of the target image frame and the global average vector information is equal to or less than a certain threshold (which can be set by the user and may be called a second ratio threshold), it indicates that the component average vector information in the horizontal direction of the target image frame occupies a very small part of the information in the global average vector information, and the target image frame also indicates that there is no significant movement in the horizontal direction, but there is a significant movement in the vertical direction.

[0081] Similarly, if the component average vector information is in the vertical direction, when the ratio between the component average vector information in the vertical direction of the target image frame and the overall average vector information is equal to or greater than a certain threshold (which may be the above-mentioned first ratio threshold), it indicates that the component average vector information in the vertical direction of the target image frame occupies a large part of the information in the overall average vector information, and the target image frame also indicates that there is a significant movement in the vertical direction.When the ratio between the component average vector information in the vertical direction of the target image frame and the overall average vector information is equal to or less than a certain threshold (which may be the above-mentioned second ratio threshold), it indicates that the component average vector information in the vertical direction of the target image frame occupies a very small part of the information in the overall average vector information, and the target image frame also indicates that there is no significant movement in the vertical direction, but there is a significant movement in the horizontal direction.

[0082] The first ratio threshold is greater than the second ratio threshold.

[0083] In other words, if the component average vector information in the horizontal direction of the target image frame occupies a large part of the overall average vector information, the component average vector information in the vertical direction of the target image frame occupies a small part of the overall average vector information. Conversely, if the component average vector information in the vertical direction of the target image frame occupies a large part of the overall average vector information, the component average vector information in the horizontal direction of the target image frame occupies a small part of the overall average vector information. In other words, there is a negative correlation between the component average vector information in the horizontal direction and the component average vector information in the vertical direction, and when one is large, the other is small.

[0084] Therefore, the above process may be summarized as follows: The ratio between the component average vector information and the overall average vector information may be referred to as a frame component ratio value, and the numerical range of the frame component ratio value is from 0 to 1. When the frame component ratio value is equal to or greater than the first ratio threshold, or when the frame component ratio value is equal to or less than the second ratio threshold, the target image frame is represented as having significant motion in the horizontal or vertical direction, and therefore, it may be determined that the frame execution policy for the motion detection operation associated with the reference direction of the target image frame is a frame start execution policy.

[0085] When the frame component ratio value is smaller than the first ratio threshold and larger than the second ratio threshold, it is indicated that the target image frame does not have significant motion in the horizontal direction or the vertical direction, so that the motion detection operation associated with the reference direction of the target image frame (i.e. Cross Search It may be determined that the frame execution policy for a particular algorithm (performing an operation of the algorithm) is the frame end execution policy.

[0086] When the server determines that the frame execution policy for the motion estimation operation associated with the reference direction of the target image frame is a frame end execution policy, the server performs a frame end execution policy for all coding units in the target image frame. Cross Search The operations of the algorithm may not be performed.

[0087] In addition, when the server determines that the frame execution policy for the motion estimation operation associated with the reference direction of the target image frame is a frame start execution policy, the server further determines which of the multiple coding units of the target image frame is a frame start execution policy. Cross Search Which coding units need to perform the algorithm's operations? Cross Search It may also be specifically determined whether or not the algorithmic operations need to be performed, thereby further improving the encoding speed for the target image frame, see the description below.

[0088] The server processes each encoding unit Cross Search The principle of determining whether an algorithmic operation needs to be performed is whether the target image frame Cross Search The principle of determining whether an algorithmic operation needs to be performed is the same, but one is at the image frame level and the other is at the coding unit level.

[0089] Specifically, the server can obtain the unit motion vector value and the unit component vector value of each coding unit according to the motion vector information of each coding unit. The unit motion component value of a coding unit may refer to the magnitude, i.e., modulus length, of the motion vector information of the coding unit. The unit component vector value of a coding unit may refer to the modulus length of the horizontal component vector information in the horizontal direction of the motion vector information of the coding unit, or may refer to the modulus length of the vertical component vector information in the vertical direction of the motion vector information of the coding unit.

[0090] Refer to Fig. 8, which is a schematic diagram of a scene for obtaining a unit component vector value provided by the present application. As shown in Fig. 8, the motion vector information 100g may be the motion vector information of one coding unit of a target image frame. The vertical component of the motion vector information 100g is vertical component vector information 101g, and the horizontal component of the motion vector information 100g is horizontal component vector information 102g.

[0091] As shown in area 103g, the server can set the modulus length 104g of the vertical component vector information 101g or the modulus length 105g of the horizontal component vector information 102g as the unit component vector value of the coding unit to which the motion vector information 100g belongs.

[0092] Now, if for a given coding unit Cross Search If an algorithmic operation needs to be performed, the unit execution policy for the motion estimation operation associated with the reference direction of the coding unit can be called the unit start execution policy. Cross Search If no algorithmic operations need to be performed, the unit execution policy for the motion estimation operations associated with the reference direction of the coding unit can be called the unit end execution policy.

[0093] Since the principle for determining whether the unit execution policy of each coding unit is a unit start execution policy or a unit end execution policy is the same, assuming that the coding units of the target image frame include coding unit bi, where i is a positive integer less than or equal to the total number of all coding units of the target image frame, here we will explain as an example the process of determining whether the unit execution policy of coding unit bi is a unit start execution policy or a unit end execution policy.

[0094] The ratio between the unit component vector value corresponding to the coding unit bi and the unit motion vector value of the coding unit bi can be called the unit component ratio value corresponding to the coding unit bi. When the unit component ratio value corresponding to the coding unit bi is equal to or greater than a certain threshold (which can be set by the user and may be called the third ratio threshold) or the unit component ratio value corresponding to the coding unit bi is equal to or less than a certain threshold (which can be set by the user and may be called the fourth ratio threshold), the coding unit bi can be considered to have significant motion in the horizontal direction or the vertical direction, and therefore it can be determined that the unit execution policy for the coding unit bi is the unit start execution policy. The third ratio threshold is greater than the fourth ratio threshold.

[0095] When the unit component ratio value of coding unit bi is smaller than the third ratio threshold and greater than the fourth ratio threshold, coding unit bi can be considered to have no significant movement in the horizontal or vertical directions, and therefore it can be determined that the unit execution policy for coding unit bi is a unit end execution policy.

[0096] Through the above process, the server can obtain a frame execution policy for the target image frame, and when the frame execution policy for the target image frame is a frame start execution policy, it can also obtain a unit execution policy for each coding unit of the target image frame.

[0097] The server can encode the video data to which the target image frame belongs according to the frame execution policy for the obtained target image frame and the unit execution policy for each coding unit, and obtain encoded data of the video data. The server can send the encoded data to the video client, and have the video client decode the encoded data to obtain decoded data of the video data, and the video client can play the decoded data to realize the playback of the video data.

[0098] For example, the video data to which the target image frame belongs may be sent by the video client of user 1 to the server, and the video client of user 1 requests the video data to be sent to the video client of user 2 by sending the video data to the server. Thus, the server can encode the video data to obtain encoded data of the video data, and then send the encoded data to the video client of user 2, and have the video client of user 2 decode the encoded data, obtain the decoded data, and play the decoded data.

[0099] The present application can obtain at least two coding units included in a target image frame, obtain motion vector information of each coding unit among the at least two coding units, determine global average vector information for the target image frame based on the motion vector information of each coding unit, determine component average vector information in the reference direction of the target image frame based on component vector information in the reference direction of the motion vector information of each coding unit, and determine a frame execution policy for a motion detection operation associated with the reference direction of the target image frame based on the global average vector information and the component average vector information, where the frame execution policy is a frame start execution policy or a frame end execution policy. As can be seen, the method proposed by the present application can determine what frame execution policy to adopt for the target image frame based on the relationship (e.g., the proportion) between the component average vector information and the global average vector information of the target image frame, and adopting different frame execution policies for the target image frame will have different coding speeds for the target image frame. Therefore, the coding speed for the target image frame can be improved by adaptively determining the frame execution policy adopted for the target image frame.

[0100] Please refer to Figure 9, which is a structural schematic diagram of a data encoding device provided by the present application. As shown in Figure 9, the data encoding device 1 may include: a motion information acquisition module 101, a total information determination module 102, a component information determination module 103, and a policy determination module 104; The motion information acquisition module 101 is used for obtaining at least two coding units included in a target image frame, and obtaining motion vector information of each coding unit among the at least two coding units; The global information determining module 102 is used for determining global average vector information for the target image frame according to the motion vector information of each coding unit; The component information determining module 103 is used for determining component average vector information in a reference direction of a target image frame according to the component vector information in a reference direction of the motion vector information of each coding unit; The policy determination module 104 is used for determining a frame execution policy for the motion detection operation in the reference direction of the target image frame based on the overall average vector information and the component average vector information, where the frame execution policy is a frame start execution policy or a frame end execution policy.

[0101] Here, for specific functional implementation methods of the motion information acquisition module 101, the overall information determination module 102, the component information determination module 103, and the policy determination module 104, please refer to steps S101 to S104 in the embodiment corresponding to Figure 3, and detailed explanations will be omitted here.

[0102] Wherein, the overall information determination module 102 includes: a component information acquisition unit 1021, a vector value determination unit 1022, and an overall information determination unit 1023; The component information obtaining unit 1021 is used for obtaining horizontal component vector information and vertical component vector information respectively included in the motion vector information of each coding unit; The motion vector value determination unit 1022 is used for determining a unit motion vector value corresponding to each coding unit according to the horizontal component vector information and the vertical component vector information corresponding to each coding unit; The global information determining unit 1023 is used for determining global average vector information according to the unit motion vector values ​​respectively corresponding to each coding unit and the unit numbers of the at least two coding units.

[0103] Here, the specific functional implementation manner of the component information obtaining unit 1021, the vector value determining unit 1022, and the overall information determining unit 1023 may refer to step S102 in the embodiment corresponding to FIG. 3, and thus detailed description is omitted here.

[0104] wherein the reference direction includes a horizontal reference direction, and component vector information in the reference direction of the motion vector information of each coding unit includes horizontal component vector information; The component information determination module 103 includes a horizontal component acquisition unit 1031, a horizontal value determination unit 1032, and a horizontal information determination unit 1033; The horizontal component obtaining unit 1031 is used for obtaining horizontal component vector information in each horizontal reference direction of the motion vector information of each coding unit; The horizontal value determining unit 1032 is used for determining a horizontal component value corresponding to each coding unit according to the horizontal component vector information in each horizontal reference direction of each coding unit; The horizontal information determining unit 1033 is used for determining component average vector information according to the horizontal component values ​​respectively corresponding to each coding unit and the unit numbers of the at least two coding units.

[0105] Here, the specific functional implementation manner of the horizontal component acquisition unit 1031, the horizontal value determination unit 1032, and the horizontal information determination unit 1033 may refer to step S103 in the embodiment corresponding to FIG. 3, and thus detailed description is omitted here.

[0106] wherein the reference direction includes a vertical reference direction, and component vector information in the reference direction of the motion vector information of each coding unit includes vertical component vector information; The component information determination module 103 includes: a vertical component acquisition unit 1034, a vertical value determination unit 1035, and a vertical information determination unit 1036; The vertical component obtaining unit 1034 is used for obtaining vertical component vector information in each vertical reference direction of the motion vector information of each coding unit; The vertical value determination unit 1035 is used for determining a vertical component value corresponding to each coding unit according to the vertical component vector information in each vertical reference direction of each coding unit; The vertical information determining unit 1036 is used for determining component average vector information according to the vertical component values ​​respectively corresponding to each coding unit and the unit numbers of the at least two coding units.

[0107] Here, the specific functional implementation manners of the vertical component acquisition unit 1034, the vertical value determination unit 1035, and the vertical information determination unit 1036 may refer to step S103 in the embodiment corresponding to FIG. 3, and thus detailed description is omitted here.

[0108] Here, the policy determination module 104 includes: a frame ratio determination unit 1041, a first policy determination unit 1042, and a second policy determination unit 1043; The frame ratio determination unit 1041 is used for determining a frame component ratio value according to the overall average vector information and the component average vector information; The first policy determination unit 1042 is used for determining that a frame execution policy for the motion detection operation of the target image frame is a frame start execution policy when the frame component ratio value is equal to or greater than a first ratio threshold or the frame component ratio value is equal to or less than a second ratio threshold; The second policy determination unit 1043 is used to determine that the frame execution policy for the motion detection operation of the target image frame is a frame end execution policy when the frame component ratio value is smaller than the first ratio threshold and larger than the second ratio threshold.

[0109] Here, the specific function implementation manner of the frame ratio determining unit 1041, the first policy determining unit 1042, and the second policy determining unit 1043 may refer to step S104 in the embodiment corresponding to FIG. 3, and detailed description thereof will be omitted here.

[0110] Here, the above-mentioned device 1 further includes a unit value determination module 105 and a unit policy determination module 106; the unit value determination module 105 is used for determining, when determining that the frame execution policy for the motion detection operation of the target image frame is a frame start execution policy, a unit motion vector value and a unit component vector value of each coding unit according to the motion vector information of each coding unit; The unit policy determination module 106 is used for determining a unit execution policy for the motion estimation operation of each coding unit based on the unit motion vector value and the unit component vector value corresponding to each coding unit.

[0111] Here, the specific functional implementation methods of the unit value determination module 105 and the unit policy determination module 106 can be referred to step S104 in the embodiment corresponding to FIG. 3, so detailed description thereof will be omitted here.

[0112] where the unit execution policy is a unit start execution policy or a unit end execution policy, the at least two coding units include a coding unit bi, and i is a positive integer less than or equal to the number of units of the at least two coding units; The unit policy determination module 106 includes: a unit value determination unit 1061, a first unit policy determination unit 1062, and a second unit policy determination unit 1063; The unit value determination unit 1061 is used for determining a unit component ratio value corresponding to the coding unit bi according to the unit motion vector value and the unit component vector value corresponding to the coding unit bi; The first unit policy determining unit 1062 is used for determining that the unit execution policy for the motion detection operation of the coding unit bi is a unit start execution policy when the unit component ratio value is equal to or greater than a third ratio threshold, or the unit component ratio value is equal to or less than a fourth ratio threshold, and the third ratio threshold is greater than the fourth ratio threshold; The second unit policy determination unit 1063 is used to determine that the unit execution policy for the motion detection operation of the coding unit bi is a unit termination execution policy when the unit component ratio value is smaller than the third ratio threshold and greater than the fourth ratio threshold.

[0113] Here, the specific functional implementation methods of the unit value determination unit 1061, the first unit policy determination unit 1062, and the second unit policy determination unit 1063 may refer to step S104 in the embodiment corresponding to FIG. 3, so detailed description is omitted here.

[0114] Wherein, the motion information acquisition module 101 includes: a reference frame acquisition unit 1011, a prediction block acquisition unit 1012, and a motion information determination unit 1013; The reference frame acquisition unit 1011 is used for acquiring a reference image frame of a target image frame; The prediction block obtaining unit 1012 is used for respectively obtaining prediction image blocks having similarity with each coding unit from a reference image frame; The motion information determining unit 1013 is used for determining the motion vector information of each coding unit according to the relative displacement between each coding unit and the corresponding prediction image block and the target motion vector accuracy.

[0115] Here, the specific functional implementation manners of the reference frame obtaining unit 1011, the prediction block obtaining unit 1012, and the motion information determining unit 1013 may refer to step S101 in the embodiment corresponding to FIG. 3, so detailed description is omitted here.

[0116] Here, the above device 1 further includes a framing module 107 and a target frame determining module 108; The framing module 107 is used for framing the video data to obtain a plurality of image frames contained in the video data; The target frame determination module 108 is used for determining a target image frame from among the plurality of image frames; The reference frame acquisition unit 1011 includes: It is used to obtain an image frame adjacent to the target image frame from among a plurality of image frames as a reference image frame.

[0117] Here, the specific functional implementation manner of the framing module 107 and the target frame determination module 108 may refer to step S101 in the embodiment corresponding to FIG. 3, and thus detailed description is omitted here.

[0118] Here, the device 1 further includes an encoding module 109 and a playback module 110; The encoding module 109 is used for encoding the video data according to the frame execution policy for the motion detection operation of the determined target image frame to obtain encoded data of the video data; The playback module 110 is used for synchronizing the encoded data with the video client, enabling the video client to decode the encoded data to obtain the decoded data of the video data, and playing the decoded data.

[0119] Here, the specific functional implementation manner of the encoding module 109 and the playback module 110 can be referred to step S104 in the embodiment corresponding to FIG. 3, so a detailed description thereof will be omitted here.

[0120] Here, the above device 1 further includes a frame division module 111, a region division module 112, and a unit determination module 113; The frame division module 111 is used for dividing the target image frame into at least two coding regions; The region division module 112 is used for performing region division on each of the at least two coding regions to obtain coding units to which each coding region belongs; The unit determination module 113 is used for determining the coding units to which each coding region belongs, respectively, among at least two coding units included in the target image frame.

[0121] Here, the specific functional implementation manner of the frame division module 111, the region division module 112, and the unit determination module 113 may refer to step S101 in the embodiment corresponding to FIG. 3, and thus detailed description is omitted here.

[0122] The present application obtains at least two coding units included in a target image frame, obtains motion vector information of each coding unit among the at least two coding units, determines overall average vector information for the target image frame based on the motion vector information of each coding unit, determines component average vector information in the reference direction of the target image frame based on component vector information in the reference direction of the motion vector information of each coding unit, and determines a frame execution policy for a motion detection operation associated with the reference direction of the target image frame based on the overall average vector information and the component average vector information, where the frame execution policy is a frame start execution policy or a frame end execution policy. As can be seen, the device proposed by the present application can determine what frame execution policy to adopt for the target image frame based on the relationship (e.g., the proportion) between the component average vector information and the overall average vector information of the target image frame, and adopting different frame execution policies for the target image frame will have different coding speeds for the target image frame. Therefore, the coding speed for the target image frame can be improved by adaptively determining the frame execution policy adopted for the target image frame.

[0123] Referring to FIG. 10, FIG. 10 is a structural schematic diagram of a computer device provided by the present application. As shown in FIG. 10, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. In addition, the computer device 1000 may further include a user interface 1003 and at least one communication bus 1002. Here, the communication bus 1002 is used to realize the connection communication between these components. Here, the user interface 1003 may include a display and a keyboard, and optionally the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (e.g., a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or may be a non-volatile memory, such as at least one magnetic disk memory. The memory 1005 may optionally be at least one storage device separate from the above-mentioned processor 1001. As shown in FIG. 10, an operating system, a network communication module, a user interface module, and a device control application program may be included in memory 1005, which is a type of computer storage medium.

[0124] In the computer device 1000 shown in Fig. 10, the network interface 1004 can provide a network communication function, the user interface 1003 is mainly used to provide an input interface to a user, and the processor 1001 may be used to call a device control application program stored in the memory 1005 to realize the description of the data encoding method in the embodiment corresponding to Fig. 3 above. It should be understood that the computer device 1000 described in this application may execute the description of the data encoding device 1 in the embodiment corresponding to Fig. 9 above, and a detailed description thereof will be omitted here. In addition, a detailed description of the beneficial effects of adopting the same method will also be omitted.

[0125] In addition, it should be noted that the present application further provides a computer-readable storage medium, in which a computer program executed by the above-mentioned data encoding device 1 is stored, and the computer program includes program instructions, enabling the processor to execute the description of the data encoding method in the embodiment corresponding to FIG. 3 when the processor executes the program instructions. Therefore, a detailed description is omitted here. Also, a detailed description of the beneficial effects of adopting the same method is omitted. For technical details not disclosed in the computer storage medium embodiment of the present application, please refer to the description of the method embodiment of the present application.

[0126] As can be understood by those skilled in the art, the implementation of all or part of the processes in the methods of the above embodiments can be completed by issuing instructions to relevant hardware through a computer program, and the above program may be stored in a computer-readable storage medium, and may include the processes of each of the above method embodiments when the program is executed. Here, the above storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0127] What has been disclosed above is merely a preferred embodiment of the present application, and of course, it is not possible to limit the scope of the present application, and therefore, equivalent changes made according to the claims of the present application still belong to the scope of the present application. [Explanation of symbols]

[0128] 1000 Computer Equipment 1001 Processor 1002 Communication Bus 1003 User Interface 1004 Network Interface 1005 Memory 1011 Reference Frame Acquisition Unit 1012 Prediction Block Acquisition Unit 1013 Information Decision Unit 1021 Composition Information Acquisition Unit 1022 Vector Value Decision Unit 1023 Global Information Decision Unit 1031 Horizontal component acquisition unit 1032 Horizontal Value Determination Unit 1033 Horizontal Information Determination Unit 1034 Vertical component acquisition unit 1035 Vertical Value Determination Unit 1036 Vertical Information Determination Unit 1041 Frame Ratio Determination Unit 1042 First Policy Decision Unit 1043 Second Policy Decision Unit 1061 Unit Value Determination Unit 1062 1st Unit Policy Decision Unit 1063 Second Unit Policy Decision Unit

Claims

1. 1. A method for encoding data, comprising the steps of: Obtaining at least two coding units included in a target image frame, and obtaining motion vector information of each coding unit of the at least two coding units; determining overall average motion vector information for the target image frame based on the motion vector information of each of the coding units; determining component average vector information in the reference direction of the target image frame based on component vector information in the reference direction of the motion vector information of each of the coding units; determining a frame execution policy for a motion detection operation in the reference direction of the target image frame based on the overall average vector information and the component average vector information, the frame execution policy being a policy for performing a motion detection operation in the reference direction of the target image frame or a policy for not performing a motion detection operation in the reference direction of the target image frame; encoding the target image frame based on the determined frame execution policy for the motion detection operation of the target image frame.

2. determining overall average motion vector information for the target image frame based on the motion vector information of each of the coding units, obtaining horizontal component vector information and vertical component vector information included in the motion vector information of each of the coding units; determining a unit motion vector value corresponding to each of the coding units based on horizontal component vector information and vertical component vector information corresponding to each of the coding units; and determining the overall average vector information based on unit motion vector values ​​corresponding to each of the coding units and a unit number of the at least two coding units.

3. the reference direction includes a horizontal reference direction, and component vector information in the reference direction of the motion vector information of each of the coding units includes horizontal component vector information; The step of determining component average vector information in the reference direction of the target image frame based on component vector information in the reference direction of the motion vector information of each of the coding units includes: obtaining horizontal component vector information in the horizontal reference direction of the motion vector information of each of the coding units; determining a horizontal component value corresponding to each of the coding units according to horizontal component vector information in the horizontal reference direction of each of the coding units; and determining the component mean vector information based on horizontal component values ​​corresponding to each of the coding units and a unit number of the at least two coding units.

4. the reference direction includes a vertical reference direction, and component vector information in the reference direction of the motion vector information of each of the coding units includes vertical component vector information; The step of determining component average vector information in the reference direction of the target image frame based on component vector information in the reference direction of the motion vector information of each of the coding units includes: obtaining vertical component vector information in the vertical reference direction of the motion vector information of each of the coding units; determining a vertical component value corresponding to each of the coding units according to vertical component vector information in the vertical reference direction of each of the coding units; and determining the component mean vector information based on a vertical component value corresponding to each of the coding units and a unit number of the at least two coding units.

5. determining a frame execution policy for a motion detection operation in the reference direction of the target image frame based on the overall average vector information and the component average vector information, determining a frame component ratio value based on the overall average vector information and the component average vector information; determining, when the frame component ratio value is greater than or equal to a first ratio threshold or when the frame component ratio value is less than or equal to a second ratio threshold, that a frame execution policy for the motion detection operation of the target image frame is a policy to execute a motion detection operation in the reference direction of the target image frame, the first ratio threshold being greater than the second ratio threshold; 2. The method of claim 1, further comprising: determining, when the frame component ratio value is less than the first ratio threshold and greater than the second ratio threshold, that a frame execution policy for the motion detection operation of the target image frame is a policy of not performing a motion detection operation in the reference direction of the target image frame.

6. The method comprises: determining a unit motion vector value and a unit component vector value of each of the coding units based on the motion vector information of each of the coding units when determining that the frame execution policy for the motion estimation operation of the target image frame is the execution policy; 2. The method of claim 1, further comprising: determining a unit execution policy for the motion estimation operation of each of the coding units based on unit motion vector values ​​and unit component vector values ​​corresponding to each of the coding units.

7. the unit execution policy is a policy of executing the motion estimation operation of each of the coding units, or a policy of not executing the motion estimation operation of each of the coding units, the at least two coding units including coding unit bi, i being a positive integer less than or equal to a unit number of the at least two coding units; determining a unit execution policy for the motion estimation operation of each of the coding units based on unit motion vector values ​​and unit component vector values ​​corresponding to each of the coding units, determining a unit component ratio value corresponding to the coding unit bi according to a unit motion vector value and a unit component vector value corresponding to the coding unit bi; determining that a unit execution policy for the motion estimation operation of the coding unit bi is a policy to execute the motion estimation operation of the coding unit bi when the unit component ratio value is equal to or greater than a third ratio threshold or when the unit component ratio value is equal to or less than a fourth ratio threshold, wherein the third ratio threshold is greater than the fourth ratio threshold; 7. The method of claim 6, further comprising: determining, when the unit component ratio value is less than the third ratio threshold and greater than the fourth ratio threshold, that the unit execution policy for the motion detection operation of the coding unit bi is a policy of not executing the motion detection operation of the coding unit bi.

8. The step of obtaining at least two coding units included in a target image frame and obtaining motion vector information of each coding unit of the at least two coding units includes: obtaining a reference image frame for the target image frame; obtaining a predicted image block having similarity to each of the coding units from the reference image frame; and determining motion vector information for each of the coding units based on a relative displacement between each of the coding units and a corresponding predicted image block and a target motion vector accuracy.

9. The method comprises: framing the video data to obtain a plurality of image frames included in the video data; determining the target image frame from among the plurality of image frames; The step of obtaining a reference image frame for the target image frame comprises:

9. The method of claim 8, further comprising the step of obtaining, from the plurality of image frames, an image frame adjacent to the target image frame as the reference image frame.

10. The method comprises: encoding the video data based on the determined frame execution policy for the motion detection operation of the target image frame to obtain encoded data of the video data; 10. The method of claim 9, further comprising: synchronizing the encoded data to a video client; causing the video client to decode the encoded data to obtain decoded data for the video data; and playing the decoded data.

11. The method comprises: dividing the target image frame into at least two coding regions; performing region division on each of the at least two coding regions to obtain a coding unit to which each of the coding regions belongs; The method according to any one of claims 1 to 10, further comprising a step of determining the coding units to which each of the coding regions respectively belong as the at least two coding units included in the target image frame.

12. 1. A data encoding device, comprising: A motion information acquisition module is used for acquiring at least two coding units included in a target image frame, and acquiring motion vector information of each coding unit among the at least two coding units; a global information determining module, which is used for determining global average vector information for the target image frame according to the motion vector information of each of the coding units; a component information determining module, which is used for determining component average vector information of the target image frame in the reference direction according to component vector information of the motion vector information of each coding unit in the reference direction; a policy determination module used for determining a frame execution policy for a motion detection operation in the reference direction of the target image frame based on the overall average vector information and the component average vector information, the frame execution policy being a policy for executing a motion detection operation in the reference direction of the target image frame, or a policy for not executing a motion detection operation in the reference direction of the target image frame; an encoding module that encodes the target image frame based on the determined frame execution policy for the motion detection operation of the target image frame.

13. The overall information determination module includes: a component information obtaining unit for obtaining horizontal component vector information and vertical component vector information respectively included in the motion vector information of each of the coding units; a vector value determination unit, which is used for determining a unit motion vector value corresponding to each of the coding units according to horizontal component vector information and vertical component vector information corresponding to each of the coding units; The apparatus of claim 12, further comprising: an overall information determining unit used for determining the overall average vector information based on unit motion vector values ​​corresponding to each of the coding units and the unit number of the at least two coding units.

14. the reference direction includes a horizontal reference direction, and component vector information in the reference direction of the motion vector information of each coding unit includes horizontal component vector information; The ingredient information determination module includes: a horizontal component obtaining unit for obtaining horizontal component vector information of the motion vector information of each coding unit in a horizontal reference direction; a horizontal value determining unit, which is used for determining a horizontal component value corresponding to each coding unit according to horizontal component vector information in a horizontal reference direction of each coding unit; The apparatus of claim 12, further comprising: a horizontal information determining unit used to determine component average vector information based on horizontal component values ​​corresponding to each coding unit and the unit number of the at least two coding units.

15. the reference direction includes a vertical reference direction, and component vector information in the reference direction of the motion vector information of each coding unit includes vertical component vector information; The ingredient information determination module includes: a vertical component acquisition unit, which is used for acquiring vertical component vector information in a vertical reference direction of the motion vector information of each coding unit; a vertical value determining unit, which is used for determining a vertical component value corresponding to each coding unit according to vertical component vector information in a vertical reference direction of each coding unit; The apparatus of claim 12, further comprising: a vertical information determining unit used to determine component average vector information based on vertical component values ​​corresponding to each coding unit and the unit number of the at least two coding units.

16. The policy decision module includes: a frame ratio determining unit for determining a frame component ratio value according to the overall average vector information and the component average vector information; a first policy determination unit used for determining that a frame execution policy for a motion detection operation of a target image frame is a policy for executing a motion detection operation in the reference direction of the target image frame when the frame component ratio value is equal to or greater than a first ratio threshold or when the frame component ratio value is equal to or less than a second ratio threshold; The apparatus of claim 12, further comprising: a second policy determination unit used to determine that a frame execution policy for a motion detection operation of a target image frame is a policy of not executing a motion detection operation in the reference direction of the target image frame when the frame component ratio value is smaller than a first ratio threshold and larger than a second ratio threshold.

17. The apparatus comprises: a unit value determination module, which is used for determining a unit motion vector value and a unit component vector value of each coding unit according to the motion vector information of each coding unit when it is determined that the frame execution policy for the motion detection operation of the target image frame is the execution policy; and a unit policy determination module, which is used to determine a unit execution policy for a motion estimation operation of each coding unit according to the unit motion vector value and the unit component vector value corresponding to each coding unit; 13. The apparatus of claim 12, wherein the unit execution policy is a policy to execute the motion estimation operation of each of the coding units or a policy to not execute the motion estimation operation of each of the coding units.

18. A computing device comprising a memory and a processor, the memory storing a computer program, the computer program causing the processor to perform a method according to any one of claims 1 to 11 when executed by the processor.

19. A computer program, characterized in that when it is executed by a processor, it performs the method according to any one of claims 1 to 11.

Citation Information

Patent Citations

  • System and device for converting number of movement compensation frames of picture signal

    JP2001024988A

  • Motion vector searching apparatus and motion vector searching method

    JP2003324743A

  • Encoding circuit and encoding method

    JP2016063358A

  • Motion vector detecting apparatus, motion vector detecting method, and motion vector detecting program

    JP2017204752A