Method and device for calculating motion vector of target macroblock in video frame

Abstract

The embodiments of the present disclosure provide a method and device for calculating the motion vector of a target macroblock in a video frame. The method includes: determining a plurality of candidate macroblocks corresponding to the target macroblock in a previous frame of the video frame; using an SSE eigenfunction to calculate a sum of absolute differences (SAD) value between the target macroblock and each of the plurality of candidate macroblocks, wherein the number of bits of the operand of the SSE eigenfunction is determined according to the size of the target macroblock; determining the most similar macroblock in the previous frame of the video frame that has the smallest difference from the target macroblock according to the SAD value; and calculating the motion vector between the target macroblock and the most similar macroblock.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a method and device for calculating a motion vector of a target macroblock in a video frame. Background Art

[0002] With the development of Internet technology, people are increasingly relying on browsers to work and live. For example, you can watch online videos or live broadcasts, conduct video communications, or directly edit videos on the browser. During the live broadcast, the terminal used by the anchor can use the browser to encode the real-time collected video in real time, so that the video file uploaded to the network is compressed, reducing the upload pressure of the terminal. In some scenarios, such as the zero-hour countdown or the live broadcast of real-time rush purchases, people expect the client to receive the live video signal with as little delay as possible to avoid missing the timely interaction with the anchor. Similarly, in the process of video communication, the user also expects that his communication with the other party is real-time. In the above process, in addition to network transmission factors, the video encoding rate is also a factor that affects the delay of the video signal. Summary of the invention

[0003] The embodiments described herein provide a method and apparatus for calculating a motion vector of a target macroblock in a video frame, and a computer-readable storage medium storing a computer program. Furthermore, the embodiments described herein also provide a video encoding method.

[0004] According to a first aspect of the present disclosure, a method for calculating a motion vector of a target macroblock in a video frame is provided. The method comprises: determining a plurality of candidate macroblocks corresponding to the target macroblock in a previous frame of the video frame; using an SSE eigenfunction to calculate a sum of absolute differences (SAD) value between the target macroblock and each of the plurality of candidate macroblocks, wherein the number of bits of the operand of the SSE eigenfunction is determined according to the size of the target macroblock; determining the most similar macroblock in the previous frame of the video frame that has the smallest difference from the target macroblock according to the SAD value; and calculating a motion vector between the target macroblock and the most similar macroblock.

[0005] In some embodiments of the present disclosure, the size of the target macroblock includes one or more of the following: the width of the target macroblock, and the height of the target macroblock.

[0006] In some embodiments of the present disclosure, if the size of the target macroblock is greater than a threshold, the number of bits of the operand of the SSE intrinsic function is equal to the size of the target macroblock multiplied by the number of bits occupied by a pixel; if the size of the target macroblock is less than or equal to the threshold, the number of bits of the operand of the SSE intrinsic function is equal to the threshold multiplied by the number of bits occupied by a pixel.

[0007] In some embodiments of the present disclosure, determining the most similar macroblock with the smallest difference from the target macroblock in the previous frame of the video frame based on the SAD value includes: determining the candidate macroblock with the smallest SAD value between the target macroblock and each of the multiple candidate macroblocks as the most similar macroblock.

[0008] In some embodiments of the present disclosure, determining the most similar macroblock in a previous frame of a video frame that has the smallest difference with a target macroblock based on a SAD value includes: determining a first candidate macroblock having the smallest SAD value between the target macroblock and each of a plurality of candidate macroblocks; determining a plurality of second candidate macroblocks based on the first candidate macroblock, the plurality of second candidate macroblocks being macroblocks around the first candidate macroblock and having a sub-pixel distance therefrom; calculating a converted absolute difference and a SATD value between the target macroblock and each of the plurality of second candidate macroblocks; and determining the second candidate macroblock having the smallest SATD value between the target macroblock and each of the second candidate macroblocks as the most similar macroblock.

[0009] In some embodiments of the present disclosure, the SSE intrinsic functions used to calculate the SAD value include one or more of the following: _mm_loadu_si128; _mm_loadl_epi64; _mm_sad_epu8; and _mm_extract_epi16.

[0010] In some embodiments of the present disclosure, the SSE intrinsic functions used to calculate the SATD value include one or more of the following: _mm_load_si128; _mm_shuffle_ps; _mm_maddubs_epi16; _mm_sub_epi16; _mm_add_epi16; _mm_add_epi32; _mm_unpacklo_epi64; _mm_blend_epi16; _mm_slli_epi32; _mm_srli_epi32; _mm_or_si128; _mm_abs_epi16; _mm_max_epi16; and _mm_shufflelo_epi16.

[0011] In some embodiments of the present disclosure, the method is used for video software encoding based on WebAssembly.

[0012] According to the second aspect of the present disclosure, a device for calculating the motion vector of a target macroblock in a video frame is provided. The device includes: at least one processor; and at least one memory storing a computer program; wherein, when the computer program is executed by the at least one processor, the device determines a plurality of candidate macroblocks corresponding to the target macroblock in a previous frame of the video frame; uses an SSE eigenfunction to calculate a sum of absolute differences (SAD) value between the target macroblock and each of the plurality of candidate macroblocks, the number of bits of the operand of the SSE eigenfunction being determined according to the size of the target macroblock; determines the most similar macroblock in the previous frame of the video frame that has the smallest difference from the target macroblock according to the SAD value; and calculates the motion vector between the target macroblock and the most similar macroblock.

[0013] In some embodiments of the present disclosure, the size of the target macroblock includes one or more of the following: the width of the target macroblock, and the height of the target macroblock.

[0014] In some embodiments of the present disclosure, if the size of the target macroblock is greater than a threshold, the number of bits of the operand of the SSE intrinsic function is equal to the size of the target macroblock multiplied by the number of bits occupied by a pixel; if the size of the target macroblock is less than or equal to the threshold, the number of bits of the operand of the SSE intrinsic function is equal to the threshold multiplied by the number of bits occupied by a pixel.

[0015] In some embodiments of the present disclosure, when the computer program is executed by at least one processor, the device determines the most similar macroblock in the previous frame of the video frame that has the smallest difference with the target macroblock based on the SAD value by performing the following operations: determining the candidate macroblock with the smallest SAD value between the target macroblock and each of the multiple candidate macroblocks as the most similar macroblock.

[0016] In some embodiments of the present disclosure, when the computer program is executed by at least one processor, the device determines the most similar macroblock in the previous frame of the video frame that has the smallest difference with the target macroblock based on the SAD value by the following operations: determining a first candidate macroblock having the smallest SAD value between the target macroblock and each of a plurality of candidate macroblocks; determining a plurality of second candidate macroblocks based on the first candidate macroblock, wherein the plurality of second candidate macroblocks are macroblocks around the first candidate macroblock and are within a sub-pixel range of distance therefrom; calculating the converted absolute difference and SATD value between the target macroblock and each of the plurality of second candidate macroblocks; and determining the second candidate macroblock having the smallest SATD value between the target macroblock and each of the second candidate macroblocks as the most similar macroblock.

[0017] In some embodiments of the present disclosure, the SSE intrinsic functions used to calculate the SAD value include one or more of the following: _mm_loadu_si128; _mm_loadl_epi64; _mm_sad_epu8; and _mm_extract_epi16.

[0018] In some embodiments of the present disclosure, the SSE intrinsic functions used to calculate the SATD value include one or more of the following: _mm_load_si128; _mm_shuffle_ps; _mm_maddubs_epi16; _mm_sub_epi16; _mm_add_epi16; _mm_add_epi32; _mm_unpacklo_epi64; _mm_blend_epi16; _mm_slli_epi32; _mm_srli_epi32; _mm_or_si128; _mm_abs_epi16; _mm_max_epi16; and _mm_shufflelo_epi16.

[0019] According to a third aspect of the present disclosure, a computer-readable storage medium storing a computer program is provided, wherein the computer program implements the steps of the method according to the first aspect of the present disclosure when executed by a processor.

[0020] According to a fourth aspect of the present disclosure, a video encoding method is provided. The method includes: encoding a video frame using a video hardware encoding method; in response to a failure of the video hardware encoding method, encoding the video frame using a video software encoding method based on WebAssembly, the video software encoding method and the video hardware encoding method provide the same user interface, and the video software encoding method uses the method described in the first aspect of the present disclosure to calculate the motion vector of the target macroblock in the video frame. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly described below. It should be noted that the drawings described below only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure, wherein:

[0022] Figure 1 is an exemplary flow chart of a method for calculating a motion vector of a target macroblock in a video frame according to an embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram of a target macroblock;

[0024] Figure 3 is a schematic diagram of a candidate macroblock;

[0025] Figure 4 is a schematic block diagram of an apparatus for calculating a motion vector of a target macroblock in a video frame according to an embodiment of the present disclosure; and

[0026] Figure 5 is an exemplary flowchart of a video encoding method according to an embodiment of the present disclosure.

[0027] The elements in the drawings are schematic and not drawn to scale. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work also fall within the scope of protection of the present disclosure.

[0029] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those commonly understood by those skilled in the art to which the subject matter of the present disclosure belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the specification and the relevant technology, and will not be interpreted in an idealized or overly formal form unless otherwise clearly defined herein. As used herein, statements that two or more parts are "connected" or "coupled" together should refer to these parts being directly combined together or combined through one or more intermediate components. In addition, terms such as "first" and "second" are only used to distinguish one component (or a part of a component) from another component (or another part of a component).

[0030] Currently, browsers do not provide systematic multimedia development capabilities. If you need to encode videos on a browser, the only way is to compile the corresponding C++ source code into WebAssembly through Emscripten. Among them, Emscripten is a complete compilation tool chain for WebAssembly on the Web platform. However, general C++ libraries have assembly optimizations for the target architecture, i.e., the CPU instruction set, which is not included in the virtual machine architecture such as WebAssembly. As a result, the compiled WebAssembly has the worst performance among all platforms. Chrome 94 browser developed hardware encoding for the first time, but because it is more complicated to use on the browser, there is still a long way to go before developers can really take advantage of it.

[0031] The embodiments of the present disclosure propose a software encoding optimization solution for the web platform, and on this basis propose a video encoding solution. The video encoding solution can combine the hardware encoding solution and the aforementioned optimized software encoding solution to improve the encoding efficiency as much as possible, bringing convenience to multimedia development.

[0032] Specifically, in a software encoding optimization solution for a web platform, an embodiment of the present disclosure proposes a method for calculating a motion vector of a target macroblock in a video frame. In video encoding, each video frame is usually divided into multiple macroblocks. The size of a macroblock is, for example, 16 (width) × 16 (height), 8 × 8 or 4 × 4. The video encoding algorithm encodes macroblocks one by one in units of macroblocks, and then organizes the encoded data into a continuous video code stream. Therefore, if the encoding efficiency of macroblocks is improved, the encoding efficiency of video frames can be improved. The method proposed in the embodiment of the present disclosure can reduce the time for calculating the motion vector of the target macroblock, thereby improving the encoding efficiency of the video frame.

[0033] Figure 1 is an exemplary flow chart of a method for calculating a motion vector of a target macroblock in a video frame according to an embodiment of the present disclosure. Figure 1 A method 100 for calculating a motion vector for a target macroblock in a video frame is described below.

[0034] In block S102 , a plurality of candidate macroblocks corresponding to a target macroblock in a previous frame of a video frame are determined. Figure 2 FIG. 2 is a schematic diagram showing a target macroblock 200 in a video frame. Figure 2 In the example, the size of the target macroblock 200 is 4×4, with a width of 4 pixels and a height of 4 pixels. The target macroblock 200 includes 16 pixels with coordinates of (1, 1), (1, 2), (1, 3), (1, 4), (2, 1), (2, 2), (2, 3), (2, 4), (3, 1), (3, 2), (3, 3), (3, 4), (4, 1), (4, 2), (4, 3), (4, 4). Due to the continuity of video recording, usually part of the picture of a video frame and its previous frame may be the same or similar. Therefore, two consecutive video frames will not change too much. If a macroblock similar to the target macroblock can be found in the previous frame, the encoding information of the macroblock can be used to encode the target macroblock, thereby reducing the data to be encoded and improving the compression rate of the video. Based on the above principle, multiple candidate macroblocks corresponding to the target macroblock can be found in the previous frame of the video frame. Since two consecutive video frames do not change much, the candidate macroblocks in the previous frame may be macroblocks near the location of the target macroblock.

[0035] Figure 3Schematic diagram showing two exemplary candidate macroblocks in the previous frame. Candidate macroblock 301 may, for example, include 16 pixels with coordinates (1,1), (1,2), (1,3), (1,4), (2,1), (2,2), (2,3), (2,4), (3,1), (3,2), (3,3), (3,4), (4,1), (4,2), (4,3), (4,4). Candidate macroblock 302 may, for example, include 16 pixels with coordinates (2,2), (2,3), (2,4), (2,5), (3,2), (3,3), (3,4), (3,5), (4,2), (4,3), (4,4), (4,5), (5,2), (5,3), (5,4), (5,5).

[0036] Back to Figure 1 In block S104, a SSE eigenfunction is used to calculate a sum of absolute differences (SAD) value between the target macroblock and each of the plurality of candidate macroblocks, and the number of bits of an operand of the SSE eigenfunction is determined according to the size of the target macroblock.

[0037] SAD is the sum of the absolute values ​​of the differences between each pixel in the target macroblock and the corresponding pixel in the candidate macroblock. Figure 2 and Figure 3 In the example of , pixel (1, 1) in target macroblock 200 corresponds to pixel (1, 1) in candidate macroblock 301. Pixel (1, 2) in target macroblock 200 corresponds to pixel (1, 2) in candidate macroblock 301. And so on. Pixel (1, 1) in target macroblock 200 corresponds to pixel (2, 2) in candidate macroblock 302. Pixel (1, 2) in target macroblock 200 corresponds to pixel (2, 3) in candidate macroblock 302. And so on.

[0038] Usually, when calculating SAD, it is necessary to take each pixel as an operand to perform an absolute value of difference operation for the pixel, and then add the results of each operation. In some embodiments of the present disclosure, it is proposed to take each row of pixels or each column of pixels in the target macroblock as an operand, and perform an absolute value of difference operation for each operand. This can significantly reduce the number of operations. To this end, some embodiments of the present disclosure propose to use SSE eigenfunction to calculate SAD value. The number of bits of the operand of SSE eigenfunction can be determined according to the size of the target macroblock. The size of the target macroblock can be the width of the target macroblock, or the height of the target macroblock, or both. For example, in the case where the number of bits of the operand of SSE eigenfunction is determined according to the width of the target macroblock, an absolute value of difference operation can be performed for each row of pixels in the target macroblock. In the case where each pixel occupies, for example, 8 bits, for a target macroblock with a width of 4 pixels, the operand of SSE eigenfunction can be 8×4=32 bits. Take the lower 16 bits from the calculation result of SSE eigenfunction as the result of this operation. In this way, SSE eigenfunction can process 4 pixels in one operation. Similarly, for a target macroblock with a width of 16 pixels, the operand of the SSE eigenfunction can be 8×16=128 bits. The lower 16 bits are also taken from the calculation result of the SSE eigenfunction as the result of this operation. In this way, the SSE eigenfunction can process 16 pixels in one operation. Compared with the solution of processing 1 pixel in one operation, the embodiment of the present disclosure can significantly reduce the number of operations and improve the operation efficiency.

[0039] In some embodiments of the present disclosure, according to the characteristics of WebAssembly, the operand of the SSE intrinsic function may have a lower limit value. Dividing the lower limit value by the number of bits occupied by a pixel can obtain a threshold value. If the size of the target macroblock is greater than the threshold value, the number of bits of the operand of the SSE intrinsic function is equal to the size of the target macroblock multiplied by the number of bits occupied by a pixel. If the size of the target macroblock is less than or equal to the threshold value, the number of bits of the operand of the SSE intrinsic function is equal to the threshold value multiplied by the number of bits occupied by a pixel, that is, the above-mentioned lower limit value.

[0040] For example, the operand of the SSE intrinsic function may be at least 64 bits. Therefore, the above lower limit is 64 bits. In the case where a pixel occupies 8 bits and the number of bits of the operand of the SSE intrinsic function is determined according to the width of the target macroblock, if the width of the target macroblock exceeds 8 pixels, the number of bits of the operand of the SSE intrinsic function is equal to the width of the target macroblock multiplied by 8. If the width of the target macroblock is equal to or less than 8 pixels, the number of bits of the operand of the SSE intrinsic function is equal to 64 bits.

[0041] In block S106, the most similar macroblock in the previous frame of the video frame with the smallest difference from the target macroblock is determined based on the SAD value. In some embodiments of the present disclosure, determining the most similar macroblock in the previous frame of the video frame with the smallest difference from the target macroblock based on the SAD value may include: determining the candidate macroblock with the smallest SAD value between the target macroblock and each of the plurality of candidate macroblocks as the most similar macroblock. The SAD value is a reference value for determining the most similar macroblock at the pixel level.

[0042] Furthermore, the most similar macroblock at the sub-pixel level can also be determined based on the converted Sum of Absolute Difference (SATD) value, where SATD is the sum of the absolute values ​​of the difference between each pixel in the target macroblock and the corresponding pixel in the candidate macroblock after Hadamard transformation. Specifically, in some embodiments of the present disclosure, determining the most similar macroblock with the smallest difference from the target macroblock in the previous frame of the video frame based on the SAD value may include: determining a first candidate macroblock with the smallest SAD value between the target macroblock and each of a plurality of candidate macroblocks (equivalent to a rough selection); determining a plurality of second candidate macroblocks (equivalent to a fine selection) based on the first candidate macroblock, the plurality of second candidate macroblocks being macroblocks around the first candidate macroblock with a distance therefrom within a sub-pixel range (for example, a macroblock with a half-pixel distance from the first candidate macroblock, a macroblock with a one-sub-pixel distance from the first candidate macroblock, a macroblock with a two-sub-pixel distance from the first candidate macroblock, etc.); calculating the SATD value between the target macroblock and each of the plurality of second candidate macroblocks; and determining the second candidate macroblock with the smallest SATD value between the target macroblock and each of the second candidate macroblocks as the most similar macroblock.

[0043] In some embodiments of the present disclosure, it is necessary to determine which SSE intrinsic functions can be used to calculate the SAD values ​​and SATD values ​​of multiple pixels in one operation according to the characteristics of WebAssembly. In some embodiments of the present disclosure, the SSE intrinsic functions used to calculate the SAD value may include one or more of the following: _mm_loadu_si128; _mm_loadl_epi64; _mm_sad_epu8; and _mm_extract_epi16. In some embodiments of the present disclosure, the SSE intrinsic functions used to calculate the SATD value may include one or more of the following: _mm_load_si128; _mm_shuffle_ps; _mm_maddubs_epi16; _mm_sub_epi16; _mm_add_epi16; _mm_add_epi32; _mm_unpacklo_epi64; _mm_blend_epi16; _mm_slli_epi32; _mm_srli_epi32; _mm_or_si128; _mm_abs_epi16; _mm_max_epi16; and _mm_shufflelo_epi16.

[0044] In block S108, the motion vector between the target macroblock and the most similar macroblock is calculated. After the most similar macroblock is determined in block S106, the motion vector between the target macroblock and the most similar macroblock can be calculated using the existing method of calculating the vector between two macroblocks. For example, if the most similar macroblock is Figure 3 , the motion vector between the target macroblock 200 and the most similar macroblock 301 is (0, 0). If the most similar macroblock is Figure 3 , then the motion vector between the target macroblock 200 and the most similar macroblock 301 is (1, 1).

[0045] In some embodiments of the present disclosure, the method 100 is used for video software encoding based on WebAssembly.

[0046] Figure 4 FIG. 4 is a schematic block diagram of an apparatus 400 for calculating a motion vector of a target macroblock in a video frame according to an embodiment of the present invention. Figure 4 As shown, the device 400 may include a processor 410 and a memory 420 storing a computer program. When the computer program is executed by the processor 410, the device 400 may perform the following operations: Figure 1Steps of method 100 shown. In one example, apparatus 400 may be a computer device or a cloud computing node. Apparatus 400 may determine a plurality of candidate macroblocks corresponding to a target macroblock in a previous frame of a video frame; use an SSE eigenfunction to calculate a sum of absolute differences (SAD) value between the target macroblock and each of the plurality of candidate macroblocks, wherein the number of bits of the operand of the SSE eigenfunction is determined according to the size of the target macroblock; determine the most similar macroblock in a previous frame of the video frame that has the smallest difference from the target macroblock according to the SAD value; and calculate a motion vector between the target macroblock and the most similar macroblock.

[0047] In some embodiments of the present disclosure, the apparatus 400 may determine the candidate macroblock having the smallest SAD value between the target macroblock and each of the plurality of candidate macroblocks as the most similar macroblock.

[0048] In some embodiments of the present disclosure, the device 400 may determine a first candidate macroblock having a minimum SAD value between a target macroblock and each of a plurality of candidate macroblocks; determine a plurality of second candidate macroblocks based on the first candidate macroblock, wherein the plurality of second candidate macroblocks are macroblocks around the first candidate macroblock and are within a sub-pixel range of distance therefrom; calculate a converted absolute difference and a SATD value between the target macroblock and each of the plurality of second candidate macroblocks; and determine the second candidate macroblock having a minimum SATD value between the target macroblock and each of the second candidate macroblocks as the most similar macroblock.

[0049] In an embodiment of the present disclosure, the processor 410 may be, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a processor based on a multi-core processor architecture, etc. The memory 420 may be any type of memory implemented using data storage technology, including but not limited to random access memory, read-only memory, semiconductor-based memory, flash memory, disk storage, etc.

[0050] In addition, in the embodiment of the present disclosure, the apparatus 400 may also include an input device 430, such as a microphone, a camera, a keyboard, a mouse, etc., for inputting video files. In addition, the apparatus 400 may also include an output device 440 for outputting compressed files.

[0051] Furthermore, the embodiments of the present disclosure also provide a video encoding method. The video encoding scheme can be combined with a hardware encoding scheme and a software encoding scheme. Figure 5 A video encoding method 500 according to an embodiment of the present disclosure is shown.

[0052] As understood by those skilled in the art, the efficiency of the hardware encoding solution is higher than that of the software encoding solution. Therefore, at box S502, a video hardware encoding method is used to encode the video frame. In some embodiments of the present disclosure, a video hardware encoding method based on WebCodecs may be used to encode the video frame. In some embodiments of the present disclosure, a new Encoder may be called to instantiate the encoder, and the callback function output may be set. When the encoder completes the encoding of a frame of data, it will package the binary code stream of the encoding result into the encoding block EncodedChunk, and use this as a parameter to call the callback function output, thereby outputting the encoding block from the encoder to the outside. In addition to carrying the code stream, the encoding block also needs to carry the pixel format, frame size, frame timestamp, and duration. In this way, if decoding is required in the same environment, the required parameters can be provided to the decoder.

[0053] In some embodiments of the present disclosure, the encoder can be configured by calling the configure function of the encoder. The encoding parameter set of the function call includes: a string representing the encoding standard, the width and height of the encoded video, the frame rate, the bit rate, and the encoding compression level;

[0054] In some embodiments of the present disclosure, the data to be encoded can be input to the encoder by calling the encoder's encode function with Frame as a parameter. Frame is responsible for carrying the YUV color space representation of a frame of image. When it is considered that the number of input frames reaches the specified group of pictures (GOP) size (each group of pictures is considered to be independent and does not affect each other when being decoded), the flush function is called to flush the encoder. Flushing will encode the remaining frames in the encoder and output the encoded blocks. After the output is completed, the state can be cleared to prepare for receiving the next set of data or shutting down the encoder. After shutting down the encoder, system resources can be released.

[0055] In some cases, the video hardware encoding method may fail. At block S504, it is determined whether the video hardware encoding method has failed. If the video hardware encoding method has not failed ("No" at block S504), the process proceeds to block S508 to encapsulate the encoded bitstream.

[0056] If the video hardware encoding method fails ("yes" at block S504), the video frame is encoded using a video software encoding method based on WebAssembly at block S506. The video software encoding method and the video hardware encoding method can provide the same user interface. The video software encoding method adopts the method based on Figure 1 The method shown is used to calculate the motion vector of the target macroblock in the video frame. Therefore, the video software encoding method has improved encoding efficiency.

[0057] After the encoding of the video frame is completed at block S506, the process proceeds to block S508 to encapsulate the encoded code stream.

[0058] Since a consistent interface is provided for hardware encoding and software encoding, it can adaptively switch between hardware encoding and software encoding according to browser support, eliminating the migration and upgrade costs for developers.

[0059] In other embodiments of the present disclosure, a computer-readable storage medium storing a computer program is further provided, wherein the computer program can achieve the following when executed by a processor: Figure 1 Or the steps of the method shown in 5.

[0060] In summary, the embodiments of the present disclosure shorten the time for calculating the motion vector of the target macroblock in the video frame by processing multiple pixel data in one operation, thereby improving the calculation efficiency. Therefore, the coding efficiency of the video software coding scheme in which the above-mentioned method for calculating the motion vector of the macroblock is adopted is also improved. In addition, the embodiments of the present disclosure also provide a video coding method including hardware coding and software coding. By providing a consistent interface for hardware coding and software coding, it is possible to adaptively switch between hardware coding and software coding according to the browser support, eliminating the migration and upgrade costs of developers.

[0061] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device and method according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0062] Unless the context clearly indicates otherwise, the singular form of the words used herein and in the appended claims includes the plural and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Similarly, the words "comprise" and "include" are to be interpreted as inclusive rather than exclusive. Likewise, the terms "include" and "or" should be interpreted as inclusive unless such interpretation is expressly prohibited herein. Where the term "example" is used herein, particularly when it is located after a group of terms, the "example" is merely exemplary and illustrative and should not be considered exclusive or comprehensive.

[0063] Further aspects and scopes of adaptability become apparent from the description provided herein. It should be understood that various aspects of the present application can be implemented individually or in combination with one or more other aspects. It should also be understood that the description and specific embodiments herein are intended for purposes of illustration only and are not intended to limit the scope of the present application.

[0064] Several embodiments of the present disclosure are described in detail above, but it is obvious that those skilled in the art can make various modifications and variations to the embodiments of the present disclosure without departing from the spirit and scope of the present disclosure. The protection scope of the present disclosure is defined by the attached claims.

Claims

1. A method for calculating a motion vector of a target macroblock in a video frame, comprising: Determine a plurality of candidate macroblocks corresponding to the target macroblock in a previous frame of the video frame, wherein a size of each of the candidate macroblocks is the same as a size of the target macroblock; Using an SSE intrinsic function to calculate the absolute difference and SAD value between the target macroblock and each of the multiple candidate macroblocks, the number of bits of the operand of the SSE intrinsic function is determined according to the size of the target macroblock, wherein, in the case where the operand of the SSE intrinsic function has a lower limit value, the ratio of the lower limit value to the number of bits occupied by a pixel is determined as a threshold, and in the case where the size of the target macroblock is greater than the threshold, the product of the size and the number of bits occupied by a pixel is determined as the number of bits of the operand of the SSE intrinsic function, and in the case where the size of the target macroblock is less than or equal to the threshold, the product of the threshold and the number of bits occupied by a pixel is determined as the number of bits of the operand of the SSE intrinsic function, and the operation result corresponding to each operand of each candidate macroblock is the lower 16 bits of the calculation result of the SSE intrinsic function on the corresponding operand; Determine a first candidate macroblock with the smallest SAD value between the target macroblock and each of the plurality of candidate macroblocks; determine a plurality of second candidate macroblocks based on the first candidate macroblock, wherein the plurality of second candidate macroblocks are macroblocks around the first candidate macroblock and within a sub-pixel range of the first candidate macroblock; calculate a SATD value between the target macroblock and each of the plurality of second candidate macroblocks; determine the second candidate macroblock with the smallest SATD value between the target macroblock and each of the second candidate macroblocks as the most similar macroblock; and A motion vector between the target macroblock and the most similar macroblock is calculated.

2. The method according to claim 1, wherein: Determining the most similar macroblock with the smallest difference from the target macroblock in the previous frame of the video frame according to the SAD value includes: The candidate macroblock with the smallest SAD value between the target macroblock and each of the multiple candidate macroblocks is determined as the most similar macroblock.

3. The method according to claim 1, wherein: The SSE eigenfunctions used to calculate the SAD value include one or more of the following: _mm_loadu_si128; _mm_loadl_epi64; _mm_sad_epu8; and _mm_extract_epi16.

4. The method according to claim 1, wherein: The SSE eigenfunctions used to calculate the SATD value include one or more of the following: _mm_load_si128; _mm_shuffle_ps; _mm_maddubs_epi16; _mm_sub_epi16; _mm_add_epi16; _mm_add_epi32; _mm_unpacklo_epi64; _mm_blend_epi16; _mm_slli_epi32; _mm_srli_epi32; _mm_or_si128; _mm_abs_epi16; _mm_max_epi16; and _mm_shufflelo_epi16.

5. The method according to claim 1, wherein: The method is used for video software encoding based on WebAssembly.

6. A device for calculating a motion vector of a target macroblock in a video frame, comprising: at least one processor; as well as at least one memory storing a computer program; Wherein, when the computer program is executed by the at least one processor, the device is caused to perform the steps of the method according to any one of claims 1-5.

7. A video encoding method, comprising: Use video hardware encoding to encode video frames; In response to the failure of the video hardware encoding method, a video software encoding method based on WebAssembly is used to encode the video frame. The video software encoding method provides the same user interface as the video hardware encoding method. The video software encoding method uses the method according to any one of claims 1 to 4 to calculate the motion vector of the target macroblock in the video frame.

Images