Video coding method and device, medium, program product and electronic equipment

Abstract

The invention relates to the technical field of video coding and decoding, and discloses a video coding method and device, a medium, a program product and electronic equipment. The video coding method comprises the following steps: acquiring a video stream to be coded; determining difference result data between the current frame and an adjacent frame adjacent to the current frame under the condition that the current frame to be coded in the video stream is not the first frame of the video stream; determining a coding type matched with the current frame according to the difference result data of the current frame and the adjacent frame; and coding the current frame according to the coding type. According to the invention, the coding type matched with the current frame is determined according to the difference result data of the current frame and the adjacent frame, and then the current frame is coded according to the determined coding type, so that the problem of large code stream caused by intra-frame coding of the video frame suitable for inter-frame coding can be avoided, the video code stream can be reduced, and the coding efficiency is improved. The video code stream transmission speed is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of video coding and decoding technology, and in particular to a video coding method, device, medium, program product and electronic equipment. Background Art

[0002] Video coding technology, as the core of multimedia data processing, is widely used in scenarios requiring efficient transmission and storage. Video coding technology is increasingly being used in applications such as live streaming / on-demand broadcasting, remote conferencing, telemedicine, and security monitoring, all of which involve video encoding, decoding, and transmission. Furthermore, with the rapid development of 5G, the Internet of Things, and artificial intelligence (AI), video transmission requirements are becoming increasingly diversified, with higher resolutions and real-time performance requirements. Given limited transmission resources, the use of inappropriate video coding technology can result in larger encoded video streams and higher transmission latency, impacting the user experience. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a video encoding method, apparatus, medium, program product, and electronic device. The present invention determines a coding type that matches the current frame based on difference data between the current frame and adjacent frames, and then encodes the current frame based on the determined coding type. This avoids the problem of large bitrates caused by intra-coding video frames suitable for inter-frame coding, thereby reducing the video bitrate, increasing video transmission speed, and improving the user experience.

[0004] In a first aspect, an embodiment of the present invention provides a video encoding method for an electronic device, the video encoding method comprising: Get the video stream to be encoded; When a current frame to be encoded in the video stream is not the first frame of the video stream, determining difference result data between the current frame and adjacent frames adjacent to the current frame; Determine the encoding type that matches the current frame based on the difference result data between the current frame and the adjacent frames; Encode the current frame according to the encoding type.

[0005] In a possible implementation of the first aspect, when a current frame to be encoded in a video stream is not the first frame of the video stream, difference result data between the current frame and adjacent frames adjacent to the current frame is determined in the following manner: Determining a plurality of target macroblocks in a current frame and a plurality of reference macroblocks corresponding to each target macroblock in adjacent frames, wherein a position of each target macroblock in the current frame is correlated with a position of the plurality of reference macroblocks corresponding to each target macroblock in adjacent frames; performing difference calculations on each target macroblock and each reference macroblock in a plurality of reference macroblocks corresponding to each target macroblock, respectively, to obtain a plurality of difference values corresponding to each target macroblock; Determining minimum difference values corresponding to each target macroblock and position information of a reference macroblock corresponding to each minimum difference value; The minimum difference value corresponding to each target macroblock and the position information of the reference macroblock corresponding to the minimum difference value are used as difference result data between the current frame and the adjacent frame adjacent to the current frame.

[0006] In a possible implementation of the first aspect, performing difference calculation on each target macroblock and each reference macroblock in the plurality of reference macroblocks corresponding to each target macroblock includes any one of the following calculation methods: Absolute error calculation, mean absolute difference calculation, and mean square error calculation are performed on each target macroblock and each reference macroblock among a plurality of reference macroblocks corresponding to each target macroblock.

[0007] In a possible implementation of the first aspect, determining the encoding type matching the current frame based on difference data between the current frame and adjacent frames includes: When the minimum difference values corresponding to each target macroblock are greater than a preset difference threshold and the position distribution of the reference macroblocks corresponding to the minimum difference values meets a preset position distribution condition, determining the coding type that matches the current frame based on the interval between the current frame and the previous frame in the video stream whose coding type is intra-frame coding; When at least one of the minimum difference values corresponding to each target macroblock is less than a preset difference threshold, or the position distribution of the reference macroblocks corresponding to the minimum difference value does not meet the preset position distribution condition, it is determined that the encoding type matching the current frame is inter-frame encoding.

[0008] In a possible implementation of the first aspect, determining the coding type that matches the current frame based on an interval between the current frame and a previous frame in the video stream whose coding type is intra-frame coding includes: Determine whether the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding is greater than a preset frame interval; If the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding is greater than the preset frame interval, the encoding type matching the current frame is determined to be intra-frame encoding; otherwise, the encoding type matching the current frame is determined to be inter-frame encoding.

[0009] In a possible implementation of the first aspect, the multiple reference macroblocks include a central reference macroblock and surrounding reference macroblocks. The position of the central reference macroblock in the adjacent frame is the same as the position of the target macroblock in the current frame; The distance between the position of the surrounding reference macroblocks in the adjacent frame and the position of the target macroblock in the current frame is within a preset reference range.

[0010] In a possible implementation of the first aspect, the adjacent frames include a previous frame adjacent to the current frame and / or a subsequent frame adjacent to the current frame.

[0011] In a possible implementation of the first aspect, the method further includes: When it is determined that the current frame is the first frame of the video stream, it is determined that the encoding type of the current frame is intra-frame encoding.

[0012] In a second aspect, an embodiment of the present invention provides a video encoding apparatus, including: An acquisition module is used to obtain the video stream to be encoded; A first determining module is configured to determine difference result data between the current frame and adjacent frames adjacent to the current frame when the current frame to be encoded in the video stream is not the first frame of the video stream; A second determination module is used to determine the encoding type that matches the current frame based on the difference result data between the current frame and the adjacent frames; The encoding module is used to encode the current frame according to the encoding type.

[0013] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on an electronic device, enables the electronic device to execute the video encoding method in the above-mentioned first aspect and any possible implementation of the first aspect.

[0014] In a fourth aspect, an embodiment of the present invention provides a computer program product, which includes instructions. When the instructions are executed by one or more processors, they are used to implement the video encoding method in the first aspect and any possible implementation of the first aspect.

[0015] In a fifth aspect, an embodiment of the present invention provides an electronic device, including: memory for storing instructions, and One or more processors. When the instruction is executed by the one or more processors, the processors execute the video encoding method as described in the first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 According to some embodiments of the present invention, a macroblock in a video stream is shown; Figure 2 According to some embodiments of the present invention, multiple frame types in a video stream are shown; Figure 3According to some embodiments of the present invention, an application scenario of video transmission is shown; Figure 4 The video stream obtained by encoding in the related technical solution is shown; Figure 5 According to some embodiments of the present invention, a flowchart of a video encoding method is shown; Figure 6 According to some embodiments of the present invention, Figure 5 A flowchart further refined in step S12; Figure 7 According to some embodiments of the present invention, the position correspondence between a target macroblock in a current frame and a reference macroblock in an adjacent frame is shown; Figure 8 According to some embodiments of the present invention, an input-output relationship diagram of a video coding scheme is shown; Figure 9 According to some embodiments of the present invention, a simplified flow chart of a video encoding method is shown; Figure 10 According to some embodiments of the present invention, a detailed flow chart of a video encoding method is shown; Figure 11 According to some embodiments of the present invention, a structural block diagram of a video encoding device is shown; Figure 12 According to some embodiments of the present invention, a structural block diagram of an electronic device is shown. DETAILED DESCRIPTION

[0017] Illustrative embodiments of the present invention include, but are not limited to, a video encoding method, apparatus, medium, program product, and electronic device.

[0018] In order to facilitate understanding of the solution of the embodiment of the present invention, the following first Figure 1 and Figure 2 This section introduces relevant terms and concepts that may be involved in the embodiments of the present invention.

[0019] Macroblock: Macroblock (MB) is the basic processing unit in video coding. A video frame can be divided into multiple slices, each slice contains continuous macroblocks. In the H.264 standard (a video compression standard), the default size of a macroblock is 16×16 pixels. For example, in Figure 1 In the illustrated embodiment, a frame of image P0 in a video stream includes a slice P01 , which in turn includes a macroblock P011 , which can be further divided into multiple sub-blocks P0111 .

[0020] An I-frame is an independent frame that carries all the information for a single image. The encoding and decoding of an I-frame are independent of the preceding image and can be independently encoded and decoded. During decoding, a complete image can be reconstructed using only the I-frame data. Because an I-frame carries all the information for a single image, the data size of an I-frame is generally large.

[0021] P-frames are forward-predicted frames. Their encoding relies on the preceding P-frame or I-frame. P-frames compress and encode the difference between the image corresponding to the P-frame and the previous frame. P-frames cannot be decoded independently and rely on the previous P-frame or I-frame for decoding. To reconstruct the complete P-frame image, the previous frame and the difference information must be summed. Because P-frames encode difference information, the data size of P-frames is generally small.

[0022] B frame: It is a bidirectional prediction frame. The encoding and decoding of B depends on the previous frame and the next frame.

[0023] For example, in Figure 2 The video stream sequence shown includes multiple groups of pictures (GOPs). Each GOP starts with an I frame, and is followed by consecutive P frames and B frames.

[0024] The technical solutions of the embodiments of the present invention are described in further detail below with reference to the accompanying drawings.

[0025] In order to better understand the technical solution of the present invention, one application scenario of an embodiment of the present invention is first introduced below.

[0026] Figure 3 According to some embodiments of the present invention, a video transmission application scenario is shown. Figure 3 The application scenario shown is an application scenario of a live broadcast of a sports event, which includes a local terminal and a remote terminal.

[0027] Furthermore, it is understood that when video transmission is performed between a local terminal and a remote terminal, the local terminal and the remote terminal can be interconnected via an underlying network. The underlying network includes, but is not limited to, distributed soft buses, Wireless Fidelity (WIFI), Wireless Local Area Network (WLAN), Bluetooth (BT), Near Field Communication (NFC), and the like, which are not limited here.

[0028] In addition, it is understandable that from a security perspective, the local terminal and the remote terminal need to be mutually trusted electronic devices. For example, the local terminal and the remote terminal are subject to unified user authorization and authentication. For example, the unified authorization and authentication can be completed by the user through the personal identification number (PIN code) authentication of the subscriber identity module (SIM) card, face recognition authentication, fingerprint authentication, voiceprint authentication, etc., and there is no restriction here.

[0029] exist Figure 3 In the embodiment shown, the commentator uses a sports live broadcast application installed on a local terminal to provide live commentary of sports events. A remote terminal with better shooting performance can be used to shoot high-definition game videos of athletes. The remote terminal transmits the real-time captured video to the local terminal for display. The commentator uses the sports live broadcast application installed on the local terminal to provide synchronous commentary on the received high-definition game video.

[0030] Before video transmission, it is usually necessary to encode the video stream. The encoding method will affect the size of the encoded code stream, thereby affecting the transmission speed of the code stream during transmission. In the related technical solution, intra-frame encoding and inter-frame encoding are alternated according to a fixed frame interval. For example, Figure 4 In the illustrated embodiment, the first frame in a GOP uses intra-frame coding, while the remaining frames use inter-frame coding. However, in some cases, if the video stream remains largely unchanged for a long period of time, meaning that the differences between consecutive video frames are small, encoding at a fixed frame interval will still generate a large bitrate. Because intra-frame coding generates a large bitrate, some of these consecutive frames will still be intra-coded, resulting in a larger encoded video bitrate. Insufficient transmission bandwidth can cause video freezes, and a large bitrate occupies more storage space, placing higher hardware requirements and potentially increasing hardware costs.

[0031] for Figure 3 In the embodiment shown, the remote terminal encodes the real-time video stream of the sports event using the video encoding method provided by the embodiment of the present invention. First, based on the difference result data between the current frame and the adjacent frames in the video stream, the encoding type that matches the current frame is determined, and then the current frame is encoded according to the determined encoding type. The resulting sports event code stream is smaller, so that the remote terminal can quickly transmit the above-mentioned sports event code stream to the local terminal, making the sports event video viewed by the commentator through the local terminal smoother, thereby improving the user experience.

[0032] It should be understood that Figure 3The application scenario shown is only an exemplary application scenario for better illustrating the technical solution of the present invention. The technical solution of the present invention can also be applied to other application scenarios involving video transmission, including but not limited to telemedicine, distance education, video conferencing, and security monitoring.

[0033] Furthermore, it is understood that the remote terminals applicable to the embodiments of the present invention may be various electronic devices with display functions and video encoding / decoding functions. To meet different application requirements, the remote terminals applicable to the embodiments of the present invention may also be various electronic devices with shooting functions and decoding functions. Such remote terminals include, but are not limited to, smart surveillance cameras, as well as mobile phones, computers, drones, tablet computers, televisions, display devices, outdoor display screens, and vehicle-mounted terminals with shooting functions.

[0034] The following will be combined Figures 5 to 7 , the embodiments of the present invention are described in further detail.

[0035] First combine Figure 5 A video encoding method provided by the present invention is introduced in detail. Figure 5 According to some embodiments of the present invention, a video encoding method is provided for use in an electronic device. Figure 5 The present invention provides a video encoding method comprising the following steps: S11: Obtain the video stream to be encoded.

[0036] In some embodiments, the electronic device includes a camera, and the electronic device can shoot a shooting area through the camera to obtain a video stream to be encoded.

[0037] In some embodiments, the electronic device is a high-computing-power electronic device with video encoding and decoding functions, and the high-computing-power electronic device can use the video stream received from the low-computing-power electronic device as the video stream to be encoded.

[0038] In some embodiments, the electronic device may also use a video stream downloaded from a network as the video stream to be encoded.

[0039] In some embodiments, the electronic device may also use a video stream stored locally in the electronic device as the video stream to be encoded.

[0040] S12: When the current frame to be encoded in the video stream is not the first frame of the video stream, difference result data between the current frame and adjacent frames adjacent to the current frame is determined.

[0041] The adjacent frames include a previous frame adjacent to the current frame and / or a subsequent frame adjacent to the current frame. In some embodiments, the adjacent frame is the previous frame adjacent to the current frame; in some embodiments, the adjacent frame is the subsequent frame adjacent to the current frame; in some embodiments, the adjacent frames are both the previous frame and the subsequent frame adjacent to the current frame.

[0042] refer to Figure 6 In some embodiments, when the current frame to be encoded in the video stream is not the first frame of the video stream, the difference result data between the current frame and the adjacent frames adjacent to the current frame is determined by: S121: Determine multiple target macroblocks in the current frame and multiple reference macroblocks corresponding to each target macroblock in an adjacent frame, wherein the position of each target macroblock in the current frame is related to the position of the multiple reference macroblocks corresponding to each target macroblock in the adjacent frame.

[0043] In some embodiments, the target macroblock is obtained by segmenting the current frame according to a preset segmentation method. 16 pixels are divided into multiple 16 16-pixel target macroblock. For example, the current frame is converted to 32 32 pixels are divided into multiple 32 32-pixel target macroblock. For another example, the current frame is converted to 64 64 pixels are segmented to obtain multiple 64 The target macroblock of 64 pixels. The size of the target macroblock in the current frame can be determined according to actual conditions, and the present invention does not limit this.

[0044] In some embodiments, the above-mentioned multiple reference macroblocks include a central reference macroblock and surrounding reference macroblocks, and the position of the central reference macroblock in the adjacent frame is the same as the position of the target macroblock in the current frame. Assuming that the above-mentioned positions are represented by coordinates, the coordinates of the central reference macroblock in the adjacent frame are the same as the coordinates of the target macroblock in the current frame. For example, Figure 7 In the embodiment shown, it is assumed that the current frame and the adjacent frame are used as coordinate systems respectively, the target macroblock A1 in the current frame has coordinates (x0, y0), and the center reference macroblock B1 in the adjacent frame has coordinates (x0′, y0′), then (x0, y0) and (x0′, y0′) are the same.

[0045] In some embodiments, the distance between the position of the surrounding reference macroblocks in the adjacent frame and the position of the target macroblock in the current frame is within a preset reference range. For the same adjacent frame, the surrounding reference macroblocks in the adjacent frame use the central reference macroblock as the reference center. For example, in some embodiments, assuming that the above-mentioned preset reference range is 2 2-pixel area, then in the adjacent frame, 9 macroblocks with coordinates including (x0′-1, y0′), (x0′, y0′-1), (x0′-2, y0′), (x0′+1, y0), (x0′+2, y0'), (x0′, y0′-2), (x0′, y0′+1) and (x0', y0′+2) can be selected as the surrounding reference macroblocks of the above-mentioned target macroblock A1.

[0046] It should be understood that the target macroblocks in the current frame can be selected as needed. All macroblocks in the current frame can be selected for subsequent difference calculation, or some macroblocks in the current frame can be selected for subsequent difference calculation.

[0047] It should be understood that the above-mentioned preset reference range can be set as needed, and the position and number of the above-mentioned reference macroblocks can also be adjusted accordingly as needed.

[0048] Furthermore, it should be understood that when the aforementioned preset reference range is larger, more reference macroblocks are selected from adjacent frames, and thus, when performing subsequent difference calculations based on the target macroblock and the reference macroblock, the resulting difference value is more accurate. Consequently, the encoding type matching the current frame, determined based on the difference value, is more accurate, resulting in a more optimized bitstream generated by the encoding method provided by the embodiment of the present invention, effectively reducing the bitstream size while ensuring the encoding quality of the current frame.

[0049] S122: performing difference calculation on each target macroblock and each reference macroblock in a plurality of reference macroblocks corresponding to each target macroblock, to obtain a plurality of difference values corresponding to each target macroblock.

[0050] By calculating the difference between each target macroblock in the current frame and the reference macroblock corresponding to each target macroblock in the adjacent frame, the difference value obtained can accurately reflect the difference between the current frame and the adjacent frame. Then, based on the size of the difference, the encoding type of the current frame can be flexibly selected, while minimizing the bitstream size while hardly affecting the encoding quality. This is especially effective for static videos or videos with scene changes, effectively reducing the bitstream size and improving the user experience.

[0051] In some embodiments, performing difference calculation between each target macroblock and each reference macroblock in a plurality of reference macroblocks corresponding to each target macroblock includes but is not limited to any one of the following calculation methods: The sum of absolute differences (SAD), mean absolute differences (MAD), and mean squared errors (MSE) are calculated for each target macroblock and each of the reference macroblocks corresponding to the target macroblock.

[0052] S123: Determine the minimum difference value corresponding to each target macroblock and the position information of the reference macroblock corresponding to each minimum difference value.

[0053] In some embodiments, the multiple difference values corresponding to the target macroblocks obtained by the above calculations may be compared to obtain the minimum difference value corresponding to each target macroblock.

[0054] In some embodiments, when performing difference calculations on each target macroblock and each reference macroblock among multiple reference macroblocks corresponding to each target macroblock, position information of each reference macroblock corresponding to each target macroblock may be stored. After obtaining the minimum difference value corresponding to each target macroblock, position information of the reference macroblock corresponding to each minimum difference value may be found from the pre-stored position information of each reference macroblock corresponding to each target macroblock.

[0055] S124: Using the minimum difference value corresponding to each target macroblock and the position information of the reference macroblock corresponding to the minimum difference value as difference result data between the current frame and the adjacent frame adjacent to the current frame.

[0056] In other words, the difference result data mentioned above not only includes the minimum difference value corresponding to each target macroblock, but also includes the position information of the reference macroblock corresponding to the minimum difference value.

[0057] S13: Determine the encoding type that matches the current frame based on the difference result data between the current frame and the adjacent frames.

[0058] In some embodiments, the above-mentioned determining the encoding type matching the current frame based on the difference result data between the current frame and the adjacent frames further includes: When the minimum difference values corresponding to each target macroblock are greater than a preset difference threshold, and the position distribution of the reference macroblocks corresponding to the minimum difference values meets the preset position distribution conditions, the encoding type that matches the current frame is determined based on the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding.

[0059] When at least one of the minimum difference values corresponding to each target macroblock is less than a preset difference threshold, or the position distribution of the reference macroblocks corresponding to the minimum difference value does not meet the preset position distribution condition, it is determined that the encoding type matching the current frame is inter-frame encoding.

[0060] In some embodiments, the above-mentioned determination of the encoding type matching the current frame based on the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding includes: Determine whether the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding is greater than a preset frame interval; If the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding is greater than the preset frame interval, the encoding type matching the current frame is determined to be intra-frame encoding; otherwise, the encoding type matching the current frame is determined to be inter-frame encoding.

[0061] The preset difference threshold can be set as needed. For example, in some embodiments, a SAD calculation is performed on each target macroblock and each reference macroblock in the plurality of reference macroblocks corresponding to each target macroblock. The minimum difference value obtained with respect to each target macroblock is also a SAD value. The preset difference threshold can be set to a SAD value of 100,000,000 as needed. The preset difference threshold can be a pre-set fixed value or dynamically calculated based on the current actual bitstream.

[0062] In some embodiments, the preset position distribution condition can also be set as needed, including but not limited to: the percentage of the distance between the position of the reference macroblock corresponding to the minimum difference value and the position of the center reference macroblock exceeding the preset distance is less than a preset percentage. The distance between the position of the reference macroblock corresponding to the minimum difference value and the position of the center reference macroblock can be represented by pixels.

[0063] For example, in some embodiments, the current frame and the reference frame are 1280x720 pixels, and when the proportion of macroblocks whose coordinates of the reference macroblock corresponding to the minimum difference value differ from the coordinates of the above-mentioned central reference macroblock by more than 1 pixel is less than 15%, and at least one of the minimum difference values corresponding to each target macroblock is less than 100000000 (SDA value), it is determined that the encoding type of the current frame is inter-frame encoding.

[0064] The above-mentioned preset frame interval may also be set according to actual needs, and the present invention does not limit this.

[0065] S14: Encode the current frame according to the encoding type.

[0066] Since the encoding type of the current frame in the above embodiment of the present invention is determined based on the difference result data between the current frame and the adjacent frames, when encoding the current frame, the encoding type of the current frame determined according to the embodiment of the present invention is more consistent with the current frame. By flexibly matching the corresponding encoding type for each frame image in the video stream, the code stream size can be effectively reduced while almost not affecting the encoding quality, thereby improving the user experience.

[0067] In some embodiments, the above method further includes: when it is determined that the current frame is the first frame of the video stream, determining that the encoding type of the current frame is intra-frame encoding.

[0068] It should be understood that a video stream is composed of consecutive video frames. The first frame in a video stream must be decoded independently, without relying on other frames. Therefore, the encoding method corresponding to the first frame in a video stream is usually intra-frame coding. For example, in live broadcast scenarios, the first frame of the live video stream is intra-frame coded to ensure fast decoding when users access it from any time point. In another example, in smart security scenarios, the first frame of the surveillance video stream is intra-frame coded to support real-time video preview.

[0069] It can be understood that the execution order of the above steps S11 to S14 and the execution order of the above steps S121 to S124 are only for reference. In other embodiments, other execution orders may be adopted, and some steps may be split or merged, which is not limited here.

[0070] Through the introduction of the above embodiments, the above Figures 5 to 7 The technical solution described in the embodiment is used Figure 8 To summarize: the encoding type of the current frame to be encoded in the unencoded video stream is dynamically monitored to determine the encoding type corresponding to each of the n video frames in the video stream. Then, each frame is encoded according to the encoding type corresponding to each frame. The resulting bitstream is small and the encoding quality is high.

[0071] The following will be combined Figure 9 The encoding process of video encoding by an electronic device including an encoder is described in detail.

[0072] refer to Figure 9 The present invention provides a video encoding method comprising the following steps: S21: Determine whether it is the first frame. If so, it indicates that the current frame is the first frame of the video stream, and the process goes to step S22. Otherwise, it indicates that the current frame is not the first frame of the video stream, and the process goes to step S23.

[0073] It should be understood that the determination here is based on each video frame in the unencoded video stream. Before encoding each video frame in the video stream, it is necessary to determine whether each video frame is the first frame of the video stream, that is, to determine whether each video frame is the first frame image of the video stream. If it is the first frame image, the process proceeds to step S22 to cache the first frame image. If it is not the first frame image, the process proceeds to step S23 to perform adaptive coding type detection.

[0074] S22: Cache.

[0075] In some embodiments, the first frame image can be cached. In other embodiments, the encoding type corresponding to each frame image detected according to the technical solution of the present invention can be cached. By caching the first frame image and the encoding type corresponding to each frame image, the encoder's direct access to the first frame image and the encoding type data corresponding to each frame image can be reduced, thereby improving encoding performance and encoding efficiency.

[0076] S23: Adaptive coding type detection.

[0077] The specific method of adaptive coding type detection here can be found in the above Figure 5 The description of steps S12 and S13 in the illustrated embodiment will not be repeated here. When encoding the unencoded video stream, adaptive encoding type detection can be used to flexibly match the appropriate encoding type for each frame image in the video stream, thereby effectively reducing the bitstream size and improving the user experience while having little impact on the encoding quality.

[0078] S24: Coding.

[0079] In some embodiments, the first frame image stored in the buffer is encoded according to the intra-frame coding type.

[0080] In some embodiments, each frame image is encoded separately according to the encoding type of each frame image stored in the cache corresponding to the non-first frame image.

[0081] By adopting the technical solution of the present invention, each frame image in the video stream is flexibly matched with a suitable encoding type, and then each frame image is encoded according to the encoding type corresponding to each frame image. The resulting code stream is smaller, the transmission delay is small, the encoding quality is higher, and the user experience is effectively improved.

[0082] S25: Determine whether it is the last frame. If yes, it indicates that the current frame is the last frame of the video stream, and the current process ends; otherwise, it indicates that the current frame is not the last frame of the video stream, and the process goes to step S23.

[0083] It can be understood that the execution order of the above steps S21 to S25 is only an illustration. In other embodiments, other execution orders may be adopted, and some steps may be split or combined, which is not limited here.

[0084] The following will be combined Figure 10 The encoding process of video encoding by an electronic device including an encoder is further introduced in detail.

[0085] refer to Figure 10 The present invention provides a video encoding method comprising the following steps: S31: Get the position of the target macroblock in the current frame.

[0086] In some embodiments, assuming that the current frame is a plane rectangular coordinate system, the position of the target macroblock in the current frame can be represented by the coordinates of the target macroblock in the coordinate system.

[0087] The current frame and the location of the current frame target macroblock are described in the above Figure 5 Step S12 and Figure 6 The text portion of step S121 has been described in detail and will not be repeated here.

[0088] S32: Obtain positions of all reference macroblocks of the target macroblock.

[0089] In some embodiments, assuming that the current frame and the adjacent frame are respectively plane rectangular coordinate systems, the position of the target macroblock in the current frame can be represented by the coordinates of the target macroblock in the coordinate system corresponding to the current frame. The position of the reference macroblock in the adjacent frame can be represented by the coordinates of the reference macroblock in the coordinate system corresponding to the adjacent frame.

[0090] S33: Calculate the difference between the target macroblock and a reference macroblock.

[0091] In some embodiments, calculating the difference between a target macroblock and a reference macroblock includes, but is not limited to, any one of the following calculation methods: performing absolute error and SAD calculations, MAD calculations, and MSE calculations on the difference between the target macroblock and a reference macroblock. By performing the difference calculations between each target macroblock in the current frame and the reference macroblocks corresponding to each target macroblock in the adjacent frame, the obtained difference value can accurately reflect the difference between the current frame and the adjacent frame. Based on the magnitude of the difference, the encoding type of the current frame can be flexibly selected, thereby minimizing the bitstream size while barely affecting the encoding quality. This can effectively reduce the bitstream size and improve the user experience, especially for static videos or videos that include scene changes.

[0092] S34: Determine whether the reference macroblock is the macroblock with the smallest difference from the target macroblock. If so, it indicates that the reference macroblock is the macroblock with the smallest difference from the target macroblock, and the process proceeds to step S35. Otherwise, it indicates that the reference macroblock is not the macroblock with the smallest difference from the target macroblock, and the process proceeds to step S36.

[0093] S35: Update the coordinates and difference value of the minimum difference value reference macroblock.

[0094] By updating the coordinates and difference values of the minimum difference reference macroblock in real time, the coordinates and difference value data of the minimum difference reference macroblock can be effectively recorded, thereby reducing the time for subsequently determining the encoding type matching the current frame based on the coordinates and difference value data of the minimum difference reference macroblock updated in real time.

[0095] S36: Determine whether the current reference macroblock is the last reference macroblock. If so, it indicates that the current reference macroblock is the last reference macroblock in the adjacent frame of the current frame, and the process proceeds to step S37. Otherwise, it indicates that the current reference macroblock is not the last reference macroblock in the adjacent frame of the current frame, and the process proceeds to step S33.

[0096] S37: Determine whether the current target macroblock is the last target macroblock of the current frame. If so, it indicates that the current target macroblock is the last target macroblock of the current frame, and the process proceeds to step S38; otherwise, it indicates that the current target macroblock is not the last target macroblock of the current frame, and the process proceeds to step S31.

[0097] S38: Counting the distribution of minimum difference macroblocks and minimum difference values of the current frame.

[0098] It should be understood that after completing the difference calculation for all target macroblocks in the current frame and the reference macroblocks corresponding to each target macroblock, the minimum difference value corresponding to each target macroblock and the position information of the reference macroblock corresponding to the minimum difference value can be obtained, thereby obtaining the distribution of the minimum difference macroblock and the minimum difference value data of the current frame. By statistically analyzing the distribution and minimum difference value of the minimum difference macroblock of the current frame, the encoding type corresponding to the current frame can be further determined based on the distribution and minimum difference value of the minimum difference macroblock of the current frame.

[0099] S39: Determine whether the distribution and minimum difference value of the minimum difference reference macroblock of the current frame are outside the range. If so, it indicates that the distribution and difference value of the minimum difference reference macroblock of the current frame are outside the range, and the process proceeds to step S40. Otherwise, it indicates that the distribution and difference value of the minimum difference reference macroblock of the current frame are within the range, and the process proceeds to step S42.

[0100] Regarding how to judge whether the distribution of the minimum difference reference macroblock of the current frame and the minimum difference value are out of range, in the above Figure 5 The text portion of step S13 has been described in detail and will not be repeated here.

[0101] S40: Determine if the interval between the current frame and the previous intra-coded frame is greater than MIN_INTRA_PERIOD. If so, it indicates that the interval between the current frame and the previous intra-coded frame is greater than MIN_INTRA_PERIOD, and the process proceeds to step S41 to perform intra-frame coding on the current frame. Otherwise, it indicates that the interval between the current frame and the previous intra-coded frame is greater than MIN_INTRA_PERIOD, and the process proceeds to step S42 to perform inter-frame coding on the current frame.

[0102] In other words, it is determined whether the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding is greater than the preset frame interval; if the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding is greater than the preset frame interval, it is determined that the encoding type matching the current frame is intra-frame encoding; otherwise, it is determined that the encoding type matching the current frame is inter-frame encoding. The specific judgment method is described in the above about Figure 5 The text portion in step S13 has been described in detail and will not be repeated here.

[0103] MIN_INTRA_PERIOD is an example of the aforementioned preset frame interval. In other embodiments, the preset frame interval can also be expressed as a minimum I-frame interval. The minimum I-frame interval can be a fixed value or dynamically selected, and the present invention is not limited to this. It should be understood that in actual applications, the specific representation of the preset frame interval can be selected according to the developer's preferences.

[0104] S41: Intra-frame coding.

[0105] Specifically, when it is determined that the encoding type of the current frame is intra-frame encoding, intra-frame encoding is performed on the current frame.

[0106] S42: Inter-frame coding.

[0107] Specifically, when the current frame's coding type is determined to be inter-frame coding, the current frame is inter-frame encoded. Inter-frame coding uses less data and results in a smaller bitrate. This can avoid scenarios where, in a video stream where the image remains unchanged for a long period of time, multiple consecutive video frames may still be intra-coded at a fixed frame interval, resulting in a larger video bitrate, causing video freezes and increased hardware costs.

[0108] The present invention determines the encoding type corresponding to each frame image in the video stream based on the difference result data between the current frame and the adjacent frames, so that the determined encoding type of the current frame is more consistent with the current frame. By flexibly matching the corresponding encoding type for each frame image in the video stream, the code stream size can be effectively reduced while hardly affecting the encoding quality, thereby improving the user experience.

[0109] It can be understood that the execution order of the above steps S31 to S42 is only an illustration. In other embodiments, other execution orders may be adopted, and some steps may be split or combined, which is not limited here.

[0110] In addition, the embodiment of the present invention also provides a video encoding device, see Figure 11 The video encoding device provided by the present invention includes: The acquisition module 501 is used to obtain the video stream to be encoded; A first determining module 502 is configured to determine difference result data between the current frame and adjacent frames adjacent to the current frame when the current frame to be encoded in the video stream is not the first frame of the video stream; A second determination module 503 is configured to determine a coding type that matches the current frame based on difference data between the current frame and adjacent frames; The encoding module 504 is configured to encode the current frame according to an encoding type.

[0111] Regarding the apparatus in the above embodiment, the specific manner in which the processor performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0112] An embodiment of the present invention further provides an electronic device 600, such as Figure 12 As shown, the electronic device 600 includes a memory 601 and a processor 602. The memory 601 is used to store computer programs executable by the processor 602; the processor 602 is used to execute the computer programs in the memory 601 to implement the video encoding method provided by any of the above embodiments.

[0113] Figure 12 The electronic device 600 shown further includes a communication interface 603. The processor 602, the memory 601 and the communication interface 603 are connected via a communication bus and communicate with each other.

[0114] The processor 602 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above-mentioned programs.

[0115] The electronic device 600 further includes an encoder (not shown). The encoder can be a specific hardware module or a functional module in the processor 602. The encoder can encode the video stream using the video encoding method provided in the embodiment of the present invention.

[0116] The communication interface 603 is used to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc.

[0117] The memory 601 may be a read-only memory (ROM) or other static storage device that can store static information and instructions, a random access memory (RAM) or other dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be independent and connected to the processor via a bus. The memory may also be integrated with the processor.

[0118] An embodiment of the present invention further provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on an electronic device, the electronic device executes the video encoding method provided in any of the above embodiments.

[0119] An embodiment of the present invention further provides a computer program product, which includes instructions. When the instructions are executed by one or more processors, they are used to implement the video encoding method provided in any of the above embodiments.

[0120] The various embodiments of the mechanisms disclosed in the present invention can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present invention can be implemented as a computer program or program code executed on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0121] It should be noted that the various units / modules mentioned in the various device embodiments of the present invention are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by the present invention. In addition, in order to highlight the innovative part of the present invention, the above-mentioned device embodiments of the present invention do not introduce units / modules that are not closely related to solving the technical problems raised by the present invention. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0122] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0123] While the present invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims

1. A video encoding method, characterized in that: For an electronic device, the method comprises: Get the video stream to be encoded; When a current frame to be encoded in the video stream is not the first frame of the video stream, determining difference result data between the current frame and adjacent frames adjacent to the current frame; Determining, based on difference result data between the current frame and the adjacent frames, a coding type that matches the current frame; The current frame is encoded according to the encoding type.

2. The video encoding method according to claim 1, wherein: When the current frame to be encoded in the video stream is not the first frame of the video stream, difference result data between the current frame and adjacent frames adjacent to the current frame is determined in the following manner: Determining a plurality of target macroblocks in the current frame and a plurality of reference macroblocks corresponding to each target macroblock in the adjacent frames, wherein a position of each target macroblock in the current frame is correlated with a position of the plurality of reference macroblocks corresponding to each target macroblock in the adjacent frames; performing difference calculations on each target macroblock and each reference macroblock in the plurality of reference macroblocks corresponding to each target macroblock, respectively, to obtain a plurality of difference values corresponding to each target macroblock; Determining minimum difference values corresponding to the target macroblocks and position information of reference macroblocks corresponding to the minimum difference values; The minimum difference value corresponding to each target macroblock and the position information of the reference macroblock corresponding to the minimum difference value are used as difference result data between the current frame and an adjacent frame adjacent to the current frame.

3. The video encoding method according to claim 2, wherein: Calculating the difference between each target macroblock and each reference macroblock in the plurality of reference macroblocks corresponding to each target macroblock includes any one of the following calculation methods: Absolute error calculation, mean absolute difference calculation, and mean square error calculation are performed on each target macroblock and each reference macroblock in the plurality of reference macroblocks corresponding to each target macroblock.

4. The video encoding method according to claim 2, wherein: The determining, based on the difference result data between the current frame and the adjacent frame, a coding type matching the current frame includes: When the minimum difference value corresponding to each of the target macroblocks is greater than a preset difference threshold and the position distribution of the reference macroblocks corresponding to the minimum difference value meets a preset position distribution condition, determining a coding type that matches the current frame based on an interval between the current frame and a previous frame in the video stream whose coding type is intra-frame coding; When at least one of the minimum difference values corresponding to each of the target macroblocks is less than a preset difference threshold, or the position distribution of the reference macroblocks corresponding to the minimum difference value does not meet the preset position distribution condition, it is determined that the encoding type matching the current frame is inter-frame encoding.

5. The video encoding method according to claim 4, wherein: Determining a coding type that matches the current frame according to an interval between the current frame and a previous frame in the video stream whose coding type is intra-frame coding, including: Determine whether an interval between the current frame and a previous frame in the video stream whose encoding type is intra-frame encoding is greater than a preset frame interval; When the interval between the current frame and the previous frame in the video stream whose encoding type is intra-frame encoding is greater than the preset frame interval, it is determined that the encoding type matching the current frame is intra-frame encoding; otherwise, it is determined that the encoding type matching the current frame is inter-frame encoding.

6. The video encoding method according to claim 2, wherein: The plurality of reference macroblocks include a central reference macroblock and surrounding reference macroblocks; The position of the central reference macroblock in the adjacent frame is the same as the position of the target macroblock in the current frame; A distance between a position of the surrounding reference macroblocks in the adjacent frame and a position of the target macroblock in the current frame is within a preset reference range.

7. The video encoding method according to claim 1, wherein: The adjacent frames include a previous frame adjacent to the current frame and / or a subsequent frame adjacent to the current frame.

8. The video encoding method according to claim 1, wherein: The method further comprises: When it is determined that the current frame is the first frame of the video stream, it is determined that the encoding type of the current frame is intra-frame encoding.

9. A video encoding device, characterized in that: include: An acquisition module is used to obtain the video stream to be encoded; A first determining module is configured to determine difference result data between the current frame and adjacent frames adjacent to the current frame when the current frame to be encoded in the video stream is not the first frame of the video stream; A second determining module is configured to determine a coding type that matches the current frame based on difference result data between the current frame and the adjacent frames; An encoding module is configured to encode the current frame according to the encoding type.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the video encoding method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The computer program product comprises instructions for implementing the video encoding method according to any one of claims 1 to 8 when the instructions are executed by one or more processors.

12. An electronic device, characterized in that: include: memory for storing instructions, and One or more processors, when the instructions are executed by the one or more processors, the processors perform the video encoding method according to any one of claims 1 to 8.

Images