A video processing method, device, computer equipment and storage medium

Abstract

Embodiments of the present application provide a video processing method, device, computer equipment and storage medium. The video processing method comprises: obtaining scene characteristic information of a service scene to which a to-be-encoded video belongs; performing scene type analysis on the service scene according to the scene characteristic information to determine a scene type of the service scene; configuring interpolation information matched with the scene type for the to-be-encoded video based on the scene type; the interpolation information comprises motion estimation interpolation information used in a motion estimation process of the to-be-encoded video and motion compensation interpolation information used in a motion compensation process of the to-be-encoded video; the motion estimation interpolation information and the motion compensation interpolation information are decoupled; and encoding the to-be-encoded video according to the interpolation information. By using the embodiments of the present application, interpolation information matched with the scene type of a video can be flexibly configured for the video, and the video encoding effect is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to the field of video coding technology, specifically to a video processing method, a video processing apparatus, a computer device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of computer technology, video encoding technology has been widely applied in video-related business scenarios. For example, it encodes session videos in video conferencing scenarios; it encodes video-on-demand videos in video-on-demand scenarios; and it encodes game videos in gaming scenarios, and so on. Video encoding technology refers to the technique of compressing video to reduce its size while meeting certain video quality requirements. Through video encoding technology, storage space can be effectively saved and transmission efficiency improved.

[0003] Fractional pixel interpolation is frequently involved in video encoding. Currently, the interpolation information used in fractional pixel interpolation is usually fixed, and this fixed interpolation information has a significant impact on the video encoding effect. Specifically, for example, using highly complex interpolation information when real-time or near-real-time encoding will severely affect the video encoding speed; conversely, using less complex interpolation information in offline transcoding, where encoding speed is not critical, will severely affect the video compression effect. Therefore, determining a more suitable interpolation information has become a current research hotspot. Summary of the Invention

[0004] This application provides a video processing method, apparatus, computer equipment, and storage medium, which can flexibly configure interpolation information that matches the video scene type to improve video encoding performance.

[0005] On one hand, embodiments of this application provide a video processing method, which includes:

[0006] Obtain scene feature information of the business scenario to which the video to be encoded belongs;

[0007] Analyze the scenario type of the business scenario based on the scenario feature information to determine the scenario type of the business scenario;

[0008] Based on scene type, interpolation information matching the scene type is configured for the video to be encoded; the interpolation information includes motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled.

[0009] The video to be encoded is encoded based on the interpolation information.

[0010] Accordingly, embodiments of this application provide a video processing apparatus, which includes:

[0011] The acquisition unit is used to acquire scene feature information of the business scenario to which the video to be encoded belongs;

[0012] The processing unit is used to perform scenario type analysis on the business scenario based on scenario feature information to determine the scenario type of the business scenario;

[0013] The processing unit is also used to configure interpolation information that matches the scene type for the video to be encoded based on the scene type; the interpolation information includes motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled.

[0014] The processing unit is also used to encode the video to be encoded based on the interpolation information.

[0015] In one implementation, the processing unit, used to perform scenario type analysis on the business scenario based on scenario feature information, specifically executes the following steps when determining the scenario type of the business scenario:

[0016] In the scene type matching relationship, target type indication information that matches the scene feature information is determined; the scene type matching relationship includes multiple type indication information and the scene type corresponding to each type indication information.

[0017] The scenario type corresponding to the target type indication information is determined as the scenario type of the business scenario.

[0018] In one implementation, the scene feature information includes at least one of the following: video resolution information, video frame rate information, video bit rate information, video playback effect information, and video encoding speed information;

[0019] The acquisition unit, used to acquire scene feature information of the business scenario to which the video to be encoded belongs, specifically performs the following steps:

[0020] Display scene feature configuration interface;

[0021] Retrieve scene feature information configured in the scene feature configuration interface.

[0022] In one implementation, each image frame in the video to be encoded includes N image component types, where N is a positive integer.

[0023] The processing unit, when configuring interpolation information that matches the scene type for the video to be encoded, specifically performs the following steps:

[0024] In response to the image component decoupling configuration request, based on the scene type, interpolation information matching the scene type is configured for each image component type of the video to be encoded.

[0025] In one implementation, motion estimation interpolation information is used for pixel interpolation during motion estimation, and motion compensation interpolation information is used for pixel interpolation during motion compensation; pixel interpolation has M pixel interpolation directions, where M is a positive integer;

[0026] The processing unit, when configuring interpolation information that matches the scene type for the video to be encoded, specifically performs the following steps:

[0027] In response to the interpolation direction decoupling configuration request, based on the scene type, interpolation information matching the scene type is configured for the interpolation direction of each pixel in the video to be encoded.

[0028] In one implementation, when the processing unit configures interpolation information matching the scene type for the video to be encoded based on the scene type, it specifically performs the following steps:

[0029] Retrieve the interpolation information configuration list; the interpolation information configuration list includes multiple reference types and the interpolation information corresponding to each reference type;

[0030] Based on the scene type, determine the reference type that matches the scene type from among the multiple reference types included in the interpolation information configuration list;

[0031] The interpolation information corresponding to the matching reference type is determined as the interpolation information that matches the scene type.

[0032] In one implementation, the processing unit, when encoding the video to be encoded based on the interpolation information, specifically performs the following steps:

[0033] Obtain the image block to be encoded from the image frame to be encoded contained in the video to be encoded;

[0034] Motion compensation is performed on the reference frame of the video frame to be encoded based on motion compensation interpolation information to reconstruct the reference frame of the video frame to be encoded.

[0035] Motion estimation is performed on the image block to be encoded based on motion estimation interpolation information, and the matching block of the image block to be encoded is determined in the reference frame of the video frame to be encoded.

[0036] The image block to be encoded is encoded based on the matching block.

[0037] In one implementation, the processing unit, when encoding the image block to be encoded based on the matching block, specifically performs the following steps:

[0038] Temporal filtering is performed on the image block to be encoded based on the matching block;

[0039] Encode the image block to be encoded after time-domain filtering.

[0040] In one implementation, the processing unit is used to perform motion estimation on the image block to be encoded based on motion estimation interpolation information. When determining the matching block of the image block to be encoded in the reference frame of the video frame to be encoded, it is specifically used to perform the following steps:

[0041] Motion estimation is performed on the image block to be encoded based on motion estimation interpolation information. The matching block of the image block to be encoded and the motion vector between the image block to be encoded and the matching block are determined in the reference frame of the video frame to be encoded.

[0042] A processing unit, configured to encode the image block to be encoded based on the matching block, includes:

[0043] The residual signal is determined based on the difference between the image block to be encoded and the matching block;

[0044] The image block to be encoded is encoded based on the residual signal and motion vector.

[0045] In one implementation, the interpolation information includes at least one of the following: a fractional pixel interpolation algorithm, a number of taps, and tap coefficients corresponding to the number of taps.

[0046] Accordingly, embodiments of this application provide a computer device, which includes:

[0047] A processor is a tool for implementing computer programs.

[0048] A computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the video processing method described above.

[0049] Accordingly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when read and executed by a processor of a computer device, causes the computer device to perform the aforementioned video processing method.

[0050] Accordingly, embodiments of this application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned video processing method.

[0051] In this embodiment, after determining the scene type of the business scenario to which the video to be encoded belongs, interpolation information matching the scene type can be configured for the video to be encoded, thereby enabling encoding of the video based on the determined interpolation information. Furthermore, the interpolation information includes motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled. In other words, this embodiment can flexibly configure interpolation information matching the scene type of the business scenario to which the video to be encoded belongs, and the configured interpolation information is decoupled from the motion estimation process and the motion compensation process, which can improve the video encoding effect. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram illustrating the principle of fractional pixel interpolation provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the architecture of a video processing system provided in an embodiment of this application;

[0055] Figure 3 This is a flowchart illustrating a video processing method provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of an encoder configuration interface provided in an embodiment of this application;

[0057] Figure 5 This is a flowchart illustrating another video processing method provided in an embodiment of this application;

[0058] Figure 6 This is a schematic diagram of a scene feature configuration interface provided in an embodiment of this application;

[0059] Figure 7 This is a schematic diagram of the structure of a video processing device provided in an embodiment of this application;

[0060] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0062] Video is a sequence of consecutive image frames. Due to the persistence of vision, when the sequence of image frames is played at a certain rate, we see a video with continuous motion. Generally speaking, the similarity between consecutive image frames in a video is relatively high. There is a lot of redundant information within each image frame and between consecutive image frames. Therefore, before storing or transmitting video, video encoding technology is often used to compress the video to reduce its size, thereby effectively saving storage space and improving transmission efficiency.

[0063] Fractional pixel interpolation is frequently involved in video encoding processes. For example, motion estimation, motion compensation, and temporal filtering in video encoding can all involve fractional pixel interpolation.

[0064] The basic idea of ​​motion estimation (ME) is to divide each image frame in an image frame sequence into many non-overlapping macroblocks, assuming that all pixels within a macroblock have the same displacement. Then, for each macroblock, within a given specific search range to a reference frame, the most similar block in the reference frame is found according to a certain matching criterion. This most similar block is called the matching block, and the relative displacement between the matching block and the current macroblock is called the motion vector (MV). During video compression, only the motion vector and the residual signal between the current macroblock and the matching block need to be saved to completely recover the current block. In other words, motion estimation is usually used to determine the matching block of a macroblock and the motion vector between the macroblock and the matching block during video encoding. Then, by encoding the residual signal between the macroblock and the matching block and the motion vector, video compression encoding can be achieved.

[0065] Motion compensation (MC) is a method for describing the differences between adjacent frames (adjacent here can mean adjacent in terms of coding relationship, but not necessarily adjacent in terms of playback order). Specifically, it describes how each block of the reference frame moves to a certain position in the current frame. Motion compensation is often used to predict the prediction block of the current macroblock based on motion vectors, and to reconstruct the current macroblock based on the prediction block and the residual signal of the current macroblock. Motion compensation is often used to reconstruct the reference frame during video coding.

[0066] Motion Compensation Temporal Filtering (MCTF) can be used to remove noise that is detrimental to compression in videos by using motion-compensated weighted filtering between adjacent frames (i.e., the current image frame and the reference frame). More specifically, the principle of MCTF is inter-frame averaging, which means taking a weighted average of the current image frame and the temporally adjacent reference frame to smooth out noise. Furthermore, MCTF takes a weighted average of the current macroblock in the current image frame and the matching block of the current macroblock in the reference frame. The process of finding the matching block of the current macroblock requires motion estimation and motion compensation.

[0067] Fractional pixel interpolation is a concept proposed to improve the accuracy of motion estimation and motion compensation. It refers to the process of interpolating pixel values ​​at fractional positions from pixel values ​​at integer positions; where integer pixels are the original pixels in the image, and fractional pixels are the pixels obtained by interpolation from the original pixels. For example... Figure 1 As shown, Figure 1 The uppercase letters AU represent integer pixel locations in an image. Figure 1 The lowercase letters j, h, b, k, u, s, t, v, w, m, r, x, and y in the text represent pixels at half-fraction positions. Figure 1 The lowercase letters a, c, d, e, f, g, i, l, n, o, p, and q represent the pixel points at the quarter-fraction positions.

[0068] The interpolation information used in fractional pixel interpolation may include, but is not limited to, at least one of the following: fractional pixel interpolation algorithm, number of taps, and tap coefficients; wherein, the fractional pixel interpolation algorithm can be used to determine the specific interpolation method, for example, the fractional pixel interpolation algorithm can determine which integer position pixels can be used to interpolate to obtain fractional position pixels, and how the pixel values ​​of these integer position pixels are calculated to obtain the pixel values ​​of fractional position pixels; the number of taps can be used to determine the number of integer position pixels to be interpolated, for example, the number of taps can be used to determine how many integer position pixels are used to interpolate to obtain fractional position pixels; the tap coefficients can be used to determine the weight of each integer position pixel in the interpolation.

[0069] by Figure 1 For example, the pixel value of pixel b at a fractional position can be obtained by interpolating the pixel values ​​of six integer positions: E, F, G, H, I, and K. That is, the number of taps is 6; the tap coefficients are (1, -5, 20, 20, -5, 1). In other words, the weight of interpolating integer position pixel E is 1, the weight of interpolating integer position pixel F is -5, the weight of interpolating integer position pixel G is 20, the weight of interpolating integer position pixel H is 20, the weight of interpolating integer position pixel I is -5, and the weight of interpolating integer position pixel K is 1. The fractional pixel interpolation algorithm can be b = (E - 5F + 20G + 20H - 5I + K + 16) / 32.

[0070] Typically, fractional pixel interpolation algorithms, the number of taps, and the tap coefficients can have a one-to-one correspondence. This one-to-one correspondence means that once the number of taps is determined, the corresponding tap coefficients and fractional pixel interpolation algorithm can be determined. For example, video coding standards (such as H.266 / VVC (Multi-Functional Video Coding Standard) or AV1 (a video coding standard), etc.) specify the tap coefficients and fractional pixel interpolation algorithms corresponding to each number of taps.

[0071] Based on the above introduction to motion estimation, motion compensation, temporal filtering for motion compensation, and fractional-pixel interpolation algorithms, this application proposes a video processing method that flexibly configures the interpolation information used for fractional-pixel interpolation based on the scene type of the video's business scenario. Furthermore, in the video processing method proposed in this application, the interpolation information used in the motion estimation process for fractional-pixel interpolation can be decoupled from the interpolation information used in the motion compensation process. This decoupling means that the interpolation information used in both motion estimation and motion compensation processes can be configured based on the scene type of the video's business scenario. In the video processing method proposed in this application, the configured interpolation information is adapted to the video scene type, which can effectively improve the video encoding effect.

[0072] The following is combined with Figure 2 A video processing system suitable for implementing the video processing method provided in the embodiments of this application is described.

[0073] like Figure 2 As shown, the video processing system may include an encoding device 201 and a decoding device 202; the encoding device 201 may be a terminal or a server, and the decoding device 202 may be a terminal or a server; the encoding device 201 and the decoding device 202 may establish a direct communication connection through wired communication or an indirect communication connection through wireless communication.

[0074] The encoding device 201 may include an encoder, which can be used to execute the video processing method provided in this embodiment. Specifically, the encoder can configure interpolation information matching the video scene type to be encoded, and encode the video based on the interpolation information to obtain a bitstream file, which is then sent to the decoding device 202. The encoder can be VTM, the reference software for the H.266 / VVC video encoding standard, or libaom, the reference software for the AV1 video encoding standard, etc. The decoding device 202 may include a decoder, which can be used to decode the bitstream file to reconstruct the video to be encoded.

[0075] It should be noted that in the video processing system provided in this application embodiment, the terminal mentioned may be any of the following, including but not limited to: smartphones, tablets, laptops, desktop computers, smartwatches, smart home appliances, smart vehicle terminals, aircraft, etc.; the server mentioned may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, etc. This application embodiment does not limit this.

[0076] For example, in a video-on-demand scenario, encoding device 201 can be a server, and decoding device 202 can be a terminal. The video-on-demand object can perform video-on-demand playback through the terminal. Based on the video-on-demand operation of the video-on-demand object, the terminal can send a video-on-demand request to the server. The video-on-demand request can carry playback information of the video requested by the video-on-demand object (e.g., the identifier of the requested video, the playback progress of the requested video, and the resolution of the requested video, etc.). The server can respond to the video-on-demand request by obtaining the requested video, configuring interpolation information matching the video scene type based on the requested video scene type, encoding the requested video based on the interpolation information, obtaining the bitstream file of the requested video, and sending the bitstream file of the requested video to the terminal. Thus, the terminal can decode the bitstream file to reconstruct the requested video and present the reconstructed requested video to the video-on-demand object.

[0077] For example, in cloud gaming scenarios, cloud gaming (also known as gaming on demand) refers to games running on a game server. In other words, in cloud gaming services, all cloud games run on the game server, not on the client-side. Encoding device 210 can be the game server, and decoding device 202 can be the terminal. The game server can configure interpolation information matching the video scene type of the cloud game video, encode the cloud game video based on the interpolation information, obtain the cloud game video bitstream file, and send the bitstream file to the terminal. The terminal does not need powerful graphics processing and data processing capabilities; it only needs basic streaming media playback capabilities to decode the cloud game video bitstream file, reconstruct the cloud game video, and present the game screen to the player. In cloud gaming scenarios, the terminal can also acquire the operation commands input by the game player and send them to the game server. When the game player performs various operations on the game interface of the terminal, the terminal will report the operation data generated by these operations to the game server. The game server will then refresh the game screen of the corresponding cloud game based on the operation data, and encode the refreshed game screen and return it to the game player for decoding and display, so as to realize the cloud game.

[0078] For example, in a video conferencing scenario, the terminals participating in the video session can be both encoding and decoding devices. The first terminal participating in the video session can capture the first video of the first session object, and based on the video scene type of the first video, configure interpolation information that matches the video scene type for the first video, and encode the first video based on the interpolation information to obtain the bitstream file of the first video, and send the bitstream file of the first video to the second terminal. The second terminal can then decode the bitstream file of the first video, reconstruct the first video, and allow the second session object to view the video of the first session object through the second terminal. The second terminal participating in the video session is similar to the first terminal, and will not be described further here.

[0079] It is understood that the video processing system described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0080] The video processing method provided in the embodiments of this application will now be described in more detail with reference to the accompanying drawings.

[0081] This application provides a video processing method, which mainly describes the configuration process of interpolation information based on video scene type, and the application of interpolation information in the video encoding process. This video processing method can be executed by a computer device, which may be one of the aforementioned... Figure 2 The encoding device 201 in the video processing system shown. For example... Figure 3 As shown, the video processing method includes, but is not limited to, the following steps S301-S304:

[0082] S301, Obtain scene feature information of the business scenario to which the video to be encoded belongs.

[0083] The scene feature information of the business scenario to which the video to be encoded belongs may include, but is not limited to, at least one of the following: video resolution information, video frame rate information, video bitrate information, video playback effect information, and video encoding speed information. Among these:

[0084] ① Video resolution information can refer to the resolution of the video to be encoded when played on a video playback device. The video resolution information can be the default setting of the video playback device. For example, the video resolution information can be the default setting of the video playback product (e.g., video playback software, video playback application, and video playback client) running on the video playback device, where the default resolution is 480P. Alternatively, the video resolution information can be the custom setting of the video consumer (e.g., the video-on-demand object mentioned above) on the video playback device. For example, the video consumer sets the resolution to 1080P in the video playback product running on the video playback device.

[0085] ② Video frame rate information can refer to the frame rate of the video to be encoded when it is played on a video playback device. Frame rate refers to the number of image frames played continuously on a video playback device per unit time (for example, the unit time can be 1 second). Similar to video resolution information, video frame rate information can be the default setting of the video playback device, or it can be a custom setting made by the video consumer on the video playback device.

[0086] ③ Video bitrate information can refer to the amount of data traffic used by the bitstream file of the video to be encoded per unit time. Similar to video resolution information, video bitrate information can be the default setting of the video playback device, or the video bitrate information can be a custom setting made by the video consumer in the video playback device. Furthermore, the set video bitrate information cannot exceed the network transmission bandwidth, so as to prevent the bitstream file of the video to be encoded from being unable to be stably transmitted to the decoding device.

[0087] ④ Video playback effect information can refer to the description of the playback effect of the video to be encoded on the video playback device; for example, video playback effect information can include any of the following: compression effect priority and encoding speed; similar to video resolution information, video playback effect information can be the default settings of the video playback device, or the video playback effect information can be the custom settings of the video consumer in the video playback device.

[0088] ⑤ Video encoding speed information can refer to the encoding speed of the video to be encoded; similar to video resolution information, video encoding speed information can be the default setting of the video playback device, or the video encoding speed information can be a custom setting made by the video consumer in the video playback device.

[0089] S302, Analyze the scenario type of the business scenario based on the scenario feature information to determine the scenario type of the business scenario.

[0090] After obtaining the scene feature information of the business scenario to which the video to be encoded belongs, scene type analysis can be performed based on the scene feature information to determine the scene type of the business scenario. For example, the scene type of the business scenario may include, but is not limited to, any of the following: offline transcoding scenario, real-time streaming media scenario, online cloud gaming scenario, etc.

[0091] Offline transcoding scenarios refer to video encoding performed offline. This type of scenario has lower requirements for video encoding speed and focuses more on video compression efficiency. For example, video-on-demand (VOD) scenarios can fall under offline transcoding. The server can encode the acquired video into a bitstream file for storage. When a VOD request is received, the bitstream file of the requested video can be sent directly to the requesting terminal. In this scenario, server storage space is often limited, and offline transcoding aims to improve video compression while maintaining good video reconstruction quality, thereby saving storage space. Real-time streaming scenarios refer to scenarios involving real-time video acquisition, encoding, and transmission. This type of scenario prioritizes video encoding speed. In cases of poor network quality, video compression quality can be reduced to some extent to minimize the impact of network quality on video encoding speed. For example, live video streaming scenarios can fall under real-time streaming scenarios. Broadcasters expect live video to be transmitted to viewers in real-time to minimize latency, thus requiring faster encoding speeds. Online cloud gaming scenarios refer to scenarios where game videos are encoded and transmitted in real time. This type of scenario has higher requirements for video resolution, video frame rate, and video encoding speed.

[0092] It should be noted that the number of scene feature information can be one or more. When there is only one scene feature information, scene type analysis can be performed based on that scene feature information to determine the scene type of the business scenario. When there are multiple scene feature information, scene type analysis can be performed by combining multiple scene feature information to determine the scene type of the business scenario.

[0093] S303, based on scene type, configures interpolation information for the video to be encoded that matches the scene type.

[0094] After determining the scene type of the business scenario to which the video to be encoded belongs, interpolation information matching the scene type can be configured for the video to be encoded. This interpolation information can include motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded, with the motion estimation interpolation information and motion compensation interpolation information decoupled.

[0095] Specifically, the encoding device can store an interpolation information configuration list, which may include multiple reference types and interpolation information corresponding to each reference type. The interpolation information configuration list can be retrieved, and based on the scene type, the reference type matching the scene type can be determined from among the multiple reference types included in the interpolation information configuration list. Specifically, the reference type matching the scene type can refer to a reference type identical to the scene type. Then, the interpolation information corresponding to the matching reference type can be determined as the interpolation information matching the scene type. An exemplary interpolation information configuration list is shown in Table 1 below:

[0096] Table 1:

[0097]

[0098] As shown in Table 1 above, the interpolation information configuration list includes the number of motion estimation interpolation taps and the number of motion compensation interpolation taps for each reference type. For cases where the number of taps, tap coefficients, and fractional pixel interpolation algorithms correspond one-to-one, once the number of taps is determined, the tap coefficients and fractional pixel interpolation algorithms are also determined. Therefore, the interpolation information configuration list only needs to include the number of taps. For cases where the number of taps, tap coefficients, and fractional pixel interpolation algorithms do not correspond one-to-one, the interpolation information configuration list can also include any one or more of the number of taps, tap coefficients, and fractional pixel interpolation algorithms, depending on the actual situation.

[0099] In this way, the interpolation information configured based on scene type is decoupled from the motion estimation and motion compensation processes, and the configuration of interpolation information is more flexible, thereby improving the video coding effect.

[0100] The above content describes the decoupling of motion estimation interpolation information from motion compensation interpolation information. Besides decoupling the motion estimation and motion compensation processes, the interpolation information can also be decoupled for different image component types, different pixel interpolation directions, or both different image component types and different pixel interpolation directions. These cases will be described in detail below:

[0101] (1) Interpolation information is decoupled for different image component types:

[0102] Specifically, each image frame in the video frame to be encoded can include N image component types, where N is a positive integer. For example, in a video generated according to YUV (a color encoding method), the N image component types of each image frame in the video can include Y components and UV components. The Y component refers to Luminance or Luma components (luminance component), which is the grayscale value, and the UV components refer to Chrominance or Chroma components, which are used to describe color and saturation. As another example, in a video generated according to RGB (another color encoding method), the N image component types of each image frame in the video can include R components, G components, and B components. The R component refers to the Red component, the G component is the Green component, and the B component is the Blue component. It should be noted that the embodiments of this application use N image component types including Y components and UV components, that is, N image component types including luminance components and chrominance components, as an example for illustration.

[0103] In this scenario, in response to an image component decoupling configuration request, interpolation information matching the scene type can be configured for each image component type of the video to be encoded, based on the scene type. That is, motion estimation interpolation information and motion compensation interpolation information can be configured for each image component type of the video to be encoded. Specifically, a configuration list of interpolation information for image component type decoupling can be obtained. This list may include multiple reference types and interpolation information for different image component types corresponding to each reference type. Based on the scene type, a reference type matching the scene type can be determined from the multiple reference types included in the configuration list. Specifically, a reference type matching the scene type can refer to a reference type identical to the scene type. Then, the interpolation information corresponding to the matching reference type can be determined as the interpolation information matching the scene type. An exemplary configuration list of interpolation information for image component type decoupling is shown in Table 2 below.

[0104] Table 2:

[0105]

[0106] As shown in Table 2 above, the interpolation information configuration list for image component decoupling includes the number of motion estimation interpolation taps and motion compensation interpolation taps for different image component types under each reference type. Furthermore, for the chroma component, the motion estimation result of the chroma component depends on the motion estimation result of the luminance component. Therefore, the interpolation information configuration list for image component decoupling does not need to include the motion estimation interpolation information of the chroma component.

[0107] It should be added that the image component decoupling configuration request can be generated based on the image component decoupling configuration operation in the encoder's configuration interface. For example... Figure 4 As shown, an encoder configuration interface can be provided to encoder administrators, which may include image component type decoupling configuration option 401 (e.g., Figure 4 The “Image Component Decoupling” option shown can generate an image component decoupling configuration request if the image component type decoupling configuration option 401 is triggered.

[0108] In this way, the interpolation information configured based on scene type can not only decouple the motion estimation and motion compensation processes, but also decouple different image component types. The configuration of interpolation information is more flexible, thereby improving the video coding effect.

[0109] (2) Interpolation information is decoupled for different pixel interpolation directions:

[0110] In addition to decoupling different image component types, interpolation information can also decouple pixel interpolation directions. Specifically, motion estimation interpolation information can be used for pixel interpolation during motion estimation, and motion compensation interpolation information can be used for pixel interpolation during motion compensation. Pixel interpolation can have M pixel interpolation directions, where M is a positive integer; for example, the M pixel interpolation directions can include horizontal and vertical directions. Horizontal pixel interpolation refers to pixel interpolation performed in the horizontal direction, and vertical pixel interpolation refers to pixel interpolation performed in the vertical direction.

[0111] In this scenario, in response to an interpolation direction decoupling configuration request, interpolation information matching the scene type can be configured for each pixel interpolation direction of the video to be encoded, based on the scene type. That is, motion estimation interpolation information and motion compensation interpolation information can be configured for each pixel interpolation direction of the video to be encoded. Specifically, a configuration list of interpolation information for pixel interpolation direction decoupling can be obtained. This list may include multiple reference types and interpolation information corresponding to different pixel interpolation directions for each reference type. Based on the scene type, a reference type matching the scene type can be determined from the multiple reference types included in the configuration list. Specifically, a reference type matching the scene type can refer to a reference type identical to the scene type. Then, the interpolation information corresponding to the matching reference type can be determined as the interpolation information matching the scene type. An exemplary configuration list of interpolation information for pixel interpolation direction decoupling is shown in Table 3 below.

[0112] Table 3:

[0113]

[0114] As shown in Table 3 above, the interpolation information configuration list for interpolation direction decoupling includes the number of motion estimation interpolation taps and the number of motion compensation interpolation taps for different pixel interpolation directions under each reference type.

[0115] It should be added that the interpolation direction decoupling configuration request can be generated in the encoder's configuration interface based on the interpolation direction decoupling configuration operation. For example... Figure 4 As shown, an encoder configuration interface can be provided to encoder administrators, and the encoder configuration interface may include pixel interpolation direction decoupling configuration option 402 (e.g., Figure 4 The “Interpolation Direction Decoupling” option shown can generate an interpolation direction decoupling configuration request if pixel interpolation direction decoupling configuration option 402 is triggered.

[0116] In this way, the interpolation information configured based on scene type not only decouples the motion estimation and motion compensation processes, but also decouples different pixel interpolation directions. The configuration of interpolation information is more flexible, thereby improving the video encoding effect.

[0117] (3) Interpolation information is decoupled for different image component types and pixel interpolation directions:

[0118] Specifically, each image frame in the video frame to be encoded may include N image component types, where N is a positive integer; this embodiment of the application takes as an example that the N image components include Y components and UV components, that is, the N image components include luminance components and chrominance components. Pixel interpolation may have M pixel interpolation directions, where M is a positive integer; for example, the M pixel interpolation directions may include the horizontal direction and the vertical direction.

[0119] In this scenario, in response to a request for a comprehensive decoupling configuration of image components and interpolation directions, interpolation information matching the scene type can be configured for each image component type of the video to be encoded in different pixel interpolation directions, based on the scene type. That is, motion estimation interpolation information and motion compensation interpolation information can be configured for each image component type of the video to be encoded in different pixel interpolation directions. Specifically, a comprehensive decoupling interpolation information configuration list for image component types and pixel interpolation directions can be obtained. This list can include multiple reference types and interpolation information corresponding to each reference type in different image component types and different pixel interpolation directions. Based on the scene type, a reference type matching the scene type can be determined from the multiple reference types included in the comprehensive decoupling interpolation information configuration list. Specifically, a reference type matching the scene type can refer to a reference type identical to the scene type. Then, the interpolation information corresponding to the matching reference type can be determined as the interpolation information matching the scene type. An exemplary interpolation direction decoupling interpolation information configuration list is shown in Table 4 below.

[0120] Table 4:

[0121]

[0122] As shown in Table 4 above, the interpolation information configuration list for the integrated decoupling of image components and interpolation directions includes the number of motion estimation interpolation taps and motion compensation interpolation taps for different image component types and different pixel interpolation directions under each reference type. Furthermore, for the chroma component, the motion estimation result of the chroma component depends on the motion estimation result of the luminance component. Therefore, the integrated decoupling interpolation information configuration list does not need to include the motion estimation interpolation information of the chroma component.

[0123] It should be added that the integrated decoupling configuration request for image components and interpolation direction can be generated based on the integrated decoupling configuration operation in the encoder's configuration interface. For example... Figure 4 As shown, an encoder configuration interface can be provided to encoder administrators, and the encoder configuration interface may include comprehensive decoupling configuration option 403 (e.g., Figure 4The “Image Components, Interpolation Direction Integrated Decoupling” option shown can generate an integrated decoupling configuration request if integrated decoupling configuration option 403 is triggered.

[0124] In this way, the interpolation information configured based on scene type not only decouples the motion estimation and motion compensation processes, but also decouples different image component types and different pixel interpolation directions. The configuration of interpolation information is more flexible, thereby improving the video encoding effect.

[0125] It should be noted that, in this embodiment, the configured interpolation information decouples the motion estimation process and the motion compensation process, which can be understood as the encoder's default setting; the decoupling of interpolation information for image component type, pixel interpolation direction, and both image component type and pixel interpolation direction can be considered as optional operations based on the default settings, for example, the above... Figure 4 The encoder is configured through various options in its configuration interface. However, in practice, the default encoder settings can be any of the following: First, the interpolation information is decoupled from the motion estimation and motion compensation processes; second, the interpolation information is decoupled from the motion estimation and motion compensation processes, and also decoupled for different image component types; third, the interpolation information is decoupled from the motion estimation and motion compensation processes, and also decoupled for different pixel interpolation directions; fourth, the interpolation information is decoupled from the motion estimation and motion compensation processes, and also decoupled for both different pixel interpolation directions and different image component types.

[0126] S304 encodes the video to be encoded based on the interpolation information.

[0127] After analyzing the scenario type of the business scenario based on the scenario feature information and determining the scenario type of the business scenario, the video to be encoded can be encoded based on the interpolation information.

[0128] Interpolation information can be used to determine the matching block of the image block to be encoded (i.e., the macroblock to be encoded) in the image frame to be encoded within the video to be encoded through motion estimation and motion compensation. This allows the image block to be encoded to be encoded based on the determined matching block. Specifically, the image block to be encoded in the image frame to be encoded within the video to be encoded can be obtained; secondly, motion compensation can be performed on the reference frame of the video frame to be encoded based on motion compensation interpolation information to reconstruct the reference frame; and thirdly, motion estimation can be performed on the image block to be encoded based on motion estimation interpolation information to determine the matching block of the image block to be encoded in the reference frame; finally, the image block to be encoded can be encoded based on the matching block.

[0129] In one implementation, a matching block can be used to perform temporal filtering on the image block to be encoded, and then the temporally filtered image block to be encoded can be encoded. Specifically, performing temporal filtering on the image block to be encoded based on the matching block can mean: weighting the pixel content of the matching block and the image block to be encoded, thereby effectively removing noise from the image block to be encoded.

[0130] In another implementation, motion estimation is performed on the image block to be encoded based on motion estimation interpolation information. This involves determining the matching block of the image block to be encoded in the reference frame of the video frame to be encoded, as well as determining the motion vector between the image block to be encoded and the matching block. In this case, encoding the image block to be encoded based on the matching block specifically means: determining a residual signal based on the difference between the matching block and the image block to be encoded. The residual signal can be the result of subtracting the pixel content of the matching block from the pixel content of the image block to be encoded, or vice versa. Then, the image block to be encoded can be encoded based on the residual signal and the motion vector. In other words, by encoding the residual signal and the motion vector, the image block to be encoded can be reconstructed in the decoding device.

[0131] In this embodiment, interpolation information matching the scene type can be flexibly configured for the video to be encoded according to the scene type of the business scenario to which the video belongs, which can improve the video encoding effect. Furthermore, the configured interpolation information is decoupled from the motion estimation process and the motion compensation process. On this basis, the interpolation information can also be configured to be decoupled for different image component types, different pixel interpolation directions, or different pixel interpolation directions and different image component types, making the interpolation information configuration method very flexible.

[0132] This application provides a video processing method, which mainly describes the configuration process of scene feature information and the process of scene type analysis based on the scene feature information. This video processing method can be executed by a computer device, which can be one of the aforementioned... Figure 2 The encoding device 201 in the video processing system shown. For example... Figure 5 As shown, the video processing method includes, but is not limited to, the following steps S501-S505:

[0133] S501, obtain scene feature information of the business scenario to which the video to be encoded belongs.

[0134] As described above, the scene feature information of the business scenario to which the video to be encoded belongs may include, but is not limited to, at least one of the following: video resolution information, video frame rate information, video bitrate information, video playback effect information, and video encoding speed information. Scene feature information can be configured in the scene feature configuration interface of the video playback device; for example... Figure 6 The diagram illustrates a scene feature configuration interface, where video resolution and frame rate information can be configured. For example, the video resolution can be set to 1080P, and the frame rate to 60fp / s (60 frames per second). Alternatively, the scene feature information can be the default settings of the video playback device.

[0135] When scene feature information is configured in a scene feature configuration interface, this interface can be a service interface within the video playback device. In one implementation, the encoding device and the video playback device can be the same device. For example, in a video session scenario, both the encoding device and the video playback device are terminals of the session object. When the encoding device and video playback device are the same device, the encoding device can directly display the scene feature configuration interface and then retrieve the scene feature information configured therein. In another implementation, the encoding device and the video playback device can be different devices. For example, in a cloud gaming scenario, the video playback device is the terminal running the game client, while the encoding device is the game server. When the encoding device and video playback device are different devices, the video playback device can display the scene feature configuration interface and send the scene feature information configured therein to the encoding device.

[0136] When the scene feature information is the default setting of the video playback device, in one implementation, the encoding device and the video playback device can be the same device; in this case, the encoding device can directly obtain the default scene feature information. In another implementation, the encoding device and the video playback device can be different devices; in this case, the video playback device can send the default scene feature information to the encoding device.

[0137] S502, in the scene type matching relationship, determine the target type indication information that matches the scene feature information.

[0138] S503, determine the scenario type corresponding to the target type indication information as the scenario type of the business scenario.

[0139] In steps S502-S503, after obtaining the scene feature information of the business scenario to which the video to be encoded belongs, a scene type matching relationship can be obtained. This relationship can include multiple type indication information and the scene type corresponding to each type indication information. Then, the target type indication information matching the scene feature information can be determined from the scene type matching relationship. The scene type corresponding to the target type indication information can then be determined as the scene type of the business scenario. The methods for determining the target type indication information matching the scene feature information are described below for both single and multiple scene feature information:

[0140] (1) When the number of scene feature information is single, the target type indication information that matches the single scene feature information can be determined, and then the scene type corresponding to the target type indication information can be determined as the scene type of the business scenario. Specifically:

[0141] Taking video encoding speed information as an example, the type indicator information in the scene type matching relationship refers to the encoding speed range. The target type indicator information that matches the scene feature information refers to the encoding speed range to which the video encoding speed information belongs. In other words, when the scene feature information is video encoding speed information, the encoding speed range to which the video encoding speed information belongs can be determined in the scene type matching relationship. Then, the scene type corresponding to the encoding speed range to which the video encoding speed information belongs can be determined as the scene type of the business scenario. An exemplary scene type matching relationship when the scene feature information is video encoding speed information can be seen in Table 5 below:

[0142] Table 5

[0143] Encoding speed range Scene type [0，100] Offline transcoding scenarios (100，1000] Real-time streaming scenarios (1000，+∞) Online cloud gaming scenarios

[0144] As shown in Table 5 above, when the video encoding speed information is within the encoding speed range [0, 100], the scenario type of the business scenario to which the video to be encoded belongs can be determined to be an offline transcoding scenario; when the video encoding speed information is within the encoding speed range (100, 1000], the scenario type of the business scenario to which the video to be encoded belongs can be determined to be a real-time streaming media scenario; when the video encoding speed information is within the encoding speed range (1000, +∞), the scenario type of the business scenario to which the video to be encoded belongs can be determined to be an online cloud gaming scenario.

[0145] It should be noted that the case where the scene feature information is video bitrate information, video resolution information, and video bitrate information is similar to the case where the scene feature information is video encoding speed information. For details, please refer to the relevant description of the case where the scene feature information is video encoding speed information. This application embodiment will not elaborate on the case where the scene feature information is video bitrate information, video resolution information, and video frame rate information.

[0146] Taking video playback effect information as an example, the type indicator information in the scene type matching relationship refers to the playback effect identifier. The target type indicator information that matches the scene feature information refers to the playback effect identifier that matches the video playback effect information. Matching the video playback effect information and the playback effect identifier can mean that the video playback effect information and the playback effect identifier are the same or similar. In other words, when the scene feature information is video playback effect information, the playback effect identifier that matches the video playback effect information can be determined in the scene type matching relationship. Then, the scene type corresponding to the playback effect identifier that matches the video playback effect information can be determined as the scene type of the business scenario. When the scene feature information is video playback effect information, an exemplary scene type matching relationship can be seen in Table 6 below:

[0147] Table 6

[0148] Playback effect indicator Scene type Prioritize compression effect Offline transcoding scenarios Encoding speed priority Real-time streaming scenarios, online cloud gaming scenarios

[0149] As shown in Table 6 above, when the video playback effect information matches the playback effect identifier that prioritizes compression effect, it can be determined that the scenario type of the business scenario to which the video to be encoded belongs is an offline transcoding scenario; when the video playback effect information matches the playback effect identifier that prioritizes encoding speed, it can be determined that the scenario type of the business scenario to which the video to be encoded belongs is a real-time streaming media scenario or an online cloud gaming scenario.

[0150] (2) When there are multiple scene feature information items, target type indication information that matches these multiple scene feature information items can be determined. Then, the scene type corresponding to the target type indication information can be determined as the scene type of the business scenario. Specifically:

[0151] Taking scene feature information including video resolution information and video encoding speed information as an example, the target type indication information that matches the scene feature information refers to: the encoding speed range to which the video encoding speed information belongs, and the resolution range to which the video resolution information belongs. In other words, when the scene feature information includes video encoding speed information and video resolution information, the encoding speed range to which the video encoding speed information belongs, and the resolution range to which the video resolution information belongs, can be determined in the scene type matching relationship. Then, the scene type that corresponds to the encoding speed range to which the video encoding speed information belongs, and the resolution range to which the video resolution information belongs, can be determined as the scene type of the business scene.

[0152] S504, based on scene type, configures interpolation information for the video to be encoded that matches the scene type.

[0153] In this embodiment, the execution process of step S504 is the same as described above. Figure 3 The execution process of step S303 in the illustrated embodiment is the same; for details, please refer to the above. Figure 3 The relevant description of step S303 in the illustrated embodiment will not be repeated here.

[0154] S505 encodes the video to be encoded based on the interpolation information.

[0155] In this embodiment, the execution process of step S505 is the same as described above. Figure 3 The execution process of step S304 in the illustrated embodiment is the same; for details, please refer to the above. Figure 3 The relevant description of step S304 in the illustrated embodiment will not be repeated here.

[0156] In this embodiment, the scene type of the business scene to which the video to be encoded belongs can be analyzed based on the scene feature information of the business scene to which the video to be encoded belongs. Thus, interpolation information matching the scene type can be flexibly configured for the video to be encoded according to the scene type of the business scene to which the video to be encoded belongs, which can improve the video encoding effect.

[0157] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below.

[0158] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a video processing device provided in an embodiment of this application. The video processing device can be installed in the computer equipment provided in the embodiment of this application. The computer equipment can be a terminal or a server. Figure 7The video processing apparatus shown may be a computer program (including program code) running on a computer device, which can be used to execute... Figure 3 or Figure 5 Some or all of the steps in the method embodiments shown. Please refer to [link / reference]. Figure 7 The video processing apparatus may include the following units:

[0159] The acquisition unit 701 is used to acquire scene feature information of the business scenario to which the video to be encoded belongs;

[0160] The processing unit 702 is used to perform scenario type analysis on the business scenario based on scenario feature information to determine the scenario type of the business scenario;

[0161] The processing unit 702 is also configured to configure interpolation information matching the scene type for the video to be encoded based on the scene type; the interpolation information includes motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled.

[0162] The processing unit 702 is also used to encode the video to be encoded based on the interpolation information.

[0163] In one implementation, the processing unit 702 is used to perform scenario type analysis on the business scenario based on scenario feature information. When determining the scenario type of the business scenario, it specifically performs the following steps:

[0164] In the scene type matching relationship, target type indication information that matches the scene feature information is determined; the scene type matching relationship includes multiple type indication information and the scene type corresponding to each type indication information.

[0165] The scenario type corresponding to the target type indication information is determined as the scenario type of the business scenario.

[0166] In one implementation, the scene feature information includes at least one of the following: video resolution information, video frame rate information, video bit rate information, video playback effect information, and video encoding speed information;

[0167] The acquisition unit 701, when acquiring scene feature information of the business scenario to which the video to be encoded belongs, specifically performs the following steps:

[0168] Display scene feature configuration interface;

[0169] Retrieve scene feature information configured in the scene feature configuration interface.

[0170] In one implementation, each image frame in the video to be encoded includes N image component types, where N is a positive integer.

[0171] The processing unit 702, when configuring interpolation information matching the scene type for the video to be encoded based on the scene type, specifically performs the following steps:

[0172] In response to the image component decoupling configuration request, based on the scene type, interpolation information matching the scene type is configured for each image component type of the video to be encoded.

[0173] In one implementation, motion estimation interpolation information is used for pixel interpolation during motion estimation, and motion compensation interpolation information is used for pixel interpolation during motion compensation; pixel interpolation has M pixel interpolation directions, where M is a positive integer;

[0174] The processing unit 702, when configuring interpolation information matching the scene type for the video to be encoded based on the scene type, specifically performs the following steps:

[0175] In response to the interpolation direction decoupling configuration request, based on the scene type, interpolation information matching the scene type is configured for the interpolation direction of each pixel in the video to be encoded.

[0176] In one implementation, when processing unit 702 configures interpolation information matching the scene type for the video to be encoded based on the scene type, it specifically performs the following steps:

[0177] Retrieve the interpolation information configuration list; the interpolation information configuration list includes multiple reference types and the interpolation information corresponding to each reference type;

[0178] Based on the scene type, determine the reference type that matches the scene type from among the multiple reference types included in the interpolation information configuration list;

[0179] The interpolation information corresponding to the matching reference type is determined as the interpolation information that matches the scene type.

[0180] In one implementation, the processing unit 702, when encoding the video to be encoded based on the interpolation information, specifically performs the following steps:

[0181] Obtain the image block to be encoded from the image frame to be encoded contained in the video to be encoded;

[0182] Motion compensation is performed on the reference frame of the video frame to be encoded based on motion compensation interpolation information to reconstruct the reference frame of the video frame to be encoded.

[0183] Motion estimation is performed on the image block to be encoded based on motion estimation interpolation information, and the matching block of the image block to be encoded is determined in the reference frame of the video frame to be encoded.

[0184] The image block to be encoded is encoded based on the matching block.

[0185] In one implementation, the processing unit 702, when encoding the image block to be encoded based on the matching block, specifically performs the following steps:

[0186] Temporal filtering is performed on the image block to be encoded based on the matching block;

[0187] Encode the image block to be encoded after time-domain filtering.

[0188] In one implementation, the processing unit 702 is used to perform motion estimation on the image block to be encoded based on motion estimation interpolation information. When determining the matching block of the image block to be encoded in the reference frame of the video frame to be encoded, it is specifically used to perform the following steps:

[0189] Motion estimation is performed on the image block to be encoded based on motion estimation interpolation information. The matching block of the image block to be encoded and the motion vector between the image block to be encoded and the matching block are determined in the reference frame of the video frame to be encoded.

[0190] Processing unit 702 is configured to encode the image block to be encoded according to the matching block, including:

[0191] The residual signal is determined based on the difference between the image block to be encoded and the matching block;

[0192] The image block to be encoded is encoded based on the residual signal and motion vector.

[0193] In one implementation, the interpolation information includes at least one of the following: a fractional pixel interpolation algorithm, a number of taps, and tap coefficients corresponding to the number of taps.

[0194] According to another embodiment of this application, Figure 7 The various units in the video processing apparatus shown can be individually or entirely combined into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the video processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0195] According to another embodiment of this application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), a device capable of performing operations such as... Figure 3 or Figure 5 The computer program (including program code) for each step involved in some or all of the methods shown, to construct such... Figure 7 The video processing apparatus shown herein, and the video processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable storage medium, loaded onto the aforementioned computing device via the computer-readable storage medium, and executed therein.

[0196] In this embodiment, after determining the scene type of the business scenario to which the video to be encoded belongs, interpolation information matching the scene type can be configured for the video to be encoded, thereby enabling encoding of the video based on the determined interpolation information. Furthermore, the interpolation information includes motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled. In other words, this embodiment can flexibly configure interpolation information matching the scene type of the business scenario to which the video to be encoded belongs, and the configured interpolation information is decoupled from the motion estimation process and the motion compensation process, which can improve the video encoding effect.

[0197] Based on the above methods and apparatus embodiments, this application provides a computer device, which may be an encoding device. Please refer to... Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 8 The computer device shown includes at least a processor 801, an input interface 802, an output interface 803, and a computer-readable storage medium 804. The processor 801, input interface 802, output interface 803, and computer-readable storage medium 804 can be connected via a bus or other means.

[0198] Computer-readable storage medium 804 can be stored in the memory of a computer device. Computer-readable storage medium 804 is used to store computer programs, including computer instructions. Processor 801 is used to execute the program instructions stored in computer-readable storage medium 804. Processor 801 (or CPU (Central Processing Unit)) is the computing and control core of the computer device. It is suitable for implementing one or more computer instructions, specifically for loading and executing one or more computer instructions to achieve corresponding method flows or corresponding functions.

[0199] This application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the operating system of the computer device. Furthermore, the storage space also stores one or more computer instructions suitable for loading and execution by a processor. These computer instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0200] In some embodiments, the processor 801 may load and execute one or more computer instructions stored in the computer-readable storage medium 804 to implement the aforementioned related... Figure 3 or Figure 5 The corresponding steps of the video processing method shown are as follows. In a specific implementation, the computer instructions in the computer-readable storage medium 804 are loaded by the processor 801 and executed as follows:

[0201] Obtain scene feature information of the business scenario to which the video to be encoded belongs;

[0202] Analyze the scenario type of the business scenario based on the scenario feature information to determine the scenario type of the business scenario;

[0203] Based on scene type, interpolation information matching the scene type is configured for the video to be encoded; the interpolation information includes motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled;

[0204] The video to be encoded is encoded based on the interpolation information.

[0205] In one implementation, when the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to perform scenario type analysis of the business scenario based on scenario feature information and determine the scenario type of the business scenario, the instructions specifically perform the following steps:

[0206] In the scene type matching relationship, target type indication information that matches the scene feature information is determined; the scene type matching relationship includes multiple type indication information and the scene type corresponding to each type indication information.

[0207] The scenario type corresponding to the target type indication information is determined as the scenario type of the business scenario.

[0208] In one implementation, the scene feature information includes at least one of the following: video resolution information, video frame rate information, video bit rate information, video playback effect information, and video encoding speed information;

[0209] When the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to obtain scene feature information of the service scene to which the video to be encoded belongs, they are specifically used to perform the following steps:

[0210] Display scene feature configuration interface;

[0211] Retrieve scene feature information configured in the scene feature configuration interface.

[0212] In one implementation, each image frame in the video to be encoded includes N image component types, where N is a positive integer.

[0213] When the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801, based on the scene type, to configure interpolation information matching the scene type for the video to be encoded, specifically, they are used to perform the following steps:

[0214] In response to the image component decoupling configuration request, based on the scene type, interpolation information matching the scene type is configured for each image component type of the video to be encoded.

[0215] In one implementation, motion estimation interpolation information is used for pixel interpolation during motion estimation, and motion compensation interpolation information is used for pixel interpolation during motion compensation; pixel interpolation has M pixel interpolation directions, where M is a positive integer;

[0216] When the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801, based on the scene type, to configure interpolation information matching the scene type for the video to be encoded, specifically, they are used to perform the following steps:

[0217] In response to the interpolation direction decoupling configuration request, based on the scene type, interpolation information matching the scene type is configured for the interpolation direction of each pixel in the video to be encoded.

[0218] In one implementation, when the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 based on the scene type to configure interpolation information matching the scene type for the video to be encoded, specifically, the following steps are performed:

[0219] Retrieve the interpolation information configuration list; the interpolation information configuration list includes multiple reference types and the interpolation information corresponding to each reference type;

[0220] Based on the scene type, determine the reference type that matches the scene type from among the multiple reference types included in the interpolation information configuration list;

[0221] The interpolation information corresponding to the matching reference type is determined as the interpolation information that matches the scene type.

[0222] In one implementation, the computer instructions in the computer-readable storage medium 804, when loaded and executed by the processor 801 to encode the video to be encoded based on the interpolation information, specifically perform the following steps:

[0223] Obtain the image block to be encoded from the image frame to be encoded contained in the video to be encoded;

[0224] Motion compensation is performed on the reference frame of the video frame to be encoded based on motion compensation interpolation information to reconstruct the reference frame of the video frame to be encoded.

[0225] Motion estimation is performed on the image block to be encoded based on motion estimation interpolation information, and the matching block of the image block to be encoded is determined in the reference frame of the video frame to be encoded.

[0226] The image block to be encoded is encoded based on the matching block.

[0227] In one implementation, when the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to encode the image block to be encoded according to the matching block, they are specifically used to perform the following steps:

[0228] Temporal filtering is performed on the image block to be encoded based on the matching block;

[0229] Encode the image block to be encoded after time-domain filtering.

[0230] In one implementation, the computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to perform motion estimation of the image block to be encoded based on motion estimation interpolation information. Specifically, when determining the matching block of the image block to be encoded in the reference frame of the video frame to be encoded, the instructions perform the following steps:

[0231] Motion estimation is performed on the image block to be encoded based on motion estimation interpolation information. The matching block of the image block to be encoded and the motion vector between the image block to be encoded and the matching block are determined in the reference frame of the video frame to be encoded.

[0232] The computer instructions in the computer-readable storage medium 804 are loaded and executed by the processor 801 to encode the image block to be encoded according to the matching block, including:

[0233] The residual signal is determined based on the difference between the image block to be encoded and the matching block;

[0234] The image block to be encoded is encoded based on the residual signal and motion vector.

[0235] In one implementation, the interpolation information includes at least one of the following: a fractional pixel interpolation algorithm, a number of taps, and tap coefficients corresponding to the number of taps.

[0236] In this embodiment, after determining the scene type of the business scenario to which the video to be encoded belongs, interpolation information matching the scene type can be configured for the video to be encoded, thereby enabling encoding of the video based on the determined interpolation information. Furthermore, the interpolation information includes motion estimation interpolation information used in the motion estimation process of the video to be encoded, and motion compensation interpolation information used in the motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled. In other words, this embodiment can flexibly configure interpolation information matching the scene type of the business scenario to which the video to be encoded belongs, and the configured interpolation information is decoupled from the motion estimation process and the motion compensation process, which can improve the video encoding effect.

[0237] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the video processing methods provided in the various alternative embodiments described above.

[0238] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of video processing, the method comprising: The method comprises: obtaining scene characteristic information of a service scene to which a video to be encoded belongs; performing scene type analysis on the service scene according to the scene characteristic information, to determine a scene type of the service scene; based on the scene type, configuring interpolation information matched with the scene type for the video to be encoded; the interpolation information comprises motion estimation interpolation information used in a motion estimation process of the video to be encoded, and motion compensation interpolation information used in a motion compensation process of the video to be encoded; the motion estimation interpolation information and the motion compensation interpolation information are decoupled; the motion estimation interpolation information or the motion compensation interpolation information comprises at least one of a fractional pixel interpolation algorithm, a tap number, and a tap coefficient corresponding to the tap number; encoding the video to be encoded according to the interpolation information.

2. The method of claim 1, wherein, The scene type analysis on the service scene according to the scene characteristic information comprises: in a scene type matching relationship, determining target type indication information matched with the scene characteristic information; the scene type matching relationship comprises a plurality of type indication information, and a scene type corresponding to each type indication information; determining the scene type corresponding to the target type indication information as the scene type of the service scene.

3. The method of claim 1, wherein, The scene characteristic information comprises at least one of video resolution information, video frame rate information, video code rate information, video playing effect information, and video encoding speed information. The obtaining of the scene characteristic information of the service scene to which the video to be encoded belongs comprises: displaying a scene characteristic configuration interface; obtaining scene characteristic information configured in the scene characteristic configuration interface.

4. The method of claim 1, wherein, Each image frame in the video to be encoded comprises N image component types, and N is a positive integer; The configuration of the interpolation information matched with the scene type for the video to be encoded based on the scene type comprises: in response to an image component decoupling configuration request, configuring interpolation information matched with the scene type for each image component type of the video to be encoded respectively based on the scene type.

5. The method of claim 1, wherein, The motion estimation interpolation information is used for pixel interpolation in the motion estimation process, and the motion compensation interpolation information is used for pixel interpolation in the motion compensation process; there are M pixel interpolation directions, and M is a positive integer; The configuration of the interpolation information matched with the scene type for the video to be encoded based on the scene type comprises: in response to an interpolation direction decoupling configuration request, configuring interpolation information matched with the scene type for each pixel interpolation direction of the video to be encoded respectively based on the scene type.

6. The method of claim 1, wherein, The configuration of the interpolation information matched with the scene type for the video to be encoded based on the scene type comprises: obtaining an interpolation information configuration list; the interpolation information configuration list comprises a plurality of reference types, and interpolation information corresponding to each reference type; determining a reference type matched with the scene type from the plurality of reference types in the interpolation information configuration list according to the scene type. The interpolation information corresponding to the matched reference type is determined as the interpolation information matched with the scene type.

7. The method of claim 1, wherein, The encoding of the to-be-encoded video according to the interpolation information comprises: obtaining a to-be-encoded image block in a to-be-encoded image frame contained in the to-be-encoded video; performing motion compensation on the reference frame of the to-be-encoded video frame based on the motion compensation interpolation information, to reconstruct the reference frame of the to-be-encoded video frame; performing motion estimation on the to-be-encoded image block based on the motion estimation interpolation information, to determine a matching block of the to-be-encoded image block in the reference frame of the to-be-encoded video frame; encoding the to-be-encoded image block according to the matching block.

8. The method of claim 7, wherein, The encoding of the to-be-encoded image block according to the matching block comprises: performing temporal filtering processing on the to-be-encoded image block according to the matching block; encoding the to-be-encoded image block after the temporal filtering processing.

9. The method of claim 7, wherein, The performing of the motion estimation on the to-be-encoded image block based on the motion estimation interpolation information, to determine a matching block of the to-be-encoded image block in the reference frame of the to-be-encoded video frame, comprises: performing the motion estimation on the to-be-encoded image block based on the motion estimation interpolation information, to determine a matching block of the to-be-encoded image block in the reference frame of the to-be-encoded video frame, and a motion vector between the to-be-encoded image block and the matching block; The encoding of the to-be-encoded image block according to the matching block comprises: determining a residual signal according to the difference between the to-be-encoded image block and the matching block; encoding the to-be-encoded image block according to the residual signal and the motion vector.

10. A video processing device, comprising: comprise: an obtaining unit, configured to obtain scene characteristic information of a service scene to which a to-be-encoded video belongs; a processing unit, configured to perform scene type analysis on the service scene according to the scene characteristic information, to determine a scene type of the service scene; the processing unit is further configured to configure, based on the scene type, interpolation information matched with the scene type for the to-be-encoded video; the interpolation information comprises motion estimation interpolation information used in a motion estimation process of the to-be-encoded video, and motion compensation interpolation information used in a motion compensation process of the to-be-encoded video; the motion estimation interpolation information and the motion compensation interpolation information are decoupled; the motion estimation interpolation information or the motion compensation interpolation information comprises at least one of the following: a fractional pixel interpolation algorithm, a tap number, and a tap coefficient corresponding to the tap number; the processing unit is further configured to encode the to-be-encoded video according to the interpolation information.

11. A computer device, comprising: comprise: a processor adapted to implement a computer program; a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being adapted to be loaded and executed by the processor to implement the video processing method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, the computer readable storage medium stores a computer program, the computer program being adapted to be loaded and executed by the processor to implement the video processing method according to any one of claims 1-9.

13. A computer program product, characterised in that, The computer program product comprises computer instructions stored in a computer readable storage medium, a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the video processing method according to any one of claims 1-9.

Images