AI-based video processing method, device, equipment and readable storage medium

Abstract

The present application discloses an AI-based video processing method, device, equipment and readable storage medium, the method comprising: determining a first coding block carrying target object information in a to-be-coded picture; when the first coding block needs to be accurately coded, determining a reference picture of the to-be-coded picture, performing SIFT feature matching of the first coding block with each corresponding second coding block in the reference picture, and taking the second coding block with the highest SIFT feature matching as the most matched block; determining a rotation angle and block index information based on the most matched block; and encoding based on the rotation angle and the block index information. The present application solves the problem of effectively encoding the rotation of the target object.

Description

Technical Field

[0001] The present application relates to the field of multimedia processing technology, and in particular to an AI-based video processing method, device, equipment and readable storage medium. Background Art

[0002] In the prior art, a general coding scheme is usually used to encode a video. Specifically, the current image is fixedly divided into blocks, and then the current block is matched with the blocks in the previous or next frame of the image, and the block with the smallest error is found as the reference block of the current block of the current image, and then the residual between the current block and the reference block is encoded.

[0003] Encoding the residual between the current block and the reference block is effective for target objects that translate in the current block. However, when the target object in the current block rotates, it is often impossible to determine the most matching reference block. In other words, the residual between the found reference block and the current block will also be large. The large residual between the current block and the reference block increases the bit rate of the bitstream, resulting in a decrease in image quality and reducing the competitiveness of audio and video products. Summary of the invention

[0004] The main purpose of this application is to provide an AI-based video processing method, device, equipment and readable storage medium, aiming to solve the technical problem that existing video compression technology cannot effectively encode the rotation of the target object.

[0005] To achieve the above objectives, the present application provides an AI-based video processing method, which is applied to an encoding end. The AI-based video processing method includes the following steps:

[0006] Determine a first coding block carrying target object information in the picture to be coded;

[0007] When the first coding block needs to be accurately coded, determining a reference picture of the picture to be coded, performing scale invariant feature transform (SIFT) feature matching between the first coding block and each corresponding second coding block in the reference picture, and taking the second coding block with the highest SIFT feature matching degree as the best matching block;

[0008] Determine a rotation angle and block index information based on the best matching block;

[0009] Encoding is performed based on the rotation angle and the block index information.

[0010] Optionally, the step of determining a first coding block carrying target object information in the picture to be encoded includes:

[0011] Obtain a picture to be encoded, and divide the picture to be encoded into blocks to obtain blocks;

[0012] Target object detection is performed on the blocks to obtain a first coding block carrying target object information.

[0013] Optionally, when the first coding block needs to be accurately encoded, before the step of determining a reference picture of the picture to be encoded, the method includes:

[0014] Determine the current number of available bytes of the first coding block;

[0015] When the available number of bytes is less than a preset number of bytes, it is determined that the first coding block needs to be precisely encoded.

[0016] Optionally, after the step of determining the first coding block carrying target object information in the picture to be encoded, the method includes:

[0017] When the first coding block does not need to be precisely coded, a motion search is performed in the reference picture to determine a second coding block having a minimum SAD value with the first coding block as the best matching block.

[0018] Optionally, the step of performing motion search in the reference picture to determine a second coding block having a minimum SAD value with the first coding block as the most matching target coding block includes:

[0019] Determining identification information of the target object;

[0020] Based on the identification information, a motion search is performed in the reference picture, and a second coding block having a minimum SAD value with the first coding block is determined as the best matching block.

[0021] Optionally, the step of performing scale invariant feature transform (SIFT) feature matching on the first coding block and corresponding second coding blocks in the reference picture, and taking the second coding block with the highest SIFT feature matching degree as the best matching block includes:

[0022] Determine, in the reference picture, each second coding block corresponding to the position of the first coding block, and perform filtering processing on the second coding block to obtain a filtered coding block;

[0023] Based on the filtered coding block, a fitting key point is obtained, and based on the fitting key point and an initial key point in the reference picture, a target key point is determined;

[0024] Determine the main direction and peak direction of the target key point;

[0025] Based on the main direction and the peak direction of the target key point, the second encoding block with the highest feature matching degree is determined as the best matching block.

[0026] Optionally, the step of encoding based on the rotation angle and the block index information includes:

[0027] Based on the rotation angle and the block index information, the picture to be encoded is encoded to obtain first encoding information, wherein the best matching block has been encoded in the reference picture, and the encoding information corresponding to the best matching block is second encoding information;

[0028] The first coding information is sent to a preset decoding end, so that the preset decoding end restores the to-be-coded picture based on the first coding information and the second coding information.

[0029] In addition, to achieve the above objectives, the present application also provides an AI-based video processing device, the device comprising:

[0030] A first determining module, used to determine a first coding block carrying target object information in the picture to be encoded;

[0031] A second determination module is used to determine a reference picture of the picture to be encoded when the first coding block needs to be accurately encoded, perform scale invariant feature transform (SIFT) feature matching between the first coding block and each corresponding second coding block in the reference picture, and take the second coding block with the highest SIFT feature matching degree as the best matching block;

[0032] A third determination module, configured to determine a rotation angle and block index information based on the best matching block;

[0033] An encoding module is used to perform encoding based on the rotation angle and the block index information.

[0034] In addition, to achieve the above-mentioned objectives, the present application also provides an AI-based video processing device, which includes a memory, a processor, and an AI-based video processing program stored in the memory and executable on the processor, and the AI-based video processing program, when executed by the processor, implements the steps of the AI-based video processing method as described above.

[0035] In addition, to achieve the above-mentioned objectives, the present application also provides a computer-readable storage medium, on which an AI-based video processing program is stored. When the AI-based video processing program is executed by a processor, the steps of the AI-based video processing method as described above are implemented.

[0036] The present application provides an AI-based video processing method, apparatus, device and storage medium. Compared with the prior art in which the residual between the current block and the reference block is encoded, it is difficult to accurately find the reference block in the case of rotation, resulting in the inability to effectively encode the rotation of the target object. In the present application, a first coding block carrying target object information in the picture to be encoded is determined; when the first coding block needs to be accurately encoded, a reference picture of the picture to be encoded is determined, and the first coding block is matched with each corresponding second coding block in the reference picture using scale-invariant feature transform (SIFT) features, and the second coding block with the highest SIFT feature matching degree is used as the most matched block; the rotation angle and block index information are determined based on the most matched block; and encoding is performed based on the rotation angle and the block index information. It can be understood that for the first coding block carrying the target object information, when precise encoding is required, the best matching block is accurately determined based on the SIFT feature matching method, and then, based on the accurately determined best matching block, the rotation angle and block index information of the first coding block are accurately obtained, so as to realize encoding of the image to be encoded based on the rotation angle and the block index information, obtain the first coding information, and then realize effective encoding of the target object in the rotated state. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0039] Figure 1 This is a flowchart of the first embodiment of the AI-based video processing method of the present application;

[0040] Figure 2 This is a schematic diagram of the detailed step flow of step S10 in the AI-based video processing method of this application;

[0041] Figure 3 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0042] Figure 4 Schematic diagram of the scenario involved in the AI-based video processing method of this application.

[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The present application embodiment provides an AI-based video processing method. In the first embodiment of the AI-based video processing method of the present application, refer to Figure 1 , the AI-based video processing method includes:

[0046] Step S10, determining a first coding block carrying target object information in the picture to be encoded;

[0047] Step S20, when the first coding block needs to be accurately coded, determining a reference picture of the picture to be coded, performing scale invariant feature transform (SIFT) feature matching between the first coding block and each corresponding second coding block in the reference picture, and taking the second coding block with the highest SIFT feature matching degree as the best matching block;

[0048] Step S30, determining a rotation angle and block index information based on the best matching block;

[0049] Step S40: Encoding based on the rotation angle and the block index information.

[0050] In this embodiment, the AI-based video processing method is applied to an AI-based video processing device, the AI-based video processing device belongs to an AI-based video processing device (specifically an encoding end), the AI-based video processing device belongs to an AI-based video processing system, and the AI-based video processing device belongs to an AI-based video processing system including an encoding end and a decoding end.

[0051] It should be noted that there are more and more application scenarios for videos, and video encoding algorithms are becoming more and more important. Video encoding mainly involves compressing images into videos. The industry already has encoding standards such as H264 / H265, so that any encoder that encodes according to the video encoding standard will be correctly decoded by other third-party decoders.

[0052] However, there are many problems with the compression quality of video encoding. If the compression is too much, a lower bit rate can be obtained, which occupies less bandwidth, but the decoded image quality will be very poor. If the video encoding compression ratio is low, good image quality can be obtained, but the bit rate will be very high and the bandwidth occupied will be very large. How to compress the bit rate while ensuring the image quality is a solution that must be considered by the encoding end, which will greatly improve the competitiveness of the product and enhance the user experience.

[0053] At present, a general coding scheme is usually used to encode videos. Specifically, the current image is fixedly divided into blocks, and then the current block is matched with the blocks in the previous or next frame of the image, and the block with the smallest error is found as the reference block of the current block of the current image, and then the residual between the current block and the reference block is encoded.

[0054] Encoding the residual between the current block and the reference block is effective for target objects that translate in the current block. However, when the target object in the current block rotates, it is often impossible to determine the most matching reference block. In other words, the residual between the found reference block and the current block will also be large. The large residual between the current block and the reference block increases the bit rate of the bitstream, resulting in a decrease in image quality and reducing the competitiveness of audio and video products.

[0055] In order to solve the problems of the prior art, in this embodiment, the picture is first divided into blocks, and then it is determined whether the currently divided blocks need to be accurately encoded. When accurate encoding is required, the SIFT area is determined based on the SIFT feature matching method, and the best matching block is determined according to the SAD values ​​corresponding to each block in the SIFT area and the first coding block. After determining the best matching block, the rotation angle and block index information are determined based on the best matching block, and the rotation angle and block index information are placed in the SEI information for video compression encoding.

[0056] The specific steps are as follows:

[0057] Step S10, determining a first coding block carrying target object information in the picture to be encoded;

[0058] In this embodiment, the picture to be encoded includes one or more target objects, or the picture to be encoded may not include any target object.

[0059] In this embodiment, if Figure 4 As shown, the picture to be encoded is a P frame type picture, and the IDR frame type picture is encoded according to the preset video encoding standard encoding method.

[0060] In this embodiment, after the picture to be encoded is obtained, the picture to be encoded is divided into blocks to determine the first encoding block carrying the target object information in the picture to be encoded.

[0061] In this embodiment, there may be multiple first coding blocks.

[0062] Among them, refer to Figure 2 The step of determining the first coding block carrying the target object information in the picture to be coded includes:

[0063] Step S11, obtaining a picture to be encoded, and dividing the picture to be encoded into blocks to obtain blocks;

[0064] Step S12: performing target object detection on the blocks to obtain a first coding block carrying target object information.

[0065] In this embodiment, after obtaining the picture to be encoded, the picture to be encoded is divided into blocks to obtain blocks. After obtaining the blocks, target object detection is performed on the picture to be encoded through a preset target object detection model to obtain a first coding block carrying target object information.

[0066] For example, after performing target object detection on the picture to be encoded, target object R1, target object R2, and target object Rx are obtained.

[0067] In this embodiment, when dividing the first target object into blocks, the current image is divided into 16×16 blocks, so that the target object can be divided into one block when dividing the blocks.

[0068] In this embodiment, the index of the first coding block where each target object is located is marked, such as R1B15 indicating that the block where the first target object is located is the 15th block (wherein the blocks are indexed sequentially from left to right and from top to bottom).

[0069] Step S20, when the first coding block needs to be accurately coded, determining a reference picture of the picture to be coded, performing scale invariant feature transform (SIFT) feature matching between the first coding block and each corresponding second coding block in the reference picture, and taking the second coding block with the highest SIFT feature matching degree as the best matching block;

[0070] In this embodiment, it is described how to determine the best matching block when the first coding block needs to be precisely coded.

[0071] Wherein, when the first coding block needs to be accurately encoded, before the step of determining the reference picture of the picture to be encoded, the method includes:

[0072] Step A1, determining the current number of available bytes of the first coding block;

[0073] Step A2: When the number of available bytes is less than a preset number of bytes, it is determined that the first coding block needs to be precisely encoded.

[0074] In this embodiment, the current number of available bytes of the first coding block is determined, wherein the current number of available bytes of the first coding block is determined in the following manner:

[0075] Assume that the number of pre-allocated available bytes of the current picture to be encoded is BT, and the number of blocks of the current picture to be encoded is N, then the average bit rate or available bytes of each block is BT / N. Assume that the current first encoding block is the Ai block, and assume that each block actually consumes Bi bytes. The bytes currently used are sum(Ai*Bi), and the number of remaining bytes is BT-sum(Ai*Bi), then the actual number of bytes available for the current block is Bri=(BT-sum(Ai*Bi)) / (Ni).

[0076] When the available number of bytes is less than the preset number of bytes, it is determined that the first coding block needs to be precisely encoded. Specifically, if Bri is less than BT / N*4 / 5 (preset number of bytes), it means that the current macroblock needs to be precisely encoded.

[0077] It should be noted that since Bri is less than the preset number of bytes, it indicates that the current bit rate or the available number of bytes is insufficient. Since the preset number of bytes is insufficient, it is necessary to save the preset number of bytes. In this embodiment, it is necessary to increase the compression ratio of the coding block through precise coding (precise coding takes more time, and by increasing the time consumption, the consumption of bytes is reduced) to save the preset number of bytes.

[0078] Obviously, when the number of available bytes is less than the preset number of bytes, it indicates that the number of bytes or bit rate used by the coding blocks before the first coding block is too many, and the first coding block where the current target object is located needs to save bit rate. At this time, it is assumed that the target object in the current picture to be encoded and the target object in the previous reference picture are rotated. In addition, when the target object in the current picture to be encoded and the target object in the previous reference picture are translated, it is also a special case of target object rotation, but the rotation angle is 0.

[0079] In this embodiment, when the first coding block needs to be accurately encoded, a reference picture of the picture to be encoded is determined, wherein the reference picture of the picture to be encoded is a picture of a previous frame of the picture to be encoded.

[0080] Perform scale invariant feature transform (SIFT) feature matching on the first coding block and the corresponding second coding blocks in the reference image, take the second coding block with the highest SIFT feature matching as the best matching block, and determine the rotation angle and block index information based on the best matching block.

[0081] The process of performing scale-invariant feature transform (SIFT) feature matching may include:

[0082] 1. Construct scale space, detect extreme points, and obtain scale invariance;

[0083] 2. Filter and accurately locate the key points of the target based on extreme points and scale space;

[0084] 3. Assign direction values ​​to target key points;

[0085] 4. Generate feature vectors.

[0086] A 16×16 neighborhood is taken as the sampling window with the feature point as the center. The relative directions of the sampling point and the target key point are classified into a direction histogram containing 8 bins after Gaussian weighting, and finally a 4×4×8 128-dimensional feature vector is obtained.

[0087] Specifically, the step of performing scale invariant feature transform (SIFT) feature matching on the first coding block and the corresponding second coding blocks in the reference picture, and taking the second coding block with the highest SIFT feature matching degree as the best matching block includes:

[0088] Step S21, determining each second coding block corresponding to the position of the first coding block in the reference picture, and performing filtering processing on the second coding block to obtain a filtered coding block;

[0089] Step S22, obtaining fitting key points based on the filtered coding block, and determining target key points based on the fitting key points and initial key points in the reference picture;

[0090] Step S23, determining the main direction and peak direction of the target key point;

[0091] Step S24: Based on the main direction and the peak direction of the target key point, determine the second encoding block with the highest feature matching degree as the best matching block.

[0092] In this embodiment, the position correspondence can be consistent position, or the position deviation can be within a preset area, and SIFT feature matching is performed on the first coding block and the blocks corresponding to different positions in the reference image. Specifically, in this embodiment, the image Gaussian pyramid (Gaussian Pyramid) in SIFT feature matching uses a Gaussian function, and the blocks of the reference image are blurred and downsampled by the Gaussian function to obtain a filtered coding block.

[0093] The fitting key points or fitting extreme points of the filter coding block are determined by using the method of determining the local extreme points of the DOG space, and then the offsets between the fitting key points or fitting extreme points and the initial key points are determined. If any offset exceeds 0.5 (the preset offset value), it means that the fitting key point should be in the adjacent position of the initial key point.

[0094] Based on the fitted key points and the initial key points in the reference picture, the target key points are determined. The fitted key points and the initial key points in the reference picture can be used as target key points, or target key points can be selected from the fitted key points and the initial key points in the reference picture.

[0095] A preset gradient calculation is performed on the target key point. After the gradient calculation of the target key point is completed, a histogram is used to count the gradient and direction of the pixels in the preset neighborhood of the target key point, where the gradient histogram divides the direction range of 0 to 360 degrees into 36 columns, with each column being 10 degrees.

[0096] Among them, the peak direction of the directional histogram represents the direction of the target key point. Specifically, the peak of the directional histogram represents the direction of the preset neighborhood gradient of the target key point, and the maximum value in the histogram is taken as the main direction of the key point.

[0097] The preset range area where the target key point is located is divided so that each target key point corresponds to a second coding block. Combined with the main direction and peak direction of the target key point, a SIFT area of ​​the second coding block is determined, where each SIFT area carries a 128-dimensional feature vector.

[0098] The 128-dimensional feature vectors of each block in the SIFT region are compared, and the second encoded block with the highest eigenvalue Vx is taken as the best matching block.

[0099] In this embodiment, after obtaining the best matching block, the entire pixel point of the best matching block is rotated clockwise to obtain a new 16x16 block Ax. Ax is used to perform SAD calculation with the first coding block, and the calculated SAD is definitely the smallest, that is, the true best matching block Bx is obtained.

[0100] Step S30, determining a rotation angle and block index information based on the best matching block;

[0101] After the best matching block is determined, the best matching block is compared with the first encoding block to obtain the rotation angle.

[0102] In this embodiment, since the index of the first coding block where each target object is located is marked, such as R1B15, which indicates that the block where the first target object is located is the 15th block (wherein the blocks are indexed sequentially from left to right and from top to bottom), the block index information can be determined.

[0103] Step S40: Encoding based on the rotation angle and the block index information.

[0104] The step of encoding based on the rotation angle and the block index information includes:

[0105] Based on the rotation angle and the block index information, the picture to be encoded is encoded to obtain first encoding information, wherein the best matching block has been encoded in the reference picture, and the encoding information corresponding to the best matching block is second encoding information;

[0106] The first coding information is sent to a preset decoding end, so that the preset decoding end restores the to-be-coded picture based on the first coding information and the second coding information.

[0107] In this embodiment, the picture to be encoded is encoded based on the rotation angle and the block index information to obtain first encoding information, wherein the best matching block has been encoded in the reference picture, and the encoding information corresponding to the best matching block is the second encoding information, thereby saving or presetting a preset number of bytes.

[0108] Specifically, for example, the image of the Bx block found in step S20 is rotated to obtain Bx1, and the index and angle of the Bx block are marked as (Bx, Ax), and placed in the SEI information of the video stream, and the first encoding information is sent to the preset decoding end, so that the preset decoding end can restore the image to be encoded based on the first encoding information and the second encoding information.

[0109] The present application provides an AI-based video processing method, apparatus, device and storage medium. Compared with the prior art in which the residual between the current block and the reference block is encoded, it is difficult to accurately find the reference block in the case of rotation, resulting in the inability to effectively encode the rotation of the target object. In the present application, a first coding block carrying target object information in the picture to be encoded is determined; when the first coding block needs to be accurately encoded, a reference picture of the picture to be encoded is determined, and the first coding block is matched with each corresponding second coding block in the reference picture using scale-invariant feature transform (SIFT) features, and the second coding block with the highest SIFT feature matching degree is used as the most matched block; the rotation angle and block index information are determined based on the most matched block; and encoding is performed based on the rotation angle and the block index information. It can be understood that for the first coding block carrying the target object information, when precise encoding is required, the best matching block is accurately determined based on the SIFT feature matching method, and then, based on the accurately determined best matching block, the rotation angle and block index information of the first coding block are accurately obtained, so as to realize encoding of the image to be encoded based on the rotation angle and the block index information, obtain the first coding information, and then realize effective encoding of the target object in the rotated state.

[0110] Further, based on the above embodiment of the present application, another embodiment of the present application is provided. In this embodiment, after the step of determining the current number of available bytes of the first coding block, the method includes:

[0111] Step B1: When the first coding block does not need to be accurately coded, a motion search is performed in the reference picture to determine a second coding block having a minimum SAD value with the first coding block as the best matching block.

[0112] In this embodiment, when the number of available bytes is greater than or equal to a preset number of bytes, it is determined that the first coding block does not need to be precisely encoded. When the first coding block does not need to be precisely encoded, a motion search is performed within the reference picture based on the first coding block to determine the second coding block with the smallest SAD value with the first coding block as the best matching block.

[0113] Specifically, based on the first coding block, a motion search is performed in the reference picture to determine the second coding block with the smallest SAD value (SAD, Sum of Absolute Difference) of the first coding block. The process can be: randomly determine a plurality of second coding blocks with the same size as the first coding block in the reference picture, and then compare the plurality of second coding blocks with the first coding block respectively to obtain the second coding block with the smallest SAD value with the first coding block, and use the second coding block where the second coding block is located as the best matching target coding block.

[0114] The step of performing motion search in the reference picture and determining a second coding block with a minimum SAD value with the first coding block as the most matching target coding block comprises:

[0115] Step C1, determining identification information of the target object;

[0116] Step C2: Based on the identification information, a motion search is performed in the reference picture, and a second coding block having a minimum SAD value with the first coding block is determined as the best matching block.

[0117] In this embodiment, the identification information of the first coding block is determined. Specifically, the identification information of the target object in the first coding block may be determined. Based on the identification information of the target object, a motion search is performed in the reference image to determine the second coding block having the smallest SAD value with the first coding block.

[0118] That is, firstly, the second coding block including the target object identifier in the reference picture is determined, and then the second coding block having the smallest SAD value with the first coding block can be quickly found.

[0119] Specifically, before searching, first query the ID of the current 16x16 first coding block or the ID of the corresponding target object, and then correspond to the corresponding target object ID of the reference image according to the ID of the first coding block or the ID of the corresponding target object, and then obtain the second coding block of the reference image corresponding to the target object.

[0120] In this embodiment, the second coding block is quickly found, thereby improving the coding efficiency.

[0121] Further, based on the above embodiments of the present application, another embodiment of the present application is provided. In this embodiment, applied to a decoding end, the AI-based video processing method includes the following steps:

[0122] Step S60, when decoding a first coding block of the picture to be decoded, obtaining a rotation angle and block index information corresponding to the first coding block;

[0123] Step S70: rotating the first coding block based on the rotation angle and the block index information to obtain a rotated first coding block;

[0124] Step S80, decoding the rotated first coding block to finally obtain a decoded picture.

[0125] In this embodiment, the decoding end obtains the picture to be decoded in the video stream, and then decodes the SEI information of the current video stream, and then obtains the information (Bx, Ax) list of each divided block of the current frame, where Bx represents the coordinates of the block using rotation matching (block index information), and Ax represents the rotation angle value of the block, and then based on the rotation angle and the block index information, the first coding block is rotated to obtain the rotated first coding block. Specifically, the residual value of the rotated first coding block is determined, and the residual value of the block is added to the second coding information to obtain the pixel value of the current first coding block.

[0126] In this embodiment, the decoding end can accurately decode the first coding block of the picture to be decoded.

[0127] Reference Figure 3 , Figure 3 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application.

[0128] like Figure 3As shown, the AI-based video processing device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005. The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 can also be a storage device independent of the aforementioned processor 1001.

[0129] Optionally, the AI-based video processing device may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. The rectangular user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0130] Those skilled in the art will understand that Figure 3 The AI-based video processing device structure shown in the figure does not constitute a limitation on the AI-based video processing device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0131] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, and an AI-based video processing program. The operating system is a program that manages and controls the hardware and software resources of the AI-based video processing device, and supports the operation of the AI-based video processing program and other software and / or programs. The network communication module is used to realize communication between the components inside the memory 1005, and communication with other hardware and software in the AI-based video processing system.

[0132] exist Figure 3 In the AI-based video processing device shown, the processor 1001 is used to execute the AI-based video processing program stored in the memory 1005 to implement the steps of any of the above-mentioned AI-based video processing methods.

[0133] The specific implementation of the AI-based video processing device of the present application is basically the same as the above-mentioned embodiments of the AI-based video processing method, and will not be repeated here.

[0134] The present application also provides an AI-based video processing device, the AI-based video processing device comprising:

[0135] A first determining module, used to determine a first coding block carrying target object information in the picture to be encoded;

[0136] A second determination module is used to determine a reference picture of the picture to be encoded when the first coding block needs to be accurately encoded, perform scale invariant feature transform (SIFT) feature matching between the first coding block and each corresponding second coding block in the reference picture, and take the second coding block with the highest SIFT feature matching degree as the best matching block;

[0137] A third determination module, configured to determine a rotation angle and block index information based on the best matching block;

[0138] An encoding module is used to perform encoding based on the rotation angle and the block index information.

[0139] Optionally, the first determining module includes:

[0140] A first acquisition unit is used to acquire a picture to be encoded, and divide the picture to be encoded into blocks to obtain blocks;

[0141] The second acquisition unit is used to perform target object detection on the block to obtain a first coding block carrying target object information.

[0142] Optionally, the device comprises:

[0143] A fourth determining module, used to determine the current number of available bytes of the first coding block;

[0144] The fifth determination module is used to determine that the first coding block needs to be precisely encoded when the number of available bytes is less than a preset number of bytes.

[0145] Optionally, the device further comprises:

[0146] The sixth determination module is used to perform a motion search in the reference picture when the first coding block does not need to be accurately encoded, and determine the second coding block with the smallest SAD value with the first coding block as the best matching block.

[0147] Optionally, the sixth determining module includes:

[0148] A first determining unit, configured to determine identification information of the target object;

[0149] A second determining unit is configured to perform a motion search in the reference picture based on the identification information, and determine a second coding block having a minimum SAD value with the first coding block as the best matching block.

[0150] Optionally, the second determining module includes:

[0151] a filtering unit, configured to determine, in the reference picture, each second coding block corresponding to the position of the first coding block, and perform filtering processing on the second coding block to obtain a filtered coding block;

[0152] A third acquisition unit, configured to obtain a fitting key point based on the filtered coding block, and determine a target key point based on the fitting key point and an initial key point in the reference picture;

[0153] A third determining unit, used to determine the main direction and the peak direction of the target key point;

[0154] Based on the main direction and the peak direction of the target key point, the second encoding block with the highest feature matching degree is determined as the best matching block.

[0155] Optionally, the encoding module includes:

[0156] an encoding unit, configured to encode the picture to be encoded based on the rotation angle and the block index information to obtain first encoding information, wherein the best matching block has been encoded in the reference picture, and the encoding information corresponding to the best matching block is second encoding information;

[0157] A sending unit is used to send the first coding information to a preset decoding end, so that the preset decoding end can restore the to-be-coded picture based on the first coding information and the second coding information.

[0158] The specific implementation of the AI-based video processing device of the present application is basically the same as the above-mentioned embodiments of the AI-based video processing method, and will not be repeated here.

[0159] The embodiment of the present application provides an AI-based video processing system, and the AI-based video processing system includes:

[0160] The encoding end is used to obtain a picture to be encoded, divide the target object in the picture to be encoded into blocks, and obtain a first encoding block; determine the current number of available bytes of the first encoding block, and when the number of available bytes is less than a preset number of bytes, determine that the first encoding block needs to be accurately encoded; and also used to determine the reference picture of the picture to be encoded when the first encoding block needs to be accurately encoded, perform scale-invariant feature transform (SIFT) feature matching between the first encoding block and each corresponding second encoding block in the reference picture, take the second encoding block with the highest SIFT feature matching as the most matching block, and determine the rotation angle and block index information based on the most matching block; and also used to encode the picture to be encoded based on the rotation angle and the block index information to obtain first encoding information, wherein the most matching block has been encoded in the reference picture, and the encoding information corresponding to the most matching block is the second encoding information; and also used to send the first encoding information to a preset decoding end, so that the preset decoding end can restore the picture to be encoded based on the first encoding information and the second encoding information;

[0161] The decoding end is used to obtain the rotation angle and block index information corresponding to the first coding block of the to-be-decoded picture when decoding the first coding block; based on the rotation angle and the block index information, the first coding block is rotated to obtain the rotated first coding block; the rotated first coding block is decoded to finally obtain a decoded picture.

[0162] The specific implementation of the AI-based video processing system of the present application is basically the same as the above-mentioned embodiments of the AI-based video processing method, and will not be repeated here.

[0163] An embodiment of the present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to implement the steps of any of the above-mentioned AI-based video processing methods.

[0164] The specific implementation of the storage medium of the present application is basically the same as the above-mentioned embodiments of the AI-based video processing method, and will not be repeated here.

[0165] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned AI-based video processing method when executed by a processor.

[0166] The specific implementation of the computer program product of the present application is basically the same as the above-mentioned AI-based video processing method embodiments, and will not be repeated here.

[0167] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0168] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0170] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A video processing method based on AI, characterized in that: Applied to the encoding end, the AI-based video processing method includes the following steps: Determine a first coding block carrying target object information in the picture to be coded; When the first coding block needs to be accurately coded, determining a reference picture of the picture to be coded, performing scale invariant feature transform (SIFT) feature matching between the first coding block and each corresponding second coding block in the reference picture, and taking the second coding block with the highest SIFT feature matching degree as the best matching block; wherein the best matching block has been coded in the reference picture; Determine a rotation angle and block index information based on the best matching block; Encoding is performed based on the rotation angle and the block index information.

2. The AI-based video processing method according to claim 1, characterized in that: The step of determining a first coding block carrying target object information in the picture to be coded includes: Obtain a picture to be encoded, and divide the picture to be encoded into blocks to obtain blocks; Target object detection is performed on the blocks to obtain a first coding block carrying target object information.

3. The AI-based video processing method according to claim 1, characterized in that: When the first coding block needs to be accurately coded, before the step of determining the reference picture of the picture to be coded, the method includes: Determine the current number of available bytes of the first coding block; When the available number of bytes is less than a preset number of bytes, it is determined that the first coding block needs to be precisely encoded.

4. The AI-based video processing method according to claim 1, characterized in that: After the step of determining the first coding block carrying the target object information in the picture to be coded, the method includes: When the first coding block does not need to be precisely coded, a motion search is performed in the reference picture to determine a second coding block having a minimum SAD value with the first coding block as the best matching block.

5. The AI-based video processing method according to claim 4, characterized in that: The step of performing motion search in the reference picture and determining a second coding block with a minimum SAD value with the first coding block as the most matching target coding block comprises: Determining identification information of the target object; Based on the identification information, a motion search is performed in the reference picture, and a second coding block having a minimum SAD value with the first coding block is determined as the best matching block.

6. The AI-based video processing method according to claim 1, characterized in that: The step of performing scale invariant feature transform (SIFT) feature matching on the first coding block and the corresponding second coding blocks in the reference picture, and taking the second coding block with the highest SIFT feature matching degree as the best matching block, comprises: Determine, in the reference picture, each second coding block corresponding to the position of the first coding block, and perform filtering processing on the second coding block to obtain a filtered coding block; Based on the filtered coding block, a fitting key point is obtained, and based on the fitting key point and an initial key point in the reference picture, a target key point is determined; Determine the main direction and peak direction of the target key point; Based on the main direction and the peak direction of the target key point, the second encoding block with the highest feature matching degree is determined as the best matching block.

7. The AI-based video processing method according to claim 1, characterized in that: The step of encoding based on the rotation angle and the block index information comprises: Based on the rotation angle and the block index information, the picture to be encoded is encoded to obtain first encoding information, wherein the best matching block has been encoded in the reference picture, and the encoding information corresponding to the best matching block is second encoding information; The first coding information is sent to a preset decoding end, so that the preset decoding end restores the to-be-coded picture based on the first coding information and the second coding information.

8. An AI-based video processing device, characterized in that: The AI-based video processing device includes: A first determining module, used to determine a first coding block carrying target object information in the picture to be encoded; A second determination module is configured to determine a reference picture of the picture to be encoded when the first coding block needs to be accurately encoded, perform scale invariant feature transform (SIFT) feature matching between the first coding block and each corresponding second coding block in the reference picture, and use the second coding block with the highest SIFT feature matching degree as the best matching block; wherein the best matching block has been encoded in the reference picture; A third determination module, configured to determine a rotation angle and block index information based on the best matching block; An encoding module is used to perform encoding based on the rotation angle and the block index information.

9. An AI-based video processing device, characterized in that: The AI-based video processing device includes a memory, a processor, and an AI-based video processing program stored in the memory and executable on the processor, and the AI-based video processing is configured to implement the steps of the AI-based video processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an AI-based video processing program, which, when executed by a processor, implements the steps of the AI-based video processing method according to any one of claims 1 to 7.

Images