A motion compensation-based video processing method and image device thereof

Abstract

The present application belongs to the technical field of video processing, and particularly relates to a video processing method based on motion compensation and an image device thereof. The present application adopts the design of adjusting the reference object and reconstructing the background of the replacement frame. When the video image is processed, the size of the reference object can be determined according to the existing video frames before and after, then the reference object in the replacement frame is adjusted according to the adjustment ratio, so that the proportion conforms to the image proportion in the target video source, and then the background of the replacement frame is reconstructed according to the background information of the existing video frames before and after, so that the reconstructed replacement frame can be obtained by combining the two, and then it is inserted into the first continuous video frame, so that the picture has good coherence when playing, and the proportion is consistent, and people will not have a sense of jumping when watching, and accordingly, the watchability is improved, and the process can also be well performed by non-professionals.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology, specifically relating to a video processing method and image device based on motion compensation. Background Technology

[0002] In recent years, with the rapid rise of short videos, more and more people are using short videos to share and record interesting things in their lives and share them with others. In order to improve the viewing experience, the videos often need to be optimized before being shared. Moreover, short videos are continuous dynamic images, and it is easy for the images to be disjointed and unclear during processing. Therefore, motion compensation technology is needed to process them so that the short videos have a better quality when played.

[0003] In existing technologies, video processing often involves direct deletion and replacement. For non-professionals, it is difficult to achieve refined processing, resulting in poor video continuity and a sense of disjointedness for viewers, thus reducing the viewing experience. Summary of the Invention

[0004] The purpose of this invention is to provide a video processing method and image device based on motion compensation, which can adjust the reference objects in the replacement frame and reconstruct the background accordingly, so that the processed video has high continuity and thus improves its viewing experience.

[0005] The specific technical solution adopted by this invention is as follows:

[0006] A video processing method based on motion compensation includes:

[0007] Obtain the first consecutive video frames of the target video source;

[0008] The first consecutive video frames are optimized according to the optimization requirements to obtain the second consecutive video frames. The optimization process includes replacing frames and deleting frames.

[0009] Obtain the common reference point between the replacement frame and the first consecutive video frame;

[0010] Based on the proportion of shared reference objects, the reference objects in the replacement frame are adjusted, and the background in the replacement frame is reconstructed based on the adjustment of the reference objects;

[0011] Obtain multiple optimized nodes from the total number of frames in the second consecutive video frame, and establish multiple discontinuity points based on the optimized nodes to obtain multiple discontinuous video sources;

[0012] The image information of the first and last frames of adjacent intermittent video sources is obtained, and the image information of the first and last frames is substituted into the matching degree calculation formula to obtain the image matching degree of the first and last frames of adjacent intermittent video sources.

[0013] Obtain the standard threshold for image matching;

[0014] Determine whether the image matching degree falls within the standard threshold of image matching degree;

[0015] If the image matching degree is within the standard threshold, it indicates that the adjacent discontinuous video sources are continuous. If the image matching degree value is not within the standard threshold, it indicates that the adjacent discontinuous video sources are not continuous. Motion compensation processing is then performed on the first and last frames at the discontinuity points to obtain the third continuous video frame.

[0016] In a preferred embodiment, the step of acquiring the first consecutive video frames of the target video source includes:

[0017] The encoder decodes the target video source to obtain the video signal;

[0018] The first video signal is transmitted to the controller through a data interface. The controller extracts video frames based on image features, wherein the image features include at least color features, texture features, shape features, and spatial relationship features.

[0019] The extracted video frames are arranged consecutively to obtain the first consecutive video frame.

[0020] In a preferred embodiment, the step of optimizing the first consecutive video frames according to optimization requirements to obtain the second consecutive video frames includes:

[0021] Obtain the video frame to be optimized from the first consecutive video frames;

[0022] Obtain optimization options, and delete and replace the video frames to be optimized according to the optimization options to obtain the deleted frames and the replacement frames;

[0023] Backup the deleted frames;

[0024] The replacement frame is inserted into the video frame to be optimized to obtain a second consecutive video frame, wherein the replacement frame can be inserted at any position in the first consecutive video frame.

[0025] In a preferred embodiment, the step of adjusting the reference objects in the replacement frame according to the proportion of shared reference objects includes:

[0026] The images in the replacement frame and the first consecutive video frame are compared to obtain the overlapping blocks and identified as common reference objects.

[0027] Extract the attribute information of the common reference object, wherein the attribute information includes scale information;

[0028] The adjustment ratio of the common reference object is determined based on the ratio information, and the image of the replacement frame is adjusted according to the adjustment ratio to obtain the reconstructed frame;

[0029] The formula for calculating the adjustment ratio is: S = I(x) i-1 ,y i-1 ) / B(x i ,y i In the formula, S represents the adjustment ratio, and I(x) i-1 ,y i-1 B(x) represents the reference image of the frame preceding the replacement frame. i ,y i () represents the reference image of the replacement frame.

[0030] In a preferred embodiment, the step of reconstructing the background in the replacement frame based on the adjustment of the reference object includes:

[0031] Establish a sampling cycle;

[0032] During the sampling period, first background information of multiple determined frames at the front and back ends of the reconstructed frame is obtained from the first consecutive video frames;

[0033] Obtain the second background information of the reconstructed frame;

[0034] Substituting the first and second background information into the reconstruction model, the third background information of the reconstructed frame is obtained. The calculation formula in the reconstruction model is as follows: In the formula, I(x) t-1 ,y t-1 P(x) represents the front-end defined frame. t+1 ,y t+1 ) indicates a backend-determined frame;

[0035] The reconstructed frame is inserted between the front-end determined frame and the back-end determined frame to obtain a second consecutive video frame.

[0036] In a preferred embodiment, the steps of obtaining multiple optimized nodes from the total number of frames in the second consecutive video frames, and establishing multiple discontinuities based on the optimized nodes to obtain multiple discontinuous video sources, include:

[0037] Obtain the positions of the replacement and deletion frames in the second consecutive video frames;

[0038] The nodes before and after the replacement frame and the position of the deleted frame are determined as optimization nodes;

[0039] Starting with the first frame of the second consecutive video frame, and using the first optimized node as the first discontinuity, an intermittent video source is established. Subsequently, intermittent video sources are established with every two adjacent optimized nodes as discontinuities.

[0040] In a preferred embodiment, the step of obtaining image information of the first and last frames of adjacent intermittent video sources, and substituting the image information of the first and last frames into the matching degree calculation formula to obtain the image matching degree of the first and last frames of adjacent intermittent video sources includes:

[0041] Obtain the first frame image information of the current intermittent video source and the last frame image of the previous intermittent video source;

[0042] The first and last frame images are compressed to remove detail points, thus obtaining the basic structural information of the first and last frame images.

[0043] Calculate the average grayscale value of all pixels in the compressed first and last frames, respectively, using the following formula: In the formula, Z represents the average grayscale value, a represents all possible grayscale values ​​(taking positive integer values), MN represents the total number of pixels, and R... j E represents the number of times a grayscale value appears in a pixel. j Indicates pixel grayscale;

[0044] The difference between the average grayscale values ​​of the first and last frames in the first and last frames of the adjacent intermittent video sources is calculated to obtain the image matching degree between the first and last frames.

[0045] In a preferred embodiment, the step of determining whether the image matching degree falls within the image matching degree standard threshold includes:

[0046] The range of the image matching degree standard threshold is determined to be [0, 3];

[0047] If the value of the image matching degree is between [0, 3], it indicates that adjacent discontinuous video sources have continuity;

[0048] If the value of the image matching degree is outside the interval [0, 3], it indicates that the adjacent discontinuous video sources are not continuous, and motion compensation processing is performed on the first and last frames at the discontinuity points.

[0049] The motion compensation process includes one of the following: global motion compensation, inter-frame interpolation motion compensation, and hierarchical estimation algorithm.

[0050] The present invention also provides a motion-compensated video processing image device, applied to the above-mentioned video processing method, comprising:

[0051] A first acquisition module is used to acquire the first consecutive video frames of the first video source.

[0052] An optimization module is used to optimize a first consecutive video frame according to optimization requirements to obtain a second consecutive video frame, wherein the optimization process includes replacing frames and deleting frames.

[0053] The second acquisition module is used to acquire the common reference between the replacement frame and the first consecutive video frame;

[0054] A reconstruction module is used to adjust the reference objects in the replacement frame according to the proportion of the common reference objects, and to reconstruct the background in the replacement frame based on the adjustment of the reference objects.

[0055] The third acquisition module is used to acquire multiple optimized nodes in the total number of frames of the second consecutive video frames, and establish multiple discontinuity points based on the optimized nodes to obtain multiple discontinuous video sources.

[0056] The calculation module is used to obtain the image information of the first frame and the last frame of the adjacent intermittent video sources, and substitute the image information of the first frame and the last frame into the matching degree calculation formula to obtain the image matching degree of the first frame and the last frame of the adjacent intermittent video sources.

[0057] The fourth acquisition module is used to acquire the image matching degree standard threshold;

[0058] The comparison module is used to determine whether the image matching degree falls within the image matching degree standard threshold.

[0059] In a preferred embodiment, the system further includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the video processing method described above.

[0060] The technical effects achieved by this invention are as follows:

[0061] This invention employs a design that adjusts the reference object and reconstructs the background of the replacement frame. When processing video images, the size of the reference object can be determined based on the existing video frames before and after it. Then, the reference object in the replacement frame is adjusted according to the adjustment ratio to make its proportion match the image proportion in the target video source. At the same time, the background of the replacement frame is reconstructed based on the background information of the video frames before and after it. Combining the two, the reconstructed replacement frame can be obtained, which is then inserted into the first consecutive video frames. This results in good continuity and consistent proportions during playback, preventing viewers from experiencing any jarring transitions and thus enhancing the viewing experience. Furthermore, this process can be easily performed by non-professionals. Attached Figure Description

[0062] Figure 1This is a flowchart of a video processing method provided in an embodiment of the present invention;

[0063] Figure 2 This is a flowchart of the replacement frame optimization process provided in an embodiment of the present invention. Detailed Implementation

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0065] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0066] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0067] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0068] Please see the appendix Figure 1 This invention provides a video processing method based on motion compensation, comprising:

[0069] S1. Obtain the first consecutive video frame of the target video source;

[0070] S2. Optimize the first consecutive video frames according to the optimization requirements to obtain the second consecutive video frames. The optimization process includes replacing frames and deleting frames.

[0071] S3. Obtain the common reference point between the replacement frame and the first consecutive video frame;

[0072] S4. Adjust the reference objects in the replacement frame according to the proportion of the common reference objects, and reconstruct the background in the replacement frame based on the adjustment of the reference objects;

[0073] S5. Obtain multiple optimized nodes from the total number of frames in the second consecutive video frame, and establish multiple discontinuity points based on the optimized nodes to obtain multiple discontinuous video sources.

[0074] S6. Obtain the image information of the first and last frames of adjacent discontinuous video sources, substitute the image information of the first and last frames into the matching degree calculation formula, and obtain the image matching degree of the first and last frames of adjacent discontinuous video sources.

[0075] S7. Obtain the standard threshold for image matching degree;

[0076] S8. Determine whether the image matching degree falls within the standard threshold of image matching degree;

[0077] If the image matching degree is within the standard threshold, it indicates that the adjacent discontinuous video sources are continuous. If the image matching degree value is not within the standard threshold, it indicates that the adjacent discontinuous video sources are not continuous. Motion compensation processing is then performed on the first and last frames at the discontinuity points to obtain the third continuous video frame.

[0078] As described in steps S1-S8 above, when processing the target video source, it is first necessary to obtain the first consecutive video frames from the target video source. During the shooting process, blurry images or substandard segments inevitably appear, requiring optimization. This optimization process can be summarized as a deletion and replacement process: deleting blurry and substandard segments and replacing them with acceptable segments, then combining these segments with the original segments to obtain the second consecutive video frame. The segments formed by the replacement frames need to be re-recorded, and the recorded video also needs to be processed by extracting video frames. This process follows the same method as the target video source processing to obtain the replacement frame. During shooting, camera positions are often manually adjusted, so the image information in the replacement frame may not match the image information in the first consecutive video frame. Therefore, the inserted replacement frame needs to be adjusted according to the ratio. The background of the adjusted replacement frame will also change accordingly, requiring background reconstruction. This process is based on the actual shooting angle. For a green background, with a single background color, there's no need to consider background reconstruction. In this embodiment, multiple discontinuities are established in the optimized second consecutive video frames, resulting in multiple discontinuous video sources. If unqualified segments are found later, it's not necessary to decode all video sources; only the discontinuous video sources containing the problematic segments need to be decoded. It's also important to consider that video images are constantly changing dynamic images. After inserting replacement frames, there might be low matching rates, leading to missing frames. Therefore, motion compensation processing is needed for parts that don't match well to fill in the missing frames. Here, motion compensation is based on the video source; therefore, the front and back images of the video frames to be processed are known data. Based on this, motion compensation of the video frames to be processed can be supported by the front and back image data, thus better reflecting the effect of motion compensation. Combining the above operations, not only can video frames be deleted and replaced in the target video source, but the image data of the first consecutive video frames can also be used as support to optimize the replacement frames, making the final video source more realistic.

[0079] In a preferred embodiment, the step of acquiring the first consecutive video frames of the target video source includes:

[0080] S11. The encoder decodes the target video source to obtain the video signal;

[0081] S12. The first video signal is transmitted to the controller through the data interface. The controller extracts the video frame according to the image features, wherein the image features include at least color features, texture features, shape features and spatial relationship features.

[0082] S13. Arrange the extracted video frames together consecutively to obtain the first consecutive video frame.

[0083] As described in steps S11-S13 above, when decoding the target video source, the ITU-R 656 decoder can be used for decoding, and the data stream is transmitted based on the RTEP protocol. Finally, the controller extracts the data based on the image features. The extraction criteria include, but are not limited to, color features, texture features, shape features, and spatial relationship features. The specific features are determined according to the actual processing requirements. Then, multiple video frames are arranged and combined in sequence to obtain the first continuous video frame, which is convenient for subsequent processing.

[0084] In a preferred embodiment, the step of optimizing the first consecutive video frames according to optimization requirements to obtain the second consecutive video frames includes:

[0085] S21. Obtain the video frame to be optimized from the first consecutive video frames;

[0086] S22. Obtain optimization options, and delete and replace the video frames to be optimized according to the optimization options to obtain the deleted frames and the replacement frames.

[0087] S23, Backup and delete frames;

[0088] S24. Insert the replacement frame into the video frame to be optimized to obtain the second consecutive video frame, wherein the replacement frame can be inserted at any position in the first consecutive video frame.

[0089] As described in steps S21-S24 above, before optimizing the first consecutive video frames, it is necessary to first determine the position of the image to be optimized. This process requires manual intervention for screening. First, the range to be optimized is determined, and then the video frames within the optimization range are specified, thus obtaining the video frames to be optimized. The processing methods for the video frames to be optimized include deletion, addition, and replacement. Adding can be regarded as replacing a blank frame. For deletion, the frame can be backed up to the memory as the data of the original video source, which may be used later and is not easy to delete directly. The replacement frame can be inserted into any position in the first consecutive video frames to obtain the second consecutive video frames. Then, the second consecutive video frames are encoded to obtain the replacement video source. By checking the replacement video source, it can be determined whether further optimization is needed. If so, the above steps are repeated; otherwise, no action is taken.

[0090] In a preferred embodiment, such as Figure 2 As shown, the steps for adjusting the reference objects in the replacement frame according to the proportion of shared reference objects include:

[0091] S41. Compare the images in the replacement frame and the first consecutive video frame to obtain the overlapping area and identify it as a common reference.

[0092] S42. Extract the attribute information of the common reference object, whereby the attribute information includes scale information;

[0093] S43. Determine the adjustment ratio of the common reference object based on the ratio information, and adjust the image of the replacement frame according to the adjustment ratio to obtain the reconstructed frame;

[0094] The formula for calculating the adjustment ratio is: S = I(x) i-1 ,y i-1 ) / B(x i ,y i In the formula, S represents the adjustment ratio, and I(x) i-1 ,y i-1 B(x) represents the reference image of the frame preceding the replacement frame. i ,y i () represents the reference image of the replacement frame.

[0095] As described in steps S41-S43 above, there is often a dynamic reference object in the video image. When people observe, their attention will be focused on its movement. If there is a phenomenon of continuous picture but different reference object sizes during the playback of the replacement video source, it will greatly affect the viewing effect. Therefore, after inserting the replacement frame into the first consecutive video frame, it is necessary to make corresponding adjustments. The most important one is to adjust the ratio of the replacement frame. Based on this, in this embodiment, the reference object of the previous frame of the replacement frame is used as the reference value to calculate the adjustment ratio. By adjusting the ratio, the ratio of the replacement frame is adjusted accordingly so that the reference object contained in it is reconstructed to be consistent with the reference object in the first consecutive video frame. In this way, the attention of the subsequent viewers will not be distracted, reducing the sense of disorientation.

[0096] In a preferred embodiment, the step of reconstructing the background in the replacement frame based on the adjustment of the reference object includes:

[0097] S44. Establish a sampling cycle;

[0098] S45. During the sampling period, obtain the first background information of multiple determined frames at the front and back ends of the reconstructed frame from the first consecutive video frames.

[0099] S46. Obtain the second background information of the reconstructed frame;

[0100] S47. Substitute the first and second background information into the reconstruction model to obtain the third background information of the reconstructed frame. The calculation formula in the reconstruction model is as follows: In the formula, I(x) t-1,y t-1 P(x) represents the front-end defined frame. t+1 ,y t+1 ) indicates a backend-determined frame;

[0101] S48. Insert the reconstructed frame between the front-end determined frame and the back-end determined frame to obtain the second consecutive video frame.

[0102] As described in steps S44-S48 above, after adjusting the reference object of the replacement frame, its background is also adjusted accordingly. However, after the background adjustment, it is difficult to match the image background in the first consecutive video frame. Moreover, the lighting and brightness of the backgrounds collected at different times are also different. Therefore, it is necessary to reconstruct the image background based on the image background of the video frames before and after the replacement frame. Since the time interval between multiple consecutive video frames is short, it is difficult to identify the magnitude of the change between each video frame with the naked eye. Therefore, when reconstructing the background of the replacement frame, the average value of the image background of the video frames before and after the replacement frame is used to establish the background, so as to achieve the purpose of reconstructing the background of the replacement frame. The reconstructed background combined with the reference object that has been adjusted above forms a new replacement frame. The matching degree of the subsequent second consecutive video frame is also improved accordingly. In this way, the viewing experience of the processed video source can be better.

[0103] In a preferred embodiment, the step of obtaining multiple optimized nodes from the total number of frames in the second consecutive video frames, and establishing multiple discontinuities based on the optimized nodes to obtain multiple discontinuous video sources, includes:

[0104] S51. Obtain the positions of the replacement frame and the deleted frame in the second consecutive video frame;

[0105] S52. Determine the nodes before and after the replacement frame and the position of the deleted frame as optimization nodes;

[0106] S53. Starting from the first frame of the second consecutive video frame, and with the first optimized node as the first discontinuity, establish a discontinuous video source. Subsequently, establish discontinuous video sources with every two adjacent optimized nodes as discontinuities.

[0107] As described in steps S51-S53 above, the initial decoding of the target video source often takes a long time. If the replacement video sources obtained after subsequent optimization, reference adjustment, and background reconstruction are aggregated together, they will require re-decoding during processing. Therefore, during the optimization of the target video source, discontinuities are established based on multiple optimization nodes, so that the replacement video source can be divided into multiple discontinuous video sources. When the replacement video source needs to be optimized later, it is not necessary to decode all of the replacement video source; only the discontinuous video sources need to be extracted and decoded, providing a good foundation for further optimization of the replacement video source.

[0108] In a preferred embodiment, the step of acquiring image information of the first and last frames of adjacent discontinuous video sources, and substituting the image information of the first and last frames into the matching degree calculation formula to obtain the image matching degree of the first and last frames of adjacent discontinuous video sources includes:

[0109] S61. Obtain the first frame image information of the current intermittent video source and the last frame image of the previous intermittent video source.

[0110] S62. Compress the first frame image and the last frame image to remove detail points and obtain the basic structural information of the first frame image and the last frame image.

[0111] S63. Calculate the average grayscale value of all pixels in the compressed first and last frame images, respectively, using the following formula: In the formula, Z represents the average grayscale value, a represents all possible grayscale values ​​(taking positive integer values), MN represents the total number of pixels, and R... j E represents the number of times a grayscale value appears in a pixel. j Indicates pixel grayscale;

[0112] S64. Calculate the difference between the average grayscale values ​​of the first and last frames in the first and last frames of adjacent discontinuous video sources to obtain the image matching degree between the first and last frames.

[0113] As described in steps S61-S64 above, after determining the intermittent video sources, the next step is to determine the continuity between connected intermittent video sources. This is specifically determined by adjusting the first frame of the current video and the last frame of the previous intermittent video source. The changes between adjacent frames are considered as a translation process of multiple blocks within them. In an image, it is divided into blocks according to pixels. For example, a compressed image with a size of 16×16 has a total of 256 pixels. This is then converted into 256 pixels, each corresponding to a different pixel level. The average grayscale value of all these pixels is calculated, and then the difference is performed to obtain the image matching degree between the first and last frames of adjacent intermittent video sources. Of course, the grayscale values ​​of all pixels can also be compared one by one, which yields a more accurate value, but requires a larger amount of computation. Although the value obtained by comparing using the average value algorithm has a deviation, if it meets the standard, it will not affect the viewing experience.

[0114] In a preferred embodiment, the step of determining whether the image matching degree falls within the image matching degree standard threshold includes:

[0115] S81. Determine the range of values ​​for the standard threshold of image matching degree, specifically [0, 3];

[0116] S82. If the image matching degree is between [0, 3], it indicates that adjacent discontinuous video sources have continuity.

[0117] S83. If the value of the image matching degree is outside the interval [0, 3], it means that the adjacent discontinuous video sources are not continuous, and motion compensation processing is performed on the first and last frames at the discontinuity points.

[0118] Motion compensation processing includes one of the following: global motion compensation, inter-frame interpolation motion compensation, and hierarchical estimation algorithm.

[0119] As described in steps S81-S83 above, since the comparison is made between the first and last frames of the connected discontinuous video sources, the deviation range is set relatively small. If all pixels are compared one by one, this range can be expanded accordingly. The specific setting depends on actual experience, and no specific restrictions are imposed in this paper. If the comparison result is a match, the second consecutive video frame can be encoded and used directly as the replacement video source. If the matching degree does not meet the standard, motion compensation processing can be performed on the last frame image of the adjacent discontinuous video sources. This process can select global motion compensation, inter-frame interpolation motion compensation, and layered estimation algorithms (this is a commonly used motion compensation algorithm) according to the specific situation. In this way, virtual matching pixel points can be established in the last frame image, and finally the third consecutive video frame is obtained. After encoding processing, the replacement video source can be obtained.

[0120] The present invention also provides a video processing image device based on motion compensation, characterized in that: applied to the above-mentioned video processing method, comprising:

[0121] The first acquisition module is used to acquire the first consecutive video frames of the first video source.

[0122] The optimization module is used to optimize the first consecutive video frames according to the optimization requirements to obtain the second consecutive video frames. The optimization process includes replacing frames and deleting frames.

[0123] The second acquisition module is used to acquire the common reference between the replacement frame and the first consecutive video frame;

[0124] The reconstruction module is used to adjust the reference objects in the replacement frame according to the proportion of the common reference objects, and to reconstruct the background in the replacement frame based on the adjustment of the reference objects.

[0125] The third acquisition module is used to acquire multiple optimized nodes in the total number of frames of the second consecutive video frames, and establish multiple discontinuity points based on the optimized nodes to obtain multiple discontinuous video sources.

[0126] The calculation module is used to obtain the image information of the first and last frames of adjacent discontinuous video sources, and substitute the image information of the first and last frames into the matching degree calculation formula to obtain the image matching degree of the first and last frames of adjacent discontinuous video sources.

[0127] The fourth acquisition module is used to acquire the standard threshold for image matching degree;

[0128] The comparison module is used to determine whether the image matching degree falls within the standard threshold of image matching degree.

[0129] In a preferred embodiment, the system further includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the video processing method described above.

[0130] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0131] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A video processing method based on motion compensation, characterized in that: include: Obtain the first consecutive video frames of the target video source; The first consecutive video frames are optimized according to the optimization requirements to obtain the second consecutive video frames. The optimization process includes replacing frames and deleting frames. Obtain the common reference point between the replacement frame and the first consecutive video frames; Based on the proportion of shared reference objects, the reference objects in the replacement frame are adjusted, and the background in the replacement frame is reconstructed based on the adjustment of the reference objects; Multiple optimized nodes are obtained from the total number of frames in the second consecutive video frame, and multiple discontinuity points are established based on the optimized nodes to obtain multiple discontinuous video sources; The image information of the first and last frames of adjacent intermittent video sources is obtained, and the image information of the first and last frames is substituted into the matching degree calculation formula to obtain the image matching degree of the first and last frames of adjacent intermittent video sources. Obtain the standard threshold for image matching; Determine whether the image matching degree falls within the standard threshold of image matching degree; If the image matching degree is within the standard threshold, it indicates that the adjacent discontinuous video sources are continuous. If the image matching degree value is not within the standard threshold, it indicates that the adjacent discontinuous video sources are not continuous. Motion compensation processing is then performed on the first and last frames at the discontinuity points to obtain the third continuous video frame.

2. The video processing method based on motion compensation according to claim 1, characterized in that: The step of obtaining the first consecutive video frames of the target video source includes: The encoder decodes the target video source to obtain the video signal; The video signal is transmitted to the controller through a data interface. The controller extracts video frames based on image features, wherein the image features include at least color features, texture features, shape features, and spatial relationship features. The extracted video frames are arranged consecutively to obtain the first consecutive video frame.

3. The video processing method based on motion compensation according to claim 1, characterized in that: The steps for optimizing the first consecutive video frames to obtain the second consecutive video frames according to the optimization requirements include: Obtain the video frame to be optimized from the first consecutive video frames; Obtain optimization options, and delete and replace the video frames to be optimized according to the optimization options to obtain the deleted frames and the replacement frames; Backup the deleted frames; The replacement frame is inserted into the video frame to be optimized to obtain a second consecutive video frame, wherein the replacement frame can be inserted at any position in the first consecutive video frame.

4. The video processing method based on motion compensation according to claim 1, characterized in that: The step of adjusting the reference objects in the replacement frame according to the proportion of shared reference objects includes: The images in the replacement frame and the first consecutive video frame are compared to obtain the overlapping blocks and identified as common reference objects. Extract the attribute information of the common reference object, wherein the attribute information includes scale information; The adjustment ratio of the common reference object is determined based on the ratio information, and the image of the replacement frame is adjusted according to the adjustment ratio to obtain the reconstructed frame; The formula for calculating the adjustment ratio is: S = I(x) i-1 ,y i-1 ) / B(x i ,y i In the formula, S represents the adjustment ratio, and I(x) i-1 ,y i-1 B(x) represents the reference image of the frame preceding the replacement frame. i ,y i () represents the reference image of the replacement frame.

5. The video processing method based on motion compensation according to claim 4, characterized in that: The steps for reconstructing the background in the replacement frame based on adjustments to a reference object include: Establish a sampling cycle; During the sampling period, first background information of multiple determined frames at the front and back ends of the reconstructed frame is obtained from the first consecutive video frames; Obtain the second background information of the reconstructed frame; Substituting the first and second background information into the reconstruction model, the third background information of the reconstructed frame is obtained. The calculation formula in the reconstruction model is as follows: In the formula, I(x) t-1 ,y t-1 P(x) represents the front-end defined frame. t+1 ,y t+1 ) indicates the backend-determined frame; The reconstructed frame is inserted between the front-end determined frame and the back-end determined frame to obtain a second consecutive video frame.

6. The video processing method based on motion compensation according to claim 1, characterized in that: The steps of obtaining multiple optimized nodes from the total number of frames in the second consecutive video frame, and establishing multiple discontinuities based on the optimized nodes to obtain multiple discontinuous video sources, include: Obtain the positions of the replacement and deletion frames in the second consecutive video frames; The nodes before and after the replacement frame and the position of the deleted frame are determined as optimization nodes; Starting with the first frame of the second consecutive video frame, and using the first optimized node as the first discontinuity, an intermittent video source is established. Subsequently, intermittent video sources are established with every two adjacent optimized nodes as discontinuities.

7. A video processing method based on motion compensation according to claim 6, characterized in that: The steps of obtaining image information of the first and last frames of adjacent intermittent video sources, and substituting the image information of the first and last frames into the matching degree calculation formula to obtain the image matching degree of the first and last frames of adjacent intermittent video sources include: Obtain the first frame image information of the current intermittent video source and the last frame image of the previous intermittent video source; The first and last frame images are compressed to remove detail points, thus obtaining the basic structural information of the first and last frame images. Calculate the average grayscale value of all pixels in the compressed first and last frames, respectively, using the following formula: In the formula, Z represents the average grayscale value, a represents all possible grayscale values ​​(taking positive integer values), MN represents the total number of pixels, and R... j E represents the number of times a grayscale value appears in a pixel. j Indicates pixel grayscale; The difference between the average grayscale values ​​of the first and last frames in the first and last frames of the adjacent intermittent video sources is calculated to obtain the image matching degree between the first and last frames.

8. The video processing method based on motion compensation according to claim 7, characterized in that: The steps for determining whether an image matching degree falls within the standard threshold for image matching degree include: The range of the image matching degree standard threshold is determined to be [0, 3]; If the value of the image matching degree is between [0, 3], it indicates that adjacent discontinuous video sources have continuity; If the value of the image matching degree is outside the interval [0, 3], it indicates that the adjacent discontinuous video sources are not continuous, and motion compensation processing is performed on the first and last frames at the discontinuity points. The motion compensation process includes one of the following: global motion compensation, inter-frame interpolation motion compensation, and hierarchical estimation algorithm.

9. A video processing image device based on motion compensation, characterized in that: The video processing method described in claims 1-8 includes: A first acquisition module is used to acquire the first consecutive video frames of the first video source. An optimization module is used to optimize a first consecutive video frame according to optimization requirements to obtain a second consecutive video frame, wherein the optimization process includes replacing frames and deleting frames. The second acquisition module is used to acquire the common reference between the replacement frame and the first consecutive video frame; A reconstruction module is used to adjust the reference objects in the replacement frame according to the proportion of the common reference objects, and to reconstruct the background in the replacement frame based on the adjustment of the reference objects. The third acquisition module is used to acquire multiple optimized nodes in the total number of frames of the second consecutive video frames, and establish multiple discontinuity points based on the optimized nodes to obtain multiple discontinuous video sources. The calculation module is used to obtain the image information of the first frame and the last frame of the adjacent intermittent video sources, and substitute the image information of the first frame and the last frame into the matching degree calculation formula to obtain the image matching degree of the first frame and the last frame of the adjacent intermittent video sources. The fourth acquisition module is used to acquire the image matching degree standard threshold; The comparison module is used to determine whether the image matching degree falls within the image matching degree standard threshold.

10. A video processing image device based on motion compensation according to claim 9, characterized in that: It also includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the video processing method of any one of claims 1 to 8.

Images