Video processing method and system
By extracting similar image frames of video and calculating redundancy boundaries, the problems of redundant data reduction and image quality reduction in video compression are solved, and efficient video compression and low distortion video processing are achieved.
Patent Information
- Application Number
- CN202410193787.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-21
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-02-21
AI Technical Summary
During the video compression process, the prior art is difficult to effectively reduce the redundant data of the video, resulting in a decrease in the video quality and affecting the user's viewing experience.
By obtaining the to-processed video and converting it into an image frame timing set, similar image frames are extracted, the rate of change between reference image domain and redundant frames is determined, the grayscale extreme value and contrast value of the local area of the reference image is selected, and the reference blur is calculated. , and then determine the redundant boundaries of the image frame and compress the video according to these boundaries.
Effectively reduce redundant data of video, reduce video distortion, improve video compression efficiency, and improve user experience.
Smart Images

Figure CN118101952B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of video processing technology, and more specifically, to a video processing method and system. Background Art
[0002] Video processing refers to a series of technical means and methods for acquiring, encoding, compressing, storing, transmitting, editing, analyzing and displaying video data. Video processing technology also involves video acquisition, post-processing, special effects production, image recognition and analysis, etc. Video processing technology plays an important role in the multimedia industry, communications, security monitoring, medical imaging, virtual reality, etc., and has brought many conveniences to people's lives and work.
[0003] Artificial intelligence-based video processing is a method that uses machine learning and deep learning technologies to process, analyze and improve video content. Artificial intelligence-based video processing methods integrate technologies such as deep learning, computer vision and image processing, and have promoted innovation in the field of video processing. Video compression is to reduce the amount of data in video files while maximizing the retention of video quality to reduce the size of video files and thereby improve video transmission efficiency and save storage space. Video compression usually eliminates some details and redundant data in the video that are not easily perceived by the human eye to reduce the size of video files. However, in the process of video compression, due to the existence of redundant data in the video, the video quality is reduced, affecting the user's viewing experience. Therefore, how to reduce redundant data in the video and reduce video distortion during video compression is an urgent problem to be solved. Summary of the invention
[0004] The present application provides a video processing method and system, which can reduce redundant data of a video and reduce video distortion during the video compression process.
[0005] In a first aspect, the present application provides a video processing method, comprising the following steps:
[0006] Obtaining a video to be processed, and converting the video to be processed into a time sequence set of image frames;
[0007] Extracting each pair of similar image frames in the image frame time series set to obtain a similar image domain, and determining a reference image domain and a redundant frame inter-frame change rate from the similar image domain;
[0008] Selecting a reference image in the reference image domain, determining a grayscale extreme value of a local area in the reference image, determining a local contrast amount of the reference image according to the grayscale extreme value, continuing to determine local contrast amounts of remaining reference images in the reference image domain, and determining a reference blur of the reference image domain according to all local contrast amounts;
[0009] Determine the redundancy limit of the image frames in the image frame time series set by using the redundant frame-to-frame change rate and the reference fuzziness;
[0010] The video to be processed is deduplicated according to the redundancy limit.
[0011] In some embodiments, extracting each pair of similar image frames in the image frame time series set to obtain a similar image domain specifically includes:
[0012] Setting an image similarity decision value of the image frame time series set;
[0013] Extracting similar image frames from the image frame time series set according to the image similarity decision value;
[0014] All similar image frames are grouped into a similar image domain.
[0015] In some embodiments, determining the reference image domain and the redundant frame inter-frame change rate from the similar image domain specifically includes:
[0016] determining a reference image for each pair of similar image frames in the similar image domain;
[0017] The set of all reference images is taken as the reference image domain;
[0018] determining the redundancy of each pair of similar image frames in the similar image domain;
[0019] The redundant frame-to-frame change rate of the video to be processed is determined according to all the redundancies.
[0020] In some embodiments, determining the grayscale extreme value of the local area in the reference image specifically includes:
[0021] The reference image is evenly divided to obtain multiple local areas;
[0022] Determine the internal grayscale value of each local area;
[0023] The grayscale extreme value of the local area in the reference image is determined by all internal grayscale values.
[0024] In some embodiments, determining the local contrast amount of the reference image by the grayscale extreme value specifically includes:
[0025] Determine the brightness characteristic value of the reference image;
[0026] The local contrast amount of the reference image is determined by the brightness characteristic value and the grayscale extreme value.
[0027] In some embodiments, determining the reference blurriness of the reference image domain by using all local contrast quantities specifically includes:
[0028] Obtaining a local contrast amount corresponding to each reference image in the reference image domain;
[0029] Determine the brightness feature gradient of the reference image domain according to all local contrast quantities;
[0030] determining a reference difference factor for the reference image domain;
[0031] A reference fuzziness of the reference image domain is determined according to a local contrast amount corresponding to each reference image in the reference image domain, the brightness feature gradient and the reference difference factor.
[0032] In some embodiments, a frame of image in the video to be processed is used as an image frame.
[0033] In a second aspect, the present application provides a video processing system, comprising:
[0034] An acquisition module, used for acquiring a video to be processed and converting the video to be processed into a time sequence set of image frames;
[0035] A processing module, used for extracting each pair of similar image frames in the image frame time series set to obtain a similar image domain, and determining a reference image domain and a redundant frame inter-frame change rate from the similar image domain;
[0036] The processing module is further configured to select a reference image in the reference image domain, determine a grayscale extreme value of a local area in the reference image, determine a local contrast amount of the reference image according to the grayscale extreme value, continue to determine local contrast amounts of remaining reference images in the reference image domain, and determine a reference blur of the reference image domain according to all local contrast amounts;
[0037] The processing module is further used to determine the redundancy limit of the image frames in the image frame time series set according to the redundant frame-to-frame change rate and the reference fuzziness;
[0038] The execution module is used to eliminate redundancy of the video to be processed according to the redundancy limit.
[0039] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned video processing method.
[0040] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions or codes, and when the instructions or codes are run on a computer, the computer implements the above-mentioned video processing method when executing.
[0041] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:
[0042] In the embodiment of the present application, the redundant limit of the image frames in the image frame time sequence set is determined by the redundant frame-to-frame change rate and the reference fuzziness. When the redundant frame-to-frame change rate is smaller, it means that the degree of change of the redundant frames in the video to be processed is smaller, and a lower redundant limit is required to detect the redundant data between the image frames in the video to be processed, so as to eliminate the redundant data between the image frames in the video to be processed. When the reference fuzziness is higher, it means that the details in the image frames in the video to be processed are fewer, which means that the difference between the image frames in the video to be processed is smaller, and thus a lower redundant limit is required to detect the redundant data between the image frames in the video to be processed. The redundant data is pixels with similar or identical colors or brightness between the image frames, so eliminating the redundant data will not affect the video to be processed. The final visual quality of the video is significantly affected. The redundant limit is determined by determining the redundant frame-to-frame change rate and the baseline blur, and the redundant limit is then used to improve the accuracy of identifying redundant data that can be compressed or eliminated in the processed video. When the difference between the image frames in the image frame time sequence set is less than the redundant limit, it means that the redundant data between the image frames in the image frame time sequence set has reached the standard that can be eliminated, that is, the image frames in the image frame time sequence set have a high redundancy rate. The redundant limit further reflects the redundant data that can be compressed or eliminated in the image frames, and the redundant data of the image frames in the image frame time sequence set of the processed video is eliminated, thereby reducing the redundant data of the processed video during the compression process of the processed video and reducing the video distortion. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0044] Figure 1 is an exemplary flow chart of a video processing method according to some embodiments of the present application;
[0045] Figure 2 is an exemplary flow chart of determining similar image domains according to some embodiments of the present application;
[0046] Figure 3 is an exemplary flow chart for determining grayscale extreme values according to some embodiments of the present application;
[0047] Figure 4is a schematic diagram of exemplary hardware and / or software of a video processing system according to some embodiments of the present application;
[0048] Figure 5 It is a structural diagram of a computer device for implementing a video processing method according to some embodiments of the present application. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0050] The embodiment of the present application provides a video processing method and system, the core of which is to obtain a video to be processed, convert the video to be processed into an image frame time sequence set, extract each pair of similar image frames in the image frame time sequence set to obtain a similar image domain, determine a reference image domain and a redundant frame-to-frame change rate from the similar image domain, select a reference image in the reference image domain, determine the grayscale extreme value of a local area in the reference image, determine the local contrast amount of the reference image from the grayscale extreme value, continue to determine the local contrast amount of the remaining reference images in the reference image domain, determine a reference blur of the reference image domain through all local contrast amounts, determine the redundant limit of the image frames in the image frame time sequence set through the redundant frame-to-frame change rate and the reference blur, and eliminate redundancy in the video to be processed according to the redundant limit, so as to reduce redundant data of the video to be processed during the compression process of the video to be processed and reduce video distortion.
[0051] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 , which is an exemplary flow chart of a video processing method according to some embodiments of the present application. The video processing method 100 mainly includes the following steps:
[0052] In step 101, a video to be processed is obtained, and the video to be processed is converted into a time sequence set of image frames.
[0053] In specific implementation, the video to be processed can be obtained from the video library, a frame of image in the video to be processed is used as an image frame, and all image frames in the video to be processed are sorted in the order of the corresponding time of each image frame to form an image frame time sequence set.
[0054] In step 102, each pair of similar image frames in the image frame time series set is extracted to obtain a similar image domain, and a reference image domain and a redundant frame change rate are determined from the similar image domain.
[0055] In some embodiments, reference Figure 2 As shown in FIG. 1 , this figure is an exemplary flow chart of determining a similar image domain in some embodiments of the present application. In this embodiment, each pair of similar image frames in the image frame time series set is extracted to obtain a similar image domain, which can be implemented by the following steps:
[0056] First, in step 1021, the image similarity decision value of the image frame time series set is set;
[0057] Next, in step 1022, similar image frames are extracted from the image frame time series set using the image similarity decision value;
[0058] Finally, in step 1023, all similar image frames are grouped into a similar image domain.
[0059] It should be noted that the image similarity decision value in this embodiment is a discriminant value for judging the degree of similarity between image frames. The image similarity decision value is used to judge whether two image frames are similar image frames. In specific implementation, the image similarity decision value can be set according to the degree of similarity between two similar images in historical experiments. The image similarity decision value is generally set to 0.9. In other embodiments, other methods can also be used to set the image similarity decision value, which is not limited here.
[0060] In specific implementation, similar image frames can be extracted from the image frame time sequence set by using the image similarity decision value in the following manner, namely: the first image frame in the image frame time sequence set is selected, and the similarity between the image frame and the remaining image frames in the image frame time sequence set can be calculated by a structural similarity algorithm (SSIM); when the maximum similarity exceeds the image similarity decision value, the two image frames corresponding to the maximum similarity are combined into a pair of similar image frames; if the maximum similarity does not exceed the image similarity decision value, the second image frame in the image frame time sequence set is selected, the similarity between the second image frame and the remaining image frames in the image frame time sequence set is recalculated, and similar image frames are extracted again until the extraction of similar image frames is completed; the above steps are repeated to extract similar image frames for the remaining image frames in the image frame time sequence set that have not been extracted, until all similar image frames in the image frame time sequence set are extracted; all similar image frames are combined into a similar image domain, that is, all similar image frames are combined into a similar image domain in the order of extraction.
[0061] It should be noted that the similarity in this embodiment is a value that measures the degree of similarity between two image frames. The value range of the similarity is usually between -1 and 1. The closer the similarity is to 1, the more similar the two image frames are. The closer the similarity is to -1, the less similar the two image frames are. We will not go into details here.
[0062] In some embodiments, determining the reference image domain and the redundant frame inter-frame change rate from the similar image domain may be implemented by the following steps:
[0063] determining a reference image for each pair of similar image frames in the similar image domain;
[0064] The set of all reference images is taken as the reference image domain;
[0065] determining the redundancy of each pair of similar image frames in the similar image domain;
[0066] The redundant frame-to-frame change rate of the video to be processed is determined according to all the redundancies.
[0067] Among them, in a specific implementation, a reference image of each pair of similar image frames in the similar image domain is determined, that is: a first pair of similar image frames in the similar image domain is selected, and the information entropy corresponding to the two image frames in the pair of similar image frames is calculated by an existing information entropy algorithm, and the image frame with a larger information entropy is used as the reference image of the first pair of similar image frames; a second pair of similar image frames in the similar image domain is selected, and the information entropy corresponding to the two image frames in the pair of similar image frames is calculated by an existing information entropy algorithm, and the image frame with a larger information entropy is used as the reference image of the second pair of similar image frames, and the reference images of the remaining similar image frames in the similar image domain are continued to be determined.
[0068] It should be noted that the reference image in the reference image domain in the present application reflects the basic information corresponding to each pair of similar image frames in the similar image domain.
[0069] Among them, in a specific implementation, determining the redundancy of each pair of similar image frames in the similar image domain can be achieved in the following manner: that is, selecting the first pair of similar image frames in the similar image domain, determining the similar area of the pair of similar image frames through Open CV, and taking the total number of all pixels in the similar area as the redundancy of the first pair of similar image frames, selecting the second pair of similar image frames in the similar image domain, determining the similar area of the pair of similar image frames through Open CV, and taking the total number of all pixels in the similar area as the redundancy of the second pair of similar image frames, and continuing to determine the redundancy of the remaining similar image frames in the similar image domain.
[0070] It should be noted that the redundancy in this embodiment reflects the redundancy degree of similar image frames. The larger the redundancy, the greater the redundancy degree of similar image frames. Conversely, the smaller the redundancy, the smaller the redundancy degree of similar image frames.
[0071] In some embodiments, determining the redundant frame-to-frame change rate of the video to be processed according to all the redundancies may be implemented by the following steps:
[0072] All redundant quantities are grouped into redundant quantity sequences;
[0073] Get the total number m of all redundant quantities;
[0074] Get the jth redundant quantity δ in the redundant quantity sequence j ;
[0075] Get the j-1th redundant quantity δ in the redundant quantity sequence j-1 ;
[0076] According to the total number m of all redundant quantities, the jth redundant quantity δ in the redundant quantity sequence j and the j-1th redundant quantity δ in the redundant quantity sequence j-1 Determine the redundant frame-to-frame change rate of the video to be processed, wherein the redundant frame-to-frame change rate can be determined using the following formula:
[0077]
[0078] Wherein, k represents the change rate between redundant frames, and j=2, 3, ..., m.
[0079] In a specific implementation, all the redundancies are grouped into a redundancy sequence, that is, the redundancies are grouped into a redundancy sequence according to the order of similar image frames corresponding to the redundancies in the similar image domain.
[0080] It should be noted that the redundant frame change rate in the present application reflects the degree of change of the redundant frames in the video to be processed. The larger the redundant frame change rate, the greater the degree of change of the redundant frames in the video to be processed. Conversely, the smaller the redundant frame change rate, the smaller the degree of change of the redundant frames in the video to be processed.
[0081] In step 103, a reference image in the reference image domain is selected, the grayscale extreme value of the local area in the reference image is determined, the local contrast amount of the reference image is determined by the grayscale extreme value, the local contrast amounts of the remaining reference images in the reference image domain are further determined, and the reference blur of the reference image domain is determined by all the local contrast amounts.
[0082] In some embodiments, reference Figure 3As shown in FIG. 1 , this figure is an exemplary flow chart for determining grayscale extreme values in some embodiments of the present application. In this embodiment, determining the grayscale extreme value of the local area in the reference image can be implemented by the following steps:
[0083] First, in step 1031, the reference image is evenly divided to obtain a plurality of local regions;
[0084] Next, in step 1032, the internal grayscale value of each local area is determined;
[0085] Finally, in step 1033, the grayscale extreme value of the local area in the reference image is determined by all the internal grayscale values.
[0086] In specific implementation, the reference image is evenly divided to obtain multiple local areas, which can be achieved in the following manner, namely: Open CV can be used to evenly divide the reference image into N local areas. In other embodiments, other methods can also be used to evenly divide the reference image. The number of local area divisions can be set according to specific needs, which will not be repeated here; the internal grayscale value of each local area is determined, namely: a local area is selected, and the sum of the grayscale values of all pixels in the local area is used as the internal grayscale value of the local area, and the internal grayscale values of the remaining local areas are continued to be determined; the grayscale extreme value of the local area in the reference image is determined by all the internal grayscale values, namely: the largest internal grayscale value among all the internal grayscale values is used as the grayscale extreme value of the local area in the reference image.
[0087] It should be noted that the grayscale extreme value in the present application reflects the brightness level of the local highlighted area of the reference image. The larger the grayscale extreme value, the higher the brightness level of the local highlighted area of the reference image. Conversely, the smaller the grayscale extreme value, the lower the brightness level of the local highlighted area of the reference image.
[0088] In some embodiments, determining the local contrast amount of the reference image by the grayscale extreme value may be implemented by the following steps:
[0089] Determine the brightness characteristic value of the reference image;
[0090] The local contrast amount of the reference image is determined by the brightness characteristic value and the grayscale extreme value.
[0091] In specific implementation, the brightness characteristic value in the present application is a value reflecting the overall brightness level of the reference image. The larger the brightness characteristic value, the higher the overall brightness level of the reference image. The smaller the brightness characteristic value, the lower the overall brightness level of the reference image. As a preferred embodiment, the brightness characteristic value of the reference image can be determined in the following manner, namely: the ratio of the sum of the grayscale values of all pixels in the reference image to the total number of pixels in the reference image is used as the brightness characteristic value of the reference image. In other embodiments, other methods can also be used, which are not limited here. The local contrast amount is a measure of the image of the highlighted area in the reference image. The parameter value of the proportion of pixel grayscale values, the larger the local contrast amount, the higher the proportion of pixel grayscale values in the highlighted area in the reference image, and the smaller the local contrast amount, the lower the proportion of pixel grayscale values in the highlighted area in the reference image. As a preferred embodiment, the local contrast amount of the reference image can be determined by the brightness characteristic value and the grayscale extreme value in the following manner, namely: determining the difference between the brightness characteristic value and the grayscale extreme value, and taking the ratio of the absolute value of the difference to the brightness characteristic value as the local contrast amount of the reference image. In other embodiments, other methods can also be used, which are not limited here.
[0092] It should be noted that the reference blurriness in the present application is an indicator for measuring the blurriness of the reference image in the reference image domain. The higher the reference blurriness, the greater the blurriness of the reference image in the reference image domain. The lower the reference blurriness, the smaller the blurriness of the reference image in the reference image domain. In some embodiments, determining the reference blurriness of the reference image domain through all local contrast amounts can be implemented by the following steps:
[0093] Obtaining a local contrast amount β(i) of the i-th reference image in the reference image domain;
[0094] Determine the brightness feature gradient of the reference image domain according to all local contrast quantities
[0095] Determining a reference difference factor σ of the reference image domain;
[0096] According to the local contrast amount β(i) of the i-th reference image in the reference image domain, the brightness feature gradient The reference fuzziness of the reference image domain is determined by the reference difference factor σ, wherein the reference fuzziness can be determined according to the following formula:
[0097]
[0098] Wherein, ε represents the reference blur, n represents the total number of reference images in the reference image domain, and i = 1, 2, …n.
[0099] It should be noted that the brightness feature gradient in the present embodiment is a parameter value for measuring the degree of brightness change of the reference image in the reference image domain. The larger the brightness feature gradient, the larger the degree of brightness change of the reference image in the reference image domain, which means that the reference image in the reference image domain is clearer. The smaller the brightness feature gradient, the smaller the degree of brightness change of the reference image in the reference image domain, which means that the reference image in the reference image domain is more blurred. In specific implementation, the brightness feature gradient of the reference image domain can be determined based on all local contrast amounts in the following manner, that is, the variance of all local contrast amounts can be used as the brightness feature gradient of the reference image domain. In other embodiments, other methods can be used to determine the brightness feature gradient of the reference image domain, which is not limited here.
[0100] It should be noted that the reference difference factor in the present embodiment reflects the degree of difference in pixel distribution of different reference images in the reference image domain. The larger the reference difference factor is, the larger the difference in pixel distribution of different reference images in the reference image domain is, which means that the reference image in the reference image domain is clearer. The smaller the reference difference factor is, the smaller the difference in pixel distribution of different reference images in the reference image domain is, which means that the reference image in the reference image domain is blurrier. In specific implementation, the reference difference factor of the reference image domain may be determined in the following manner, namely: selecting a reference image in the reference image domain, counting the total number of different grayscale pixel values in the reference image, continuing to determine the total number of different grayscale pixel values in the remaining reference images in the reference image domain, and taking the standard deviation of all the total numbers as the reference difference factor of the reference image domain. In other embodiments, other methods may be used to determine the reference difference factor of the reference image domain, which is not limited here.
[0101] In step 104, the redundancy limit of the image frames in the image frame time sequence set is determined by the redundant frame-to-frame change rate and the reference blur.
[0102] In some embodiments, determining the redundancy limit of the image frames in the image frame time sequence set by using the redundant frame-to-frame change rate and the reference blur can be implemented by the following steps:
[0103] Obtaining the redundant frame-to-frame change rate;
[0104] Acquiring the reference ambiguity;
[0105] Determining an adjustable coefficient of the image frame timing set;
[0106] Determining a compression impact factor of the image frame timing set;
[0107] The redundancy limit of the image frames in the image frame timing set is determined by the redundant frame inter-frame change rate, the reference blur, the adjustable coefficient of the image frame timing set and the compression influence factor of the image frame timing set, wherein the value of the redundancy limit can be determined according to the following formula:
[0108]
[0109] Among them, ψ represents the value of the redundant limit of the image frame in the image frame timing set, k represents the redundant frame-to-frame change rate, ε represents the reference blur, α represents the adjustable coefficient of the image frame timing set, and φ represents the compression influence factor of the image frame timing set.
[0110] In addition, it should be noted that the adjustable coefficient in the present application is a parameter used to adjust the redundant limit of the image frame in the image frame timing set. The adjustable coefficient is adjusted within an adjustable range of 0-2. When the adjustable coefficient is set between 0-1, it means that the value of the redundant limit of the image frame in the image frame timing set is reduced. When the adjustable coefficient is set to 1, it means that the value of the redundant limit of the image frame in the image frame timing set remains unchanged. When the adjustable coefficient is set between 1-2, it means that the value of the redundant limit of the image frame in the image frame timing set is increased. The adjustable coefficient can be set according to specific needs, and it will not be repeated here. The compression impact factor reflects the degree of influence of external factors on the compression of image frames in the video. Different types of video content have different sensitivities to compression. Therefore, the compression impact factor needs to be adjusted according to the specific application scenario to meet different application requirements. For example, a video with more movement requires a higher compression rate, and the compression impact factor can be set within the range of 0-0.5. A static scene requires a lower compression rate, and the compression impact factor can be set within the range of 0.5-1. In other embodiments, other methods can also be used to set the compression impact factor, which is not limited here.
[0111] It should be noted that the redundancy limit in the present application is a decision value used to define the degree of redundancy between image frames in an image frame time sequence set. When the degree of difference between image frames in an image frame time sequence set is less than the redundancy limit, it means that the redundant data between image frames in the image frame time sequence set has reached the standard that can be eliminated, that is, there is a very high redundancy rate in the image frames in the image frame time sequence set. The redundancy limit further reflects the redundant data in the image frames that can be compressed or eliminated, and the redundant data will not have a significant impact on the final visual quality of the processed video.
[0112] In step 105, the video to be processed is deduplicated according to the redundancy limit.
[0113] In a specific implementation, an image frame in the image frame time sequence set of the video to be processed is selected, and the next image frame of the image frame is used as the adjacent image frame of the image frame, and the difference between the average value of each row of pixel values in the image frame and the average value of each row of pixel values in the adjacent image frame is calculated, and the sum obtained by accumulating all the differences is used as the inter-frame redundancy between the image frame and the adjacent image frame of the image frame. When the inter-frame redundancy is less than the value of the redundancy limit, it means that there is a high redundancy rate between the image frame corresponding to the inter-frame redundancy and the adjacent image frame, and the redundant data between the image frame corresponding to the inter-frame redundancy and the adjacent image frame reaches the standard that can be eliminated, and the redundant data is repeated information between the image frames, that is, the pixels between the image frames have similar or identical colors or brightness, and the redundant data between the image frame and the adjacent image frame can be eliminated by the existing compression technology, and the remaining image frames in the image frame time sequence set of the video to be processed are continuously eliminated, thereby reducing the redundant data of the video to be processed during the compression process of the video to be processed and reducing the video distortion.
[0114] It should be noted that the inter-frame redundancy in this embodiment is a parameter for measuring the degree of difference between image frames in the image frame time sequence set, and further reflects the proportion of repeated information between image frames and adjacent image frames in the image frame time sequence set. The smaller the inter-frame redundancy, the greater the proportion of repeated information between image frames and adjacent image frames in the image frame time sequence set, which indicates that the redundancy rate between adjacent frames in the video to be processed is higher. The larger the inter-frame redundancy, the smaller the proportion of repeated information between image frames and adjacent image frames in the image frame time sequence set, which indicates that the redundancy rate between adjacent frames in the video to be processed is lower.
[0115] In addition, in another aspect of the present application, in some embodiments, the present application provides a video processing system, referring to Figure 4 , which is a schematic diagram of exemplary hardware and / or software of a video processing system according to some embodiments of the present application, the video processing system 400 includes: an acquisition module 401, a processing module 402 and an execution module 403, which are described as follows:
[0116] Acquisition module 401, in the present application, the acquisition module 401 is mainly used to acquire the video to be processed and convert the video to be processed into a time sequence set of image frames;
[0117] Processing module 402, in the present application, the processing module 402 is mainly used to extract each pair of similar image frames in the image frame time series set to obtain a similar image domain, and determine the reference image domain and the redundant frame change rate from the similar image domain;
[0118] The processing module 402 in the present application is further used to select a reference image in the reference image domain, determine the grayscale extreme value of a local area in the reference image, determine the local contrast amount of the reference image according to the grayscale extreme value, continue to determine the local contrast amounts of the remaining reference images in the reference image domain, and determine the reference blur of the reference image domain according to all the local contrast amounts;
[0119] In addition, the processing module 402 in the present application is also used to determine the redundancy limit of the image frame in the image frame time sequence set according to the redundant frame-to-frame change rate and the reference fuzziness;
[0120] The execution module 403 in the present application is mainly used to eliminate redundancy of the video to be processed according to the redundancy limit.
[0121] The above describes in detail the examples of the video processing method and system provided by the embodiments of the present application. It can be understood that in order to realize the above functions, the corresponding device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.
[0122] In some embodiments, the present application also provides a computer device, which includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the above-mentioned video processing method.
[0123] In some embodiments, reference Figure 5 , the dotted line in the figure indicates that the unit or the module is optional, and the figure is a structural schematic diagram of a computer device according to a video processing method provided in an embodiment of the present application. The above-mentioned video processing method in the above-mentioned embodiment can be Figure 5 The computer device 500 shown in the figure is implemented, and the computer device 500 includes at least one processor 501, a memory 502 and at least one communication unit 505. The computer device 500 can be a terminal device, a server or a chip.
[0124] The processor 501 may be a general-purpose processor or a special-purpose processor. For example, the processor 501 may be a central processing unit (CPU), which may be used to control the computer device 500, execute software programs, and process data of the software programs. The computer device 500 may also include a communication unit 505 to implement signal input (reception) and output (transmission).
[0125] For example, the computer device 500 may be a chip, the communication unit 505 may be an input and / or output circuit of the chip, or the communication unit 505 may be a communication interface of the chip, and the chip may be a component of a terminal device, a network device, or other devices.
[0126] For another example, the computer device 500 may be a terminal device or a server, and the communication unit 505 may be a transceiver of the terminal device or the server, or the communication unit 505 may be a transceiver circuit of the terminal device or the server.
[0127] The computer device 500 may include one or more memories 502, on which a program 504 is stored. The program 504 can be executed by the processor 501 to generate instructions 503, so that the processor 501 performs the method described in the above method embodiment according to the instructions 503. Optionally, data (such as a target audit model) may also be stored in the memory 502. Optionally, the processor 501 may also read the data stored in the memory 502, and the data may be stored at the same storage address as the program 504, or the data may be stored at a different storage address from the program 504.
[0128] The processor 501 and the memory 502 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of the terminal device.
[0129] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or software-based instructions in the processor 501. The processor 501 can be a CPU, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.
[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0131] For example, in some embodiments, the present application also provides a computer-readable storage medium, in which instructions or codes are stored. When the instructions or codes are run on a computer, the computer implements the above-mentioned video processing method when executing.
[0132] In summary, in the video processing method and system disclosed in the embodiments of the present application, first, a video to be processed is obtained, the video to be processed is converted into an image frame time sequence set, each pair of similar image frames in the image frame time sequence set is extracted to obtain a similar image domain, a reference image domain and a redundant frame-to-frame change rate are determined by the similar image domain, a reference image in the reference image domain is selected, the grayscale extreme value of a local area in the reference image is determined, the local contrast amount of the reference image is determined by the grayscale extreme value, the local contrast amount of the remaining reference images in the reference image domain is further determined, the reference blur of the reference image domain is determined by all the local contrast amounts, the redundant limit of the image frame in the image frame time sequence set is determined by the redundant frame-to-frame change rate and the reference blur, and the video to be processed is deduplicated according to the redundant limit, so that redundant data of the video to be processed can be reduced during the compression process of the video to be processed, and the video distortion can be reduced.
[0133] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0134] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A video processing method, characterized in that: The steps include: Obtaining a video to be processed, and converting the video to be processed into a time sequence set of image frames; Extracting each pair of similar image frames in the image frame time sequence set to obtain a similar image domain, wherein extracting each pair of similar image frames in the image frame time sequence set to obtain a similar image domain specifically includes: setting an image similarity decision value of the image frame time sequence set; Extracting similar image frames from the image frame time series set according to the image similarity decision value; All similar image frames are formed into a similar image domain, and the reference image domain and the inter-redundant frame change rate are determined by the similar image domain, wherein determining the reference image domain and the inter-redundant frame change rate by the similar image domain specifically includes: determining a reference image for each pair of similar image frames in the similar image domain; The set of all reference images is taken as the reference image domain; determining the redundancy of each pair of similar image frames in the similar image domain; Determine the redundant frame-to-frame change rate of the video to be processed according to all the redundant amounts; A reference image in the reference image domain is selected, a grayscale extreme value of a local area in the reference image is determined, and a local contrast amount of the reference image is determined by the grayscale extreme value, wherein determining the local contrast amount of the reference image by the grayscale extreme value specifically includes: Determine the brightness characteristic value of the reference image; Determine the local contrast amount of the reference image by using the brightness characteristic value and the grayscale extreme value, continue to determine the local contrast amounts of the remaining reference images in the reference image domain, and determine the reference fuzziness of the reference image domain by using all the local contrast amounts, wherein determining the reference fuzziness of the reference image domain by using all the local contrast amounts specifically includes: Obtaining a local contrast amount corresponding to each reference image in the reference image domain; Determine the brightness feature gradient of the reference image domain according to all local contrast quantities; determining a reference difference factor for the reference image domain; Determining a reference fuzziness of the reference image domain according to a local contrast amount corresponding to each reference image in the reference image domain, the brightness feature gradient and the reference difference factor; Determine the redundancy limit of the image frames in the image frame time series set by using the redundant frame-to-frame change rate and the reference fuzziness; Eliminate the redundancy of the video to be processed according to the redundancy limit; Wherein, the reference image of each pair of similar image frames in the similar image domain is determined, namely: Selecting a first pair of similar image frames in the similar image domain; The information entropy corresponding to the two image frames in the pair of similar image frames is calculated by using an existing information entropy algorithm, and the image frame with a larger information entropy is used as a reference image of the first pair of similar image frames; selecting a second pair of similar image frames in the similar image domain; The information entropy corresponding to the two image frames in the pair of similar image frames is calculated by using an existing information entropy algorithm, and the image frame with a larger information entropy is used as a reference image of the second pair of similar image frames; Continue to determine the reference images of the remaining similar image frames in the similar image domain, wherein the reference images in the reference image domain reflect the basic information corresponding to each pair of similar image frames in the similar image domain; The local contrast amount of the reference image is determined by the brightness characteristic value and the grayscale extreme value in the following manner, namely: Determine a difference between the brightness characteristic value and the grayscale extreme value, and use a ratio of an absolute value of the difference to the brightness characteristic value as a local contrast amount of the reference image; The reference fuzziness of the reference image domain is determined according to the following formula based on the local contrast amount corresponding to each reference image in the reference image domain, the brightness feature gradient and the reference difference factor: Where ε represents the reference blur, n represents the total number of reference images in the reference image domain, β(i) represents the local contrast of the i-th reference image in the reference image domain, represents the brightness feature gradient, σ represents the reference difference factor, i = 1, 2, ... n; The following steps are used to determine the redundancy limit of the image frames in the image frame time series set by using the redundant frame-to-frame change rate and the reference fuzziness: Obtaining the redundant frame-to-frame change rate; Acquiring the reference ambiguity; Determining an adjustable coefficient of the image frame timing set; Determining a compression impact factor of the image frame timing set; The redundancy limit of the image frames in the image frame timing set is determined by the redundant frame inter-frame change rate, the reference blur, the adjustable coefficient of the image frame timing set and the compression influence factor of the image frame timing set, wherein the value of the redundancy limit is determined according to the following formula: Among them, ψ represents the value of the redundant limit of the image frame in the image frame timing set, k represents the redundant frame-to-frame change rate, ε represents the reference blur, α represents the adjustable coefficient of the image frame timing set, and φ represents the compression influence factor of the image frame timing set.
2. The method according to claim 1, characterized in that Determining the grayscale extreme value of the local area in the reference image specifically includes: The reference image is evenly divided to obtain multiple local areas; Determine the internal grayscale value of each local area; The grayscale extreme value of the local area in the reference image is determined by all internal grayscale values.
3. The method according to claim 1, characterized in that Determining the reference fuzziness of the reference image domain by all local contrast quantities specifically includes: Obtaining a local contrast amount corresponding to each reference image in the reference image domain; Determine the brightness feature gradient of the reference image domain according to all local contrast quantities; determining a reference difference factor for the reference image domain; A reference fuzziness of the reference image domain is determined according to a local contrast amount corresponding to each reference image in the reference image domain, the brightness feature gradient and the reference difference factor.
4. The method according to claim 1, characterized in that A frame of image in the video to be processed is used as an image frame.
5. A video processing system, which uses the method according to any one of claims 1 to 4 to perform video processing, characterized in that: The video processing system comprises: An acquisition module, used for acquiring a video to be processed and converting the video to be processed into a time sequence set of image frames; A processing module, used for extracting each pair of similar image frames in the image frame time series set to obtain a similar image domain, and determining a reference image domain and a redundant frame inter-frame change rate from the similar image domain; The processing module is further configured to select a reference image in the reference image domain, determine a grayscale extreme value of a local area in the reference image, determine a local contrast amount of the reference image according to the grayscale extreme value, continue to determine local contrast amounts of remaining reference images in the reference image domain, and determine a reference blur of the reference image domain according to all local contrast amounts; The processing module is further used to determine the redundancy limit of the image frames in the image frame time series set according to the redundant frame-to-frame change rate and the reference fuzziness; The execution module is used to eliminate redundancy of the video to be processed according to the redundancy limit.
6. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the computer device executes the video processing method according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions or codes, and when the instructions or codes are executed on a computer, the computer implements the video processing method according to any one of claims 1 to 4.
Citation Information
Patent Citations
Video processing method and related device
CN111131825A
Video compression method and device
CN113596473A
Image video uploading method and device, storage medium and electronic equipment
CN113810654A
Video compression method and system, storage medium and equipment
CN114125463A
Video image compression redundancy elimination method based on artificial intelligence classification mode
CN115150623A
