Method, device, equipment and medium for determining image quality loss applied to video

Abstract

This application discloses a method, apparatus, device, and medium for determining image quality loss in video, addressing the problem that existing methods for measuring image quality loss are not applicable to video. Since images in video change over time, and the human eye's perception of image quality loss is not static like that of static images, this application proposes an adjustment factor based on the human eye's perception of spatial domain quality loss in static images. This adjustment factor is used to adjust the spatial domain quality loss of each frame in the video, equivalent to data smoothing, thereby obtaining an image quality loss measurement more suitable for video image quality evaluation.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a method, apparatus, device, and medium for determining image quality loss in video. Background Art

[0002] Video is an indispensable multimedia medium for recording life, capable of capturing dynamic processes of change. For example, video can visually demonstrate the progression of a story over time and directly convey the changing processes of natural landscapes.

[0003] Currently, image quality evaluation is mostly based on spatial domain quality loss. For example, the spatial domain quality loss of traditional still images... It can be obtained by methods such as PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity).

[0004] However, these methods are all applicable to static images, and their evaluation results for the quality loss of target images are not accurate enough. Therefore, how to determine the image quality loss of a target image remains to be solved. Summary of the Invention

[0005] This application provides a data storage method, apparatus, electronic device, and medium based on a key-value system, to at least solve the problem of complex reading of indexed data in the system architecture used in related technologies. The technical solution of this application is as follows:

[0006] According to a first aspect of the embodiments of this application, a method for determining image quality loss in video is provided, wherein a source video is compressed to obtain a target video, the method comprising:

[0007] Obtain the first image frame sequence from the source video;

[0008] For each target image in the first image frame sequence, a compressed image of the target image is obtained from the target video;

[0009] Determine the spatial domain quality loss between each frame of the target image and its corresponding compressed image;

[0010] Data smoothing is performed on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence.

[0011] In some embodiments, performing data smoothing on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence includes:

[0012] Perform the following for each frame of the target image:

[0013] Determine the duration of the target image in the source video;

[0014] The adjustment factor corresponding to the target image is determined based on the correspondence between duration and adjustment factor;

[0015] The spatial domain quality loss of the target image is adjusted using the adjustment factor to obtain the image quality loss of the target image.

[0016] In some embodiments, determining the spatial domain quality loss between each frame of the target image and the corresponding compressed image includes:

[0017] Perform the following for each frame of the target image:

[0018] The target image is divided into a specified number of image blocks;

[0019] The spatial domain quality loss at each image block position between the target image and the compressed image is determined, and the set of spatial domain quality loss of each image block is taken as the spatial domain quality loss between the target image and the corresponding compressed image.

[0020] In some embodiments, performing data smoothing on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence includes:

[0021] Perform the following steps for each frame of the target image:

[0022] Determine the duration of each image block of the target image in the source video;

[0023] Based on the correspondence between duration and adjustment factor, the adjustment factor corresponding to the duration of each image block is determined respectively;

[0024] Using the adjustment factor of each image patch as a weighting coefficient, the spatial domain quality loss of each image patch is weighted and summed to obtain the image quality loss of the target image.

[0025] In some embodiments, if the duration between the playback time of the target image and the current playback time is used as a reference duration, then in the correspondence:

[0026] When the reference duration is less than or equal to the duration, the adjustment factor is positively correlated with the reference duration;

[0027] When the reference duration is greater than the duration, the adjustment factor is negatively correlated with the reference duration.

[0028] In some embodiments, the correspondence is constructed based on an impulse response function that conforms to visual characteristics.

[0029] In some embodiments, before determining the adjustment factor corresponding to the duration of each image block based on the correspondence between duration and adjustment factor, the method further includes:

[0030] In the source video, images within a specified duration are obtained starting from the next frame of the target image, forming a second image frame sequence of the target image;

[0031] Determining the duration of each image block of the target image in the source video includes:

[0032] For each image block, from each frame of the second image frame sequence, a reference image block that is most similar to the image block is detected, and the matching degree between the image block and the reference image block is determined;

[0033] The duration of the image patch is determined based on the positive correlation between matching degree and duration.

[0034] In some embodiments, in the positive correlation between matching degree and duration, the cumulative matching degree of each reference image block in the first image frame sequence is linearly positively correlated with the duration.

[0035] In some embodiments, after performing data smoothing on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence, the method further includes:

[0036] The image quality loss of each target image in the first image frame sequence is weighted and summed to obtain the overall quality loss of the first image frame sequence.

[0037] In some embodiments, the method further includes:

[0038] When the overall quality loss of the first image frame sequence is higher than a preset loss value, the bit rate of the compression processing of the first image frame sequence is increased.

[0039] The first image frame sequence is recompressed using the increased bitrate.

[0040] According to a second aspect of this application, an image quality loss determination apparatus for video is provided, wherein a target video is obtained after compression processing of a source video, the apparatus comprising:

[0041] The sequence acquisition module is configured to acquire a first image frame sequence from the source video;

[0042] The image acquisition module is configured to perform the following: for each target image in the first image frame sequence, acquire a compressed image of the target image from the target video;

[0043] The spatial loss determination module is configured to determine the amount of spatial quality loss between each frame of the target image and the corresponding compressed image.

[0044] The quality loss determination module is configured to perform a data smoothing operation on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence.

[0045] In some embodiments, the quality loss determination module is configured to perform:

[0046] Perform the following for each frame of the target image:

[0047] Determine the duration of the target image in the source video;

[0048] The adjustment factor corresponding to the target image is determined based on the correspondence between duration and adjustment factor;

[0049] The spatial domain quality loss of the target image is adjusted using the adjustment factor to obtain the image quality loss of the target image.

[0050] In some embodiments, the spatial loss determination module is configured to perform:

[0051] Perform the following for each frame of the target image:

[0052] The target image is divided into a specified number of image blocks;

[0053] The spatial domain quality loss at each image block position between the target image and the compressed image is determined, and the set of spatial domain quality loss of each image block is taken as the spatial domain quality loss between the target image and the corresponding compressed image.

[0054] In some embodiments, the quality loss determination module is configured to perform:

[0055] Perform the following steps for each frame of the target image:

[0056] Determine the duration of each image block of the target image in the source video;

[0057] Based on the correspondence between duration and adjustment factor, the adjustment factor corresponding to the duration of each image block is determined respectively;

[0058] Using the adjustment factor of each image patch as a weighting coefficient, the spatial domain quality loss of each image patch is weighted and summed to obtain the image quality loss of the target image.

[0059] In some embodiments, if the duration between the playback time of the target image and the current playback time is used as a reference duration, then in the correspondence:

[0060] When the reference duration is less than or equal to the duration, the adjustment factor is positively correlated with the reference duration;

[0061] When the reference duration is greater than the duration, the adjustment factor is negatively correlated with the reference duration.

[0062] In some embodiments, the correspondence is constructed based on an impulse response function that conforms to visual characteristics.

[0063] In some embodiments, the sequence acquisition module is further configured to perform:

[0064] Before the quality loss determination module determines the adjustment factor corresponding to the duration of each image block based on the correspondence between duration and adjustment factor, images within a specified duration are obtained in the source video with the next frame of the target image as the starting frame, thus forming the second image frame sequence of the target image;

[0065] The quality loss determination module is configured to execute:

[0066] For each image block, from each frame of the second image frame sequence, a reference image block that is most similar to the image block is detected, and the matching degree between the image block and the reference image block is determined;

[0067] The duration of the image patch is determined based on the positive correlation between matching degree and duration.

[0068] In some embodiments, in the positive correlation between matching degree and duration, the cumulative matching degree of each reference image block in the first image frame sequence is linearly positively correlated with the duration.

[0069] In some embodiments, the apparatus further includes:

[0070] The overall loss determination module is configured to perform a data smoothing operation on the spatial domain quality loss of each frame of the target image in the first image frame sequence to obtain the image quality loss of each frame of the target image in the first image frame sequence, and then perform a weighted summation of the image quality loss of each frame of the target image in the first image frame sequence to obtain the overall quality loss of the first image frame sequence.

[0071] In some embodiments, the apparatus further includes:

[0072] The compression module is configured to increase the bitrate of the compression process on the first image frame sequence when the overall quality loss of the first image frame sequence is higher than a preset loss value; and to re-compress the first image frame sequence using the increased bitrate.

[0073] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising:

[0074] processor;

[0075] Memory used to store the processor's executable instructions;

[0076] The processor is configured to execute the instructions to implement the data storage method based on a key-value system as described in any one of the first aspects of the embodiments of this application.

[0077] According to a fourth aspect of the embodiments of this application, a non-volatile readable storage medium is provided, which, when executed by a computer program electronic device in the storage medium, enables the electronic device to perform the image quality loss determination method applied to video as described in any one of the first aspects of the embodiments of this application.

[0078] According to a fifth aspect of the embodiments of this application, a computer program product is provided, which, when running on an electronic device, causes the electronic device to execute the image quality loss determination method for video that implements the first aspect of the embodiments of this application and any one of the first aspects.

[0079] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects: Since the images in the video change over time, the human eye's perception of image quality loss is not as constant as that of static images. Therefore, this application proposes an adjustment factor based on the human eye's perception of the spatial domain quality loss of static images to adjust the spatial domain quality loss of each frame in the video, which is equivalent to performing data smoothing operations, so as to obtain an image quality loss that is more suitable for the evaluation of video image quality.

[0080] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0081] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0082] Figure 1 This is a schematic diagram illustrating an application scenario of an image quality loss determination method for video, according to an exemplary embodiment.

[0083] Figure 2 This is a schematic flowchart illustrating an image quality loss determination method applied to video according to an exemplary embodiment.

[0084] Figure 3 This is a schematic diagram illustrating the effect of image segmentation according to an exemplary embodiment.

[0085] Figure 4 This is a flowchart illustrating an image quality loss determination method applied to video according to an exemplary embodiment.

[0086] Figure 5 This is an illustrative diagram illustrating a method for determining image quality loss applied to video, according to an exemplary embodiment.

[0087] Figure 6 This is a comparison chart based on image quality loss, as illustrated in an exemplary embodiment.

[0088] Figure 7 This is a schematic diagram of the structure of an image quality loss determination device applied to video, according to an exemplary embodiment.

[0089] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0090] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0091] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0092] The application scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems. In the description of this application, unless otherwise stated, the word "multiple" has the following meanings.

[0093] Traditional spatial domain quality loss of still images The quality loss can be evaluated using methods such as PSNR or SSIM. The inventors discovered that the spatial domain quality loss of static images is not applicable to target images. One possible reason is that static images only provide static image information, while moving images can convey not only the information of a single static frame but also changes in image content over time. Therefore, the inventors found that when video is compressed to reduce the bitrate, a single frame is compressed. The visual loss presented to viewers before and after compression is related not only to the image content but also to the human eye's visual perception in the temporal domain. That is, on the one hand, the human eye's sensitivity to image quality loss is affected by the duration of image content stability. On the other hand, each frame in a video has only a finite duration. Therefore, an adjustment factor needs to be introduced into the definition of objective image quality loss measurement. The adjustment factor is used to quantitatively describe the intensity of the spatial domain quality loss perceived by the human eye before and after compression. Multiplying the adjustment factor by the spatial domain quality loss characterizes the quality loss perceived by the human eye.

[0094] The inventors discovered that introducing an adjustment factor based on human visual perception to adjust the spatial domain quality loss of an image is equivalent to smoothing the data. Therefore, this application proposes a method for determining image quality loss suitable for video. In this method, based on the effect of the human eye's adjustment factor on quality loss, a data smoothing operation is performed on the spatial domain quality loss calculated from static images to obtain an image quality loss evaluation scheme suitable for video.

[0095] The image quality loss determination method applied to video involved in this application will be further explained below with reference to the accompanying drawings.

[0096] first, Figure 1 This is a schematic diagram illustrating an application scenario according to an exemplary embodiment. This application scenario may include, for example, a storage system 10, a server 20, and terminal devices 30. Each terminal device 30 can be any suitable electronic device used for network access, including but not limited to computers, laptops, smartphones, tablets, or other types of terminals. The storage system 10 stores data accessed by the terminal devices, and the server 20 interacts with the terminal devices 30, retrieving data from the storage system 10 and returning it to the terminal devices 30. For example, after capturing a short video, the terminal device 30 transcodes it to obtain a low-bitrate video, which can then be transmitted to the server 20, whereby the server 20 stores the video in the storage system 10. Terminal devices 30 (e.g., between 30_1 and 30_2 or 30_N) can also communicate with each other via a network 40. The network 40 can be a broad network for information transmission, including one or more communication networks, such as wireless communication networks, the Internet, private domain networks, local area networks, metropolitan area networks, wide area networks, or cellular data networks.

[0097] The description in this application details only a single server or terminal device. However, those skilled in the art should understand that the illustrated single server 20, terminal device 30, and storage system 10 are intended to illustrate the operation of terminal devices, servers, and storage systems involved in the technical solutions of this application. The detailed description of a single terminal device, server, and storage system is at least for ease of explanation and does not imply any limitation on the number, type, or location of terminal devices and servers. It should be noted that adding additional modules to or removing individual modules from the illustrated environment will not change the underlying concept of the exemplary embodiments of this application. Furthermore, although detailed descriptions are provided for ease of explanation... Figure 1 The diagram shows a bidirectional arrow from storage system 10 to server 20, but those skilled in the art will understand that the sending and receiving of the aforementioned data can also be achieved through network 40.

[0098] In this embodiment, the source video is transcoded (i.e., compressed) to obtain a low-bitrate video. The image quality loss of a frame is the image quality loss of the same frame of the target image before and after transcoding. To improve the accuracy of the image quality loss of the target image, this application provides a method for determining the image quality loss of the target image. This method can be as follows: Figure 2 As shown, the following steps are included:

[0099] The source video is transcoded to obtain compressed images of each frame of the target image. When evaluating the image quality loss of the source video, it can be implemented as follows: Figure 2 As shown, the following steps are included:

[0100] In step 201, a first image frame sequence is obtained from the source video;

[0101] For example, if we want to obtain the image quality loss of each frame in a source video within a playback time of a cutoff value T, the first image frame sequence can be composed of the image frame sequence from the first frame of the source video up to the frame corresponding to playback time T. Of course, we can also evaluate the image quality loss of a single frame. The specific implementation will be explained later and will not be detailed here.

[0102] In step 202, for each target image in the first image frame sequence, a compressed image of the target image is obtained from the target video;

[0103] In step 203, the spatial domain quality loss between each frame of the target image and the corresponding compressed image is determined.

[0104] In step 204, a data smoothing operation is performed on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence.

[0105] For each frame of the target image, the spatial domain quality loss between the target image and the compressed image is determined based on either PSNR or SSIM methods. This application provides the following two methods for determining the spatial domain quality loss:

[0106] Method 1: Calculate the spatial quality loss between the target image of the frame and its compressed image using the entire target image as the calculation unit.

[0107] Method 2 involves dividing the target image into a specified number of image blocks; then, for each image block, determining the spatial domain quality loss at that block's location between the target image and the compressed reference image. The spatial domain quality loss of each image block is then accumulated or weighted to obtain the spatial domain quality loss of the target object.

[0108] Then, in order to obtain the final image quality loss of each frame of the target image, in step 204, a data smoothing operation can be performed on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence.

[0109] For example, data smoothing operations can be implemented as low-pass filtering in the time domain, and an adjustment factor for image quality loss based on human visual perception can be introduced to adjust the amount of spatial domain quality loss in each frame of the target image.

[0110] The adjustment factor is related to the effective duration of a frame. During implementation, a correspondence between duration and adjustment factor can be pre-established to determine the image quality loss for each target frame. The following explains how to determine the effective duration and establish this correspondence:

[0111] Methods for determining duration:

[0112] Since video has temporal content continuity, image content generally changes gradually within the source video. Therefore, the duration of image content in the source video can be determined by referring to the effective duration of the image content within an image block. For any given frame of the target image, the duration can be illustrated using an image block as an example:

[0113] You can first capture images within a specified duration from the source video, starting with the next frame of the target image, to form a second image frame sequence. This specified duration can be determined empirically, and is generally set to the average duration until the image content completely disappears.

[0114] For each image patch, from each frame in the second image frame sequence, the reference image patch most similar to that image patch is detected, and the matching degree between the image patch and its reference image patch is determined. This matching degree can be determined by the proportion of the image content of the image patch remaining in other images. For example, such as... Figure 3 As shown, let image P m For the m-th frame in the source video (i.e., the source video sequence before transcoding), P m+l It is the (m+l)th frame in the source video sequence. For P m The source image block in the image is k×k pixels in size. for The static mass loss (i.e., spatial domain mass loss). A motion prediction algorithm is used in P... m+l Obtaining from frame image The most similar image patch according to and Calculate the chromaticity and luminance information. and Normalized information matching degree between For example, the matching degree between image patches can be represented as the correlation of an array, assuming array A represents... The color values ​​(i.e., brightness and / or chromaticity information), represented by array B. If the color value is given, the matching degree can be expressed as shown in formula (1):

[0115] (A*B) / (|A|*|B|) (1)

[0116] In practice, the pixel values ​​of each pixel in an image block can be converted to the YUV color space. The differences between the Y, U, and V color components of each pixel between two image blocks are calculated, and then normalized. The matching degree between the image blocks is determined by weighted summation of the normalized differences between the color components.

[0117] After obtaining the matching degree, the duration of the image patch can be determined based on the positive correlation between the matching degree and the duration. For example, the duration of each image patch in the source video can be determined using the following formula (2):

[0118]

[0119] in, τ represents the duration of the image block; τ represents the average playback duration of a frame in the source video (e.g., if the frame rate is 30 / s, then τ is (1 / 30); l represents the l-th frame in the second image frame sequence; m represents the identifier of the image block; This represents the matching degree between the image patch and the most similar reference image patch in the l-th frame.

[0120] Furthermore, while the above example uses image patches to illustrate the duration of an image patch in the source video, in practice, the duration of the entire target image can also be determined by the matching degree of each frame in the entire target image and the second image frame sequence. That is, a frame of the target image is treated as an image patch. For example, the matching degree of each frame in the entire target image and the second image frame sequence can be expressed as the residual rate of the target image's content in other images, i.e., the image difference rate, which is less than or equal to 1. Then, (1 - difference rate) is used to obtain the matching degree, which is then used for subsequent calculations. This can be represented as shown in formula (3):

[0121]

[0122] in, τ represents the duration of the image content in the target image; τ represents the average playback duration of a frame in the source video; S represents the matching degree.

[0123] The relationship between duration and adjustment factor:

[0124] After introducing how to determine the duration, we will now introduce the correspondence between duration and adjustment factor.

[0125] After obtaining the duration, a correspondence between the duration and the adjustment factor can be established. This correspondence can be derived based on an impulse response function that conforms to visual characteristics. In this correspondence, the duration between the playback time of the target image and the current playback time is used as the reference duration. When the reference duration is less than or equal to the duration, the adjustment factor is positively correlated with the reference duration; when the reference duration is greater than the duration, the adjustment factor is negatively correlated with the reference duration. That is, this correspondence can describe the visual perception of the degree of image content retention over time to express the human eye's adjustment factor for the amount of spatial domain quality loss, and can also describe how the human eye's adjustment factor for the amount of spatial domain quality loss gradually decreases after the image content completely disappears.

[0126] Therefore, in some embodiments, this correspondence can be represented by the following formula (4):

[0127]

[0128] Wherein, t represents the reference duration; Represents the adjustment factor, the The duration is indicated by μ; μ is a constant used to represent the average perception duration of the human eye, typically 100ms ≤ μ ≤ 200ms.

[0129] In implementation, if the entire frame of the target image is not divided into image blocks according to method 1 described above, each frame of the target image can correspond to an adjustment factor. That is, for each frame of the target image, based on the pre-established correspondence between the duration and the adjustment factor for the amount of image quality loss perceived by human vision, the adjustment factor corresponding to the target image is determined, and then the spatial domain quality loss of the target image is adjusted using the adjustment factor to obtain the image quality loss of the target image. When image blocks are not divided, the calculation can be simplified and the processing efficiency improved.

[0130] In another implementation, if the target image is divided into image blocks as described in method 2 above, such as Figure 4 As shown, the method for determining the final quality loss of the target image may include the following steps:

[0131] In step 401, the target image is divided into image blocks, and the spatial domain quality loss of each image block is obtained.

[0132] In step 402: the duration of each image block of the target image in the source video is determined.

[0133] In step 403, based on the correspondence between duration and adjustment factor, the adjustment factor corresponding to the duration of each image block is determined.

[0134] In step 404, the spatial domain quality loss of each image block is accumulated or weighted and summed using the adjustment factor of each image block as the weight coefficient to obtain the image quality loss of the target image.

[0135] Determining the image quality loss of a target image by image patch can take into account the differences between image patches, making the final determined image quality loss accurate.

[0136] For the entire source video, the overall quality loss up to the playback time can be measured by the image quality loss of all target images up to any playback time. In practice, this can be achieved by directly using the sum of the final quality losses of each frame of target images as the overall quality loss, or by using a weighted sum of the final quality losses of each frame of target images as the overall quality loss. Alternatively, the average of the final quality losses of each frame of target images can be used as the overall quality loss.

[0137] like Figure 5 As shown, assuming P1, P2, and P3 are target images played sequentially in the source video frame sequence, and the playback times of each frame are T1, T2, and T3 respectively, and M1, M2, and M3 are compressed images of P1, P2, and P3 respectively, then:

[0138] t1 = T3 - T1;

[0139] t1 = T3 - T2;

[0140] t3 = T3 - T3 = 0;

[0141] Substituting t1, t2, and t3 into formula (4) respectively, we can obtain the adjustment factors of frames P1, P2, and P3 as W(t1), W(t2), and W(t3), respectively.

[0142] If the spatial domain quality losses of the three frames P1, P2, and P3 are D1, D2, and D3 respectively, then the overall quality loss of the source video frames up to time T3 is:

[0143] Dsum=D1*W(t1)+D2*W(t2)+D3*W(t3);

[0144] Since W(t3) = 0, then finally, Dsum = D1*W(t1) + D2*W(t2).

[0145] In summary, after obtaining the overall quality loss of the source video frames at any given time, the bitrate of the transcoding process can be adjusted based on this overall quality loss. For example, if the overall quality loss exceeds a preset loss threshold, it indicates that the original transcoding bitrate is too low, resulting in poor video image quality. In this case, the bitrate can be appropriately increased to re-transcode the source video frame sequence, yielding a high-quality transcoded video.

[0146] like Figure 6 The figure shown is a comparison of the image quality loss of a moving image as determined in this application compared to the image quality loss of a traditional static image after introducing the adjustment factor for image quality loss based on human visual perception. Figure 6 In the diagram, the line corresponding to "original" represents the spatial domain quality loss of each frame. The line corresponding to "corrected" represents the image quality loss after adjusting for the introduced adjustment factor. It is evident that introducing the adjustment factor is equivalent to data smoothing. After correction, the video's image quality loss better reflects the characteristics of human vision.

[0147] Based on the same inventive concept, this application also provides an image quality loss determination device for video, wherein the source video is compressed to obtain the target video, such as... Figure 7 As shown, the device includes:

[0148] The sequence acquisition module 701 is configured to acquire a first image frame sequence from the source video;

[0149] The image acquisition module 702 is configured to perform a compressed image acquisition of the target image from the target video for each target image in the first image frame sequence;

[0150] The spatial domain loss determination module 703 is configured to determine the amount of spatial domain quality loss between each frame of the target image and the corresponding compressed image.

[0151] The quality loss determination module 704 is configured to perform a data smoothing operation on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence.

[0152] In some embodiments, the quality loss determination module is configured to perform:

[0153] Perform the following for each frame of the target image:

[0154] Determine the duration of the target image in the source video;

[0155] The adjustment factor corresponding to the target image is determined based on the correspondence between duration and adjustment factor;

[0156] The spatial domain quality loss of the target image is adjusted using the adjustment factor to obtain the image quality loss of the target image.

[0157] In some embodiments, the spatial loss determination module is configured to perform:

[0158] Perform the following for each frame of the target image:

[0159] The target image is divided into a specified number of image blocks;

[0160] The spatial domain quality loss at each image block position between the target image and the compressed image is determined, and the set of spatial domain quality loss of each image block is taken as the spatial domain quality loss between the target image and the corresponding compressed image.

[0161] In some embodiments, the quality loss determination module is configured to perform:

[0162] Perform the following steps for each frame of the target image:

[0163] Determine the duration of each image block of the target image in the source video;

[0164] Based on the correspondence between duration and adjustment factor, the adjustment factor corresponding to the duration of each image block is determined respectively;

[0165] Using the adjustment factor of each image patch as a weighting coefficient, the spatial domain quality loss of each image patch is weighted and summed to obtain the image quality loss of the target image.

[0166] In some embodiments, if the duration between the playback time of the target image and the current playback time is used as a reference duration, then in the correspondence:

[0167] When the reference duration is less than or equal to the duration, the adjustment factor is positively correlated with the reference duration;

[0168] When the reference duration is greater than the duration, the adjustment factor is negatively correlated with the reference duration.

[0169] In some embodiments, the correspondence is constructed based on an impulse response function that conforms to visual characteristics.

[0170] In some embodiments, the sequence acquisition module is further configured to perform:

[0171] Before the quality loss determination module determines the adjustment factor corresponding to the duration of each image block according to the correspondence between duration and adjustment factor, images within a specified duration are obtained in the source video with the next frame of the target image as the starting frame, forming the second image frame sequence of the target image;

[0172] The quality loss determination module is configured to execute:

[0173] For each image block, from each frame of the second image frame sequence, a reference image block that is most similar to the image block is detected, and the matching degree between the image block and the reference image block is determined;

[0174] The duration of the image patch is determined based on the positive correlation between matching degree and duration.

[0175] In some embodiments, in the positive correlation between matching degree and duration, the cumulative matching degree of each reference image block in the first image frame sequence is linearly positively correlated with the duration.

[0176] In some embodiments, the apparatus further includes:

[0177] The overall loss determination module is configured to perform a data smoothing operation on the spatial domain quality loss of each frame of the target image in the first image frame sequence to obtain the image quality loss of each frame of the target image in the first image frame sequence, and then perform a weighted summation of the image quality loss of each frame of the target image in the first image frame sequence to obtain the overall quality loss of the first image frame sequence.

[0178] In some embodiments, the apparatus further includes:

[0179] The compression module is configured to increase the bitrate of the compression process on the first image frame sequence when the overall quality loss of the first image frame sequence is higher than a preset loss value; and to re-compress the first image frame sequence using the increased bitrate.

[0180] Regarding the apparatus in the above embodiments, the specific manner in which each unit executes the request has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0181] Having introduced the image quality loss determination method and apparatus for video according to exemplary embodiments of the present disclosure, a computing device according to another exemplary embodiment of the present disclosure will now be described.

[0182] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0183] In some possible implementations, the computing device according to this disclosure may include at least one processor and at least one memory. The memory stores program code that, when executed by the processor, causes the processor to perform the steps in the image quality loss determination method for video according to various exemplary embodiments of this disclosure described above. For example, the processor may perform steps in an image quality loss determination method for video.

[0184] The following reference Figure 8 To describe a computing device 130 according to such an embodiment of the present disclosure. Figure 8 The computing device 130 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0185] like Figure 8 As shown, the computing device 130 is presented in the form of a general-purpose computing device. The components of the computing device 130 may include, but are not limited to: at least one processor 131, at least one memory 132, and a bus 133 connecting different system components (including memory 132 and processor 131).

[0186] Bus 133 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0187] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0188] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0189] The computing device 130 can also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), one or more devices that enable a user to interact with the computing device 130, and / or any device that enables the computing device 130 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 135. Furthermore, the computing device 130 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 136. As shown, network adapter 136 communicates with other modules used in the computing device 130 via bus 133. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the computing device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0190] In some possible implementations, various aspects of the method for determining image quality loss in video provided in this disclosure can also be implemented in the form of a program product, which includes program code that, when the program product is run on a computer device, causes the computer device to perform the steps in the method for determining image quality loss in video according to various exemplary embodiments of this disclosure described above.

[0191] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0192] The program product of the method for determining image quality loss applied to video according to embodiments of this disclosure can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a computing device. However, the program product of this disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0193] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take many forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0194] The program code contained on the readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0195] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0196] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0197] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0198] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0200] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0201] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0202] Although preferred embodiments of this disclosure have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this disclosure.

[0203] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for determining image quality loss in video, characterized in that, The method for obtaining the target video after compressing the source video includes: Obtain the first image frame sequence from the source video; For each target image in the first image frame sequence, a compressed image of the target image is obtained from the target video; Determine the spatial domain quality loss between each frame of the target image and its corresponding compressed image; Data smoothing operation is performed on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence. The step of performing data smoothing on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence includes: Perform the following for each frame of the target image: Determine the duration of the target image in the source video; determine the adjustment factor corresponding to the target image based on the correspondence between duration and adjustment factor; adjust the spatial domain quality loss of the target image using the adjustment factor to obtain the image quality loss of the target image; The duration of the target image in the source video is determined as follows: The source video is used to obtain images within a specified duration, starting with the next frame of the target image, to form a second image frame sequence; wherein, the specified duration indicates the average duration during which the image content completely disappears; Calculate the matching degree between the target image and each frame in the second image frame sequence; The duration of the target image in the source video is determined based on the matching degree and the average playback duration of a frame in the source video.

2. The method according to claim 1, characterized in that, The step of determining the spatial domain quality loss between each frame of the target image and the corresponding compressed image includes: Perform the following for each frame of the target image: The target image is divided into a specified number of image blocks; The spatial domain quality loss at each image block position between the target image and the compressed image is determined, and the set of spatial domain quality loss of each image block is taken as the spatial domain quality loss between the target image and the corresponding compressed image.

3. The method according to claim 2, characterized in that, If the target image is divided into a specified number of image blocks, then the step of performing data smoothing operation on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence includes: Perform the following steps for each frame of the target image: Determine the duration of each image block of the target image in the source video; Based on the correspondence between duration and adjustment factor, the adjustment factor corresponding to the duration of each image block is determined respectively; Using the adjustment factor of each image patch as a weighting coefficient, the spatial domain quality loss of each image patch is weighted and summed to obtain the image quality loss of the target image. The duration of each image block of the target image in the source video is determined as follows: The source video is used to obtain images within a specified duration, starting with the next frame of the target image, to form a second image frame sequence; wherein, the specified duration indicates the average duration during which the image content completely disappears; For each image patch, the most similar reference image patch is detected from each frame in the second image frame sequence, and the matching degree between the image patch and the reference image patch is determined; The duration of the image patch is determined based on the positive correlation between the matching degree and the duration.

4. The method according to claim 1 or 3, characterized in that, If the duration between the playback time of the target image and the current playback time is used as the reference duration, then when the reference duration is less than or equal to the duration, the adjustment factor is positively correlated with the reference duration. When the reference duration is greater than the duration, the adjustment factor is negatively correlated with the reference duration.

5. The method according to claim 4, characterized in that, The correspondence is constructed based on the impulse response function that conforms to visual characteristics.

6. The method according to claim 3, characterized in that, Before determining the adjustment factor corresponding to the duration of each image block based on the correspondence between duration and adjustment factor, the method further includes: In the source video, images within a specified duration are obtained starting from the next frame of the target image, forming a second image frame sequence of the target image; Determining the duration of each image block of the target image in the source video includes: For each image block, from each frame of the second image frame sequence, a reference image block that is most similar to the image block is detected, and the matching degree between the image block and the reference image block is determined; The duration of the image patch is determined based on the positive correlation between matching degree and duration.

7. The method according to claim 6, characterized in that, In the positive correlation between the matching degree and the duration, the cumulative value of the matching degree of each reference image block in the first image frame sequence has a linear positive correlation with the duration.

8. The method according to any one of claims 1-3 and 5-7, characterized in that, After performing data smoothing on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence, the method further includes: The image quality loss of each target image in the first image frame sequence is weighted and summed to obtain the overall quality loss of the first image frame sequence.

9. The method according to claim 8, characterized in that, The method further includes: When the overall quality loss of the first image frame sequence is higher than a preset loss value, the bit rate of the compression processing of the first image frame sequence is increased. The first image frame sequence is recompressed using the increased bitrate.

10. An image quality loss determination device for video, characterized in that, The source video is compressed to obtain the target video. The device includes: The sequence acquisition module is configured to acquire a first image frame sequence from the source video; The image acquisition module is configured to perform the following: for each target image in the first image frame sequence, acquire a compressed image of the target image from the target video; The spatial loss determination module is configured to determine the amount of spatial quality loss between each frame of the target image and the corresponding compressed image. The quality loss determination module is configured to perform a data smoothing operation on the spatial domain quality loss of each frame of the target image to obtain the image quality loss of each frame of the target image in the first image frame sequence. The quality loss determination module is configured to execute: Perform the following for each frame of the target image: Determine the duration of the target image in the source video; The adjustment factor corresponding to the target image is determined based on the correspondence between duration and adjustment factor; The spatial domain quality loss of the target image is adjusted using the adjustment factor to obtain the image quality loss of the target image; The quality loss determination module is further configured to determine the duration of the target image in the source video by means of the following method: The source video is used to obtain images within a specified duration, starting with the next frame of the target image, to form a second image frame sequence; wherein, the specified duration indicates the average duration during which the image content completely disappears; Calculate the matching degree between the target image and each frame in the second image frame sequence; The duration of the target image in the source video is determined based on the matching degree and the average playback duration of a frame in the source video.

11. The apparatus according to claim 10, characterized in that, The spatial loss determination module is configured to execute: Perform the following for each frame of the target image: The target image is divided into a specified number of image blocks; The spatial domain quality loss at each image block position between the target image and the compressed image is determined, and the set of spatial domain quality loss of each image block is taken as the spatial domain quality loss between the target image and the corresponding compressed image.

12. The apparatus according to claim 11, characterized in that, When the target image is divided into a specified number of image blocks, the quality loss determination module is configured to perform: Perform the following steps for each frame of the target image: Determine the duration of each image block of the target image in the source video; Based on the correspondence between duration and adjustment factor, the adjustment factor corresponding to the duration of each image block is determined respectively; Using the adjustment factor of each image patch as a weighting coefficient, the spatial domain quality loss of each image patch is weighted and summed to obtain the image quality loss of the target image. The quality loss determination module is further configured to determine the duration of each image block of the target image in the source video in the following manner: The source video is used to obtain images within a specified duration, starting with the next frame of the target image, to form a second image frame sequence; wherein, the specified duration indicates the average duration during which the image content completely disappears; For each image patch, the most similar reference image patch is detected from each frame in the second image frame sequence, and the matching degree between the image patch and the reference image patch is determined; The duration of the image patch is determined based on the positive correlation between the matching degree and the duration.

13. The apparatus according to claim 10 or 12, characterized in that, If the duration between the playback time of the target image and the current playback time is used as a reference duration, then in the correspondence: When the reference duration is less than or equal to the duration, the adjustment factor is positively correlated with the reference duration; When the reference duration is greater than the duration, the adjustment factor is negatively correlated with the reference duration.

14. The apparatus according to claim 13, characterized in that, The correspondence is constructed based on the impulse response function that conforms to visual characteristics.

15. The apparatus according to claim 12, characterized in that, The sequence acquisition module is also configured to execute: Before the quality loss determination module determines the adjustment factor corresponding to the duration of each image block according to the correspondence between duration and adjustment factor, images within a specified duration are obtained in the source video with the next frame of the target image as the starting frame, thus forming the second image frame sequence of the target image; The quality loss determination module is configured to execute: For each image block, from each frame of the second image frame sequence, a reference image block that is most similar to the image block is detected, and the matching degree between the image block and the reference image block is determined; The duration of the image patch is determined based on the positive correlation between matching degree and duration.

16. The apparatus according to claim 15, characterized in that, In the positive correlation between the matching degree and the duration, the cumulative value of the matching degree of each reference image block in the first image frame sequence has a linear positive correlation with the duration.

17. The apparatus according to any one of claims 10-12, 14-16, characterized in that, The device further includes: The overall loss determination module is configured to perform a data smoothing operation on the spatial domain quality loss of each frame of the target image in the first image frame sequence to obtain the image quality loss of each frame of the target image in the first image frame sequence, and then perform a weighted summation of the image quality loss of each frame of the target image in the first image frame sequence to obtain the overall quality loss of the first image frame sequence.

18. The apparatus according to claim 17, characterized in that, The device further includes: The compression module is configured to increase the bitrate of the compression process on the first image frame sequence when the overall quality loss of the first image frame sequence is higher than a preset loss value; and to re-compress the first image frame sequence using the increased bitrate.

19. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image quality loss determination method for video as described in any one of claims 1-9.

20. A storage medium, characterized in that, The storage medium contains a computer program that, when executed by a processor of an electronic device, enables the electronic device to perform the image quality loss determination method for video as described in any one of claims 1-9.

21. A computer program product, characterized in that, When the computer program product is run on an electronic device, the electronic device performs the image quality loss determination method applied to video as described in any one of claims 1-9.

Images