Media file screen distortion detection method, device, electronic device and storage medium

Abstract

The present application relates to the field of image processing, and discloses a method, device, electronic device, and storage medium for detecting a distorted screen in a media file, the method comprising: obtaining a coded image frame corresponding to the media file; screening a target image frame whose coding type is forward reference or bidirectional reference from the coded image frame; obtaining a proportion value of a first type of macroblock corresponding to the target image frame in a target detection area; the first type is a macroblock type encoded within the frame; the target detection area is an image area in which the image content in the target image frame remains unchanged; determining whether a distorted screen exists in the target detection area based on a change in proportion represented by the proportion value, and whether a distorted screen exists in the target detection area represents whether the media file has a distorted screen. The present application can improve the accuracy of distorted screen detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for detecting a screen corruption of a media file. Background Art

[0002] When the screen distortion detection is performed at the bitstream layer, the media file is checked for correctness at the syntax layer. When a syntax error is identified, it is considered that the screen distortion exists.

[0003] However, if the media file needs to be transcoded, that is, decoded and then encoded, during the encoding stage, the encoder will treat the distorted screen as normal content, and the media file generated after transcoding will not have syntax errors. Based on this, when performing distorted screen detection on the transcoded media file, syntax errors cannot be detected, and distorted screens cannot be detected, resulting in distorted screens being missed. Summary of the invention

[0004] In view of the above problems, the present application provides a method, device, electronic device and storage medium for detecting a distorted screen of a media file, so as to achieve the purpose of improving the accuracy of distorted screen detection. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for detecting a distorted screen of a media file, the method comprising:

[0006] Obtaining the encoded image frame corresponding to the media file;

[0007] Filtering target image frames whose encoding type is forward reference or bidirectional reference from the encoded image frames;

[0008] Obtaining a proportion value of a first type of macroblock corresponding to the target image frame in the target detection area; the first type is an intra-frame coded macroblock type; the target detection area is an image area in which the image content in the target image frame remains unchanged;

[0009] It is determined whether there is a distorted screen in the target detection area according to a change state of the proportion represented by the proportion value, and whether there is a distorted screen in the target detection area represents whether there is a distorted screen in the media file.

[0010] In a possible implementation, determining whether there is a screen distortion in the target detection area according to a change state of the proportion represented by the proportion value includes:

[0011] From the proportion values, a proportion set corresponding to at least three consecutive target image frames is obtained, wherein the proportion set at least includes: a first proportion value corresponding to a current target image frame, a second proportion value corresponding to a target image frame of a frame before the current target image frame, and a third proportion value corresponding to a target image frame of a frame after the current target image frame;

[0012] It is determined whether a screen distortion exists in the target detection area according to the first proportion value, the second proportion value and the third proportion value.

[0013] In a possible implementation, determining whether there is a screen distortion in the target detection area according to the first proportion value, the second proportion value, and the third proportion value includes:

[0014] Obtaining a first proportion change amount of the first proportion value relative to the second proportion value and a second proportion change amount of the third proportion value relative to the first proportion value;

[0015] It is determined whether a screen distortion exists in the target detection area according to the first proportion change amount and the second proportion change amount.

[0016] In a possible implementation, determining whether there is a screen distortion in the target detection area according to the first proportion change amount and the second proportion change amount includes:

[0017] When the first proportion change is greater than or equal to a first change threshold and the second proportion change is greater than or equal to a second change threshold, it is determined that there is a screen distortion in the target detection area; the first change threshold is a value greater than 1; and the second change threshold is a value greater than 0 and less than 1;

[0018] When the first proportion change is greater than or equal to the first change threshold and the second proportion change is less than the second change threshold, it is determined that there is no screen distortion in the target detection area.

[0019] In a possible implementation, the first proportion change is: a ratio of the first proportion value to the second proportion value;

[0020] The second proportion change is: the ratio of the third proportion value to the first proportion value.

[0021] In a possible implementation, the target detection area is determined by:

[0022] Extracting multiple candidate image frames from the target image frame;

[0023] Performing region division on each candidate image frame to obtain a plurality of grid regions in each candidate image frame, wherein the grid regions divided into each candidate image frame correspond to each other in terms of grid positions;

[0024] For each of the candidate image frames, respectively, a candidate area whose second type of macroblock ratio meets the first condition is screened out; the second type is a SKIP macroblock type;

[0025] In the multiple candidate image frames, obtaining the number of candidate regions at the same grid position;

[0026] The grid area whose number of candidate areas at the same grid position meets the second condition is determined as the target detection area.

[0027] In a possible implementation, the first condition: a proportion of macroblocks of the second type is greater than or equal to a first threshold;

[0028] The second condition includes: the number of candidate areas at the same grid position is greater than or equal to a second threshold.

[0029] A second aspect of the present application provides a device for detecting a distorted screen of a media file, the device comprising:

[0030] A frame obtaining unit, used to obtain a coded image frame corresponding to a media file;

[0031] A frame screening unit, used for screening target image frames whose coding type is forward reference or bidirectional reference from the coded image frames;

[0032] A proportion obtaining unit, used to obtain a proportion value of a first type of macroblock corresponding to the target image frame in the target detection area; the first type is an intra-frame coded macroblock type; the target detection area is an image area in which the image content in the target image frame remains unchanged;

[0033] The area detection unit is used to determine whether there is a distorted screen in the target detection area according to a change state of the proportion represented by the proportion value, and whether there is a distorted screen in the target detection area represents whether there is a distorted screen in the media file.

[0034] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the method for detecting a media file screen distortion according to the first aspect or any implementation of the first aspect.

[0035] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0036] The memory is used to store computer programs;

[0037] The processor is used to execute the computer program so that the electronic device can implement the method for detecting a distorted screen of a media file according to the first aspect or any implementation manner of the first aspect.

[0038] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the method for detecting screen noise of media files according to the first aspect or any implementation of the first aspect.

[0039] By means of the above technical scheme, in a method, device, electronic device and storage medium for detecting a screen distortion of a media file provided by the present application, by obtaining a coded image frame corresponding to the media file, a screen distortion detection is performed on the media file at the code stream layer, specifically: from the coded image frame, a target image frame for forward reference or bidirectional reference is screened out, so that the proportion value of the macroblock of the intra-frame coding type corresponding to the image area with unchanged image content in the target image frame can be obtained, thereby using the proportion change state represented by the proportion value of the macroblock of the intra-frame coding type in the image area with unchanged image content to represent the change state of the image area with unchanged image content, thereby determining whether there is a screen distortion in the image area, and using whether there is a screen distortion in the image area to represent whether there is a screen distortion in the media file, thereby avoiding the situation where the media file cannot recognize a syntax error due to transcoding, resulting in a low reliability of screen distortion detection. Therefore, in the present application, screen distortion detection is realized by detecting the change state of the image area with unchanged image content in the coded image frame, without involving syntax detection, and screen distortion detection can be realized regardless of whether the media file needs to be converted, thereby improving the accuracy of screen distortion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A flowchart of a method for detecting a distorted screen of a media file provided in an embodiment of the present application;

[0042] Figure 2 This is an example diagram of a target image frame in a media file in an embodiment of the present application;

[0043] Figure 3 A partial flow chart of a method for detecting a distorted screen of a media file provided in an embodiment of the present application;

[0044] Figure 4 This is an example diagram of dividing candidate image frames in an embodiment of the present application;

[0045] Figure 5 Another partial flow chart of a method for detecting a distorted screen of a media file provided in an embodiment of the present application;

[0046] Figure 6 This is an example diagram of three consecutive target image frames in an embodiment of the present application;

[0047] Figure 7 A schematic diagram of the structure of a device for detecting a screen distortion of a media file provided in an embodiment of the present application;

[0048] Figure 8 Another structural schematic diagram of a device for detecting a screen distortion of a media file provided in an embodiment of the present application;

[0049] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0050] Fig.10 This is a flowchart for determining the target detection area in the flower screen detection of movie media files applicable to this application;

[0051] Fig.11 This is a flowchart of performing flower screen detection based on a target detection area in flower screen detection for movie media files applicable to this application. DETAILED DESCRIPTION

[0052] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0055] Reference Figure 1, is a flow chart of a method for detecting screen distortion of a media file provided in an embodiment of the present application. The method can be applied to electronic devices with image processing functions, such as local computers or servers with image processing applications deployed, and cloud servers with image processing applications deployed. The technical solution in this embodiment is mainly used to improve the reliability of screen distortion detection of media files.

[0056] Specifically, the method in this embodiment may include the following steps:

[0057] Step 101: Obtain a coded image frame corresponding to a media file.

[0058] The coded image frame may be the coded image data obtained by the media file at the bitstream layer. It should be noted that the decoding process of the media file may include entropy decoding, inverse quantization, inverse cosine transform, interpolation, loop filtering, intra-frame prediction, motion compensation, etc. In this embodiment, at the bitstream layer, it is not necessary to decode the media file into an RGB image, but the media file can be subjected to screen distortion detection at the entropy decoding stage, that is, the coded image frame corresponding to the media file is obtained, and then screen distortion detection is performed based on the coded image frame.

[0059] Step 102: Filter target image frames whose encoding type is forward reference or bidirectional reference from the encoded image frames.

[0060] The coded image frame with coding types of forward reference and bidirectional reference includes a first type of macroblock and a second type of macroblock. The first type is an intra-frame coded macroblock type, and the second type is a SKIP macroblock type, such as a SKIP macroblock type without residual and motion vector.

[0061] In a specific implementation, step 102 may start from the first frame in the coded image frame, and sequentially filter out the coded image frames whose coding type is forward reference or bidirectional reference, i.e., P frames and B frames, i.e., exclude the coded image frames whose coding type is intra-frame reference, and the remaining coded image frames are the target image frames, such as Figure 2 as shown in .

[0062] Step 103: Obtain a ratio of the first type of macroblocks in the target image frame corresponding to the target detection area.

[0063] The target detection area is an image area in which the image content in the target image frame remains unchanged.

[0064] In one implementation, the target detection area may be determined by the proportion of the second type of macroblocks included in the target image frame.

[0065] It should be noted that the percentage value refers to: the percentage of the first type of macroblocks contained in the target image frame in the target detection area among all the macroblocks contained in the target detection area. In this embodiment, the number of the first type of macroblocks contained in the target image frame in the target detection area can be divided by the number of all the macroblocks contained in the target detection area to obtain the percentage value.

[0066] Step 104: Determine whether there is a screen distortion in the target detection area according to the change state of the proportion represented by the proportion value.

[0067] Whether there is a distorted screen in the target detection area indicates whether there is a distorted screen in the media file.

[0068] Specifically, the proportion of the first type of macroblocks contained in the target detection area of ​​two adjacent target image frames represents: the change state of the proportion of the first type of macroblocks contained in the target detection area of ​​two adjacent target image frames, specifically represented by: the change in the number of the first type of macroblocks contained in the target detection area of ​​the current target image frame relative to the target image frame of the previous frame. Based on this, in step 104, it can be determined whether there is a distorted screen in the target detection area according to the change in the number of the first type of macroblocks contained in the target detection area of ​​two adjacent target image frames, so as to represent whether there is a distorted screen in the media file.

[0069] According to the above technical scheme, in a method for detecting a distorted screen of a media file provided by an embodiment of the present application, by obtaining a coded image frame corresponding to the media file, distorted screen detection is performed on the media file at the bitstream layer, specifically: from the coded image frame, a target image frame for forward reference or bidirectional reference is screened out, so that the proportion value of the macroblock of the intra-frame coding type corresponding to the image area with unchanged image content in the target image frame can be obtained, thereby using the proportion change state represented by the proportion value of the macroblock of the intra-frame coding type in the image area with unchanged image content to represent the change state of the image area with unchanged image content, thereby determining whether there is a distorted screen in the image area, and using whether there is a distorted screen in the image area to represent whether there is a distorted screen in the media file, thereby avoiding the situation where the media file cannot recognize a syntax error due to transcoding, resulting in low reliability of distorted screen detection. Therefore, in the present application, distorted screen detection is realized by detecting the change state of the image area with unchanged image content in the coded image frame, without involving syntax detection. Regardless of whether the media file needs to be converted, distorted screen detection can be realized, thereby improving the accuracy of distorted screen detection.

[0070] In one implementation, the target detection area can be obtained by: Figure 3 As shown in:

[0071] Step 301: extract multiple candidate image frames from the target image frame.

[0072] For example, in step 301, N coded image frames whose coding type is forward reference or bidirectional reference, i.e., candidate image frames, may be extracted from the coded image frames. N is a positive integer greater than 0. In order to improve accuracy, N may be set to a larger value in this embodiment.

[0073] Specifically, N can be the total number of coded image frames of the media file divided by a multiple value such as 10. The total number of coded image frames of the media file can be obtained based on the total duration and frame rate of the media file. For example, in this embodiment, the total number of frames of the media file, frame_num, is estimated based on the duration of the media file, seconds, and the frame rate, fps, specifically seconds×fps. Then, starting from the first frame of the media file, N frames of coded image frames whose coding type is forward reference or bidirectional reference are selected, where N is the quotient of frame_num divided by 10.

[0074] Step 302: performing region division on each candidate image frame to obtain a plurality of grid regions in each candidate image frame.

[0075] Among them, the grid areas divided into each candidate image frame correspond to each other with respect to the grid position. The grid areas do not overlap each other. Specifically, each candidate image frame is divided into Q*P grid areas, Q and P are both positive integers greater than 0, and Q and P can be the same or different. And each grid area divided in each candidate image frame corresponds to the corresponding grid position. That is to say, at the same grid position, the size of the grid area divided into each candidate image frame is the same. For example, Q and P are both 3, as Figure 4 As shown in , each candidate image frame is divided into 3*3 grid areas, and the grid areas divided into each candidate image frame at the upper left corner, upper right corner, lower left corner, lower right corner and other grid positions have the same size.

[0076] Step 303: For each candidate image frame, select candidate regions whose proportion of second-type macroblocks meets the first condition.

[0077] The first condition may be that the proportion of the second type of macroblocks is greater than or equal to a first threshold. The first threshold may be represented by H0, and the first threshold is a natural number greater than 0 and less than 1.

[0078] For example, there are 9 grid areas in the candidate image frame, and three grid areas are screened out, in which the proportion of the second type of macroblocks meets the first condition, and these three grid areas are the candidate areas.

[0079] Step 304: In the multiple candidate image frames, obtain the number of candidate regions at the same grid position.

[0080] Specifically, in step 304, for each grid position, in the N candidate image frames, the number of candidate regions (i.e., grid regions whose percentage of the second type of macroblocks meets the first condition) is counted. Figure 4 For example, among N frames of candidate image frames, the number of candidate image frames determined as candidate areas at the upper left grid position is counted, that is, the number of candidate areas, the number of grid positions determined as candidate areas at the upper right grid position is counted, the number of grid positions determined as candidate areas at the lower left grid position is counted, and so on, and the number of candidate areas determined at each grid position is counted.

[0081] Step 305: Determine the grid area whose number of candidate areas at the same grid position meets the second condition as the target detection area.

[0082] The second condition may be: the number of candidate regions at the same grid position is greater than or equal to a second threshold. The second threshold may be represented by H1, and the second threshold is a positive integer greater than 0 and less than or equal to N.

[0083] For example, Figure 4 For example, at the grid position in the lower left corner, there is a candidate area in the N candidate image frames that exceeds the second threshold. At this time, the grid area corresponding to the grid position in the lower left corner is determined as the target detection area, such as the upper black border area or the lower black border area of ​​a movie video.

[0084] It should be noted that there may be one or more grid areas whose number of candidate areas at the same grid position meets the second condition. In one implementation, if there are multiple grid areas whose number of candidate areas at the same grid position meets the second condition, one of them may be selected as the target detection area. Specifically, in this embodiment, the grid area with the largest number of candidate areas at the same grid position may be used as the target detection area. Alternatively, if there are multiple grid areas whose number of candidate areas at the same grid position meets the second condition, the set of all grid areas whose number of candidate areas at the same grid position meets the second condition may be referred to as the target detection area.

[0085] In one implementation, when determining whether there is a distorted screen in the target detection area in step 104, it can be implemented in the following manner: Figure 5 As shown in:

[0086] Step 501: Obtain a set of proportions corresponding to at least three consecutive target image frames from the proportion values.

[0087] The proportion set includes at least: a first proportion value corresponding to the current target image frame, a second proportion value corresponding to the previous target image frame of the current target image frame, and a third proportion value corresponding to the next target image frame of the current target image frame.

[0088] For example, Figure 6 As shown in, starting from the first frame of the coded image frame whose coding type is forward reference or bidirectional reference, the proportion of the first type of macroblocks contained in the target detection area of ​​each of the three consecutive target image frames is obtained, such as the first proportion of the first type of macroblocks contained in the target detection area of ​​the current frame, which can be represented by R(n), the second proportion of the first type of macroblocks contained in the target detection area of ​​the previous frame of the current frame, which can be represented by R(n-1), and the third proportion of the first type of macroblocks contained in the target detection area of ​​the subsequent frame of the current frame, which can be represented by R(n+1).

[0089] Step 502: Determine whether there is a screen distortion in the target detection area according to the first ratio, the second ratio and the third ratio.

[0090] Specifically, in step 501, starting from the first frame in the target image frame, a ratio set corresponding to three consecutive target image frames can be obtained, and according to the first ratio value, the second ratio value and the third ratio value in the ratio set, it is determined whether there is a flower screen in the target detection area of ​​the three consecutive target image frames; if there is a flower screen in the target detection area of ​​the three consecutive target image frames, then the current process can be terminated; if there is no flower screen in the target detection area of ​​the three consecutive target image frames, then return to step 501, starting from the second frame in the target image frame, the ratio set corresponding to the three consecutive target image frames is obtained. According to the first proportion value, the second proportion value and the third proportion value in the proportion set, it is determined whether the new three consecutive target image frames have flower screens in the target detection area; and so on, until it is determined that the three consecutive target image frames have flower screens in the target detection area, that is, the media file has flower screens, or, starting from the third to last frame of the target image frame, the proportion set corresponding to the three consecutive target image frames obtained determines that the three consecutive target image frames still do not have flower screens in the target detection area, then it can be determined that each target image frame does not have a flower screen in the target detection area, that is, the media file does not have a flower screen.

[0091] In one implementation, when determining whether there is a distorted screen in the target detection area in step 502, it can be implemented in the following manner:

[0092] First, a first ratio change of the first ratio value relative to the second ratio value and a second ratio change of the third ratio value relative to the first ratio value are obtained. Then, according to the first ratio change and the second ratio change, it is determined whether there is a screen distortion in the target detection area.

[0093] The first proportion change may be a ratio of the first proportion value to the second proportion value, or the first proportion change may be an absolute value of the difference between the first proportion value and the second proportion value. The second proportion change may be a ratio of the third proportion value to the first proportion value, or the second proportion change may be an absolute value of the difference between the third proportion value and the first proportion value.

[0094] Specifically, in step 502, the magnitude relationship between the first proportion change amount and the second proportion change amount and their respective thresholds may be determined to determine whether there is a screen distortion in the target detection area.

[0095] For example, taking the case where the first proportion change is the ratio of the first proportion value to the second proportion value, and the second proportion change is the ratio of the third proportion value to the first proportion value, when the first proportion change is greater than or equal to the first change threshold and the second proportion change is greater than or equal to the second change threshold, it is determined that there is a screen distortion in the target detection area.

[0096] When the first proportion change is greater than or equal to the first change threshold and the second proportion change is less than the second change threshold, it is determined that there is no screen distortion in the target detection area.

[0097] The first change threshold may be a value greater than 1, and the second change threshold may be a value greater than 0 and less than 1. For example, the first change threshold may be represented by H2, and the value of H2 may be 2; the second change threshold may be represented by H3, and the value of H3 may be 0.3.

[0098] For example, if the ratio of R(n) to R(n-1) is greater than H2, and the ratio of R(n+1) to R(n) is greater than H3, then it can be determined that there is a flower screen in the current three consecutive target image frames, that is, there is a flower screen in the media file; if the ratio of R(n) to R(n-1) is greater than H2, and the ratio of R(n+1) to R(n) is less than or equal to H3, it can be determined that there is no flower screen in the current three consecutive target image frames; then, starting from the next target image frame, three consecutive target image frames are obtained again, and then the new R(n), R(n-1), and R(n+1) are compared until there is a flower screen in the three consecutive target image frames, that is, there is a flower screen in the media file, and the detection process is terminated, or until there is no flower screen in the last three consecutive target image frames obtained, that is, there is no flower screen in the media file, and the detection process is terminated.

[0099] A method for detecting a distorted screen of a media file provided in an embodiment of the present application is described above. A device for executing the method for detecting a distorted screen of a media file is described below.

[0100] See also Figure 7 , Figure 7A schematic diagram of the structure of a device for detecting screen distortion of a media file provided in an embodiment of the present application. The device can be configured in an electronic device with an image processing function, such as a local computer or server with an image processing application deployed, or a cloud server with an image processing application deployed. The technical solution in this embodiment is mainly used to improve the reliability of screen distortion detection of media files.

[0101] Specifically, the device in this embodiment may include the following units:

[0102] The frame obtaining unit 701 is used to obtain the coded image frame corresponding to the media file;

[0103] A frame screening unit 702 is used to screen target image frames whose coding type is forward reference or bidirectional reference from the coded image frames;

[0104] The proportion obtaining unit 703 is used to obtain the proportion value of the first type of macroblocks corresponding to the target image frame in the target detection area; the first type is the macroblock type of intra-frame coding; the target detection area is an image area in the target image frame where the image content remains unchanged;

[0105] The area detection unit 704 is used to determine whether there is a distorted screen in the target detection area according to a change state of the proportion represented by the proportion value, and whether there is a distorted screen in the target detection area represents whether there is a distorted screen in the media file.

[0106] According to the above technical scheme, in a media file flower screen detection device provided by an embodiment of the present application, by obtaining the coded image frame corresponding to the media file, flower screen detection is performed on the media file at the code stream layer, specifically: from the coded image frame, a target image frame for forward reference or bidirectional reference is screened out, so that the proportion value of the macroblock of the intra-frame coding type corresponding to the image area with unchanged image content in the target image frame can be obtained, thereby using the proportion change state represented by the proportion value of the macroblock of the intra-frame coding type in the image area with unchanged image content to represent the change state of the image area with unchanged image content, thereby determining whether there is a flower screen in the image area, and using the existence of a flower screen in the image area to represent whether there is a flower screen in the media file, thereby avoiding the situation where the media file cannot recognize syntax errors due to transcoding, resulting in low reliability of flower screen detection. Therefore, in the present application, flower screen detection is realized by detecting the change state of the image area with unchanged image content in the coded image frame, without involving syntax detection. Regardless of whether the media file needs to be converted, flower screen detection can be realized, thereby improving the accuracy of flower screen detection.

[0107] In one implementation, the area detection unit 704 is specifically used to: obtain a set of proportions corresponding to at least three consecutive frames of the target image frames from the proportion values, the set of proportions including at least: a first proportion value corresponding to the current target image frame, a second proportion value corresponding to the target image frame in the previous frame of the current target image frame, and a third proportion value corresponding to the target image frame in the next frame of the current target image frame; determine whether there is a screen distortion in the target detection area based on the first proportion value, the second proportion value and the third proportion value.

[0108] Among them, when the area detection unit 704 determines whether there is a flower screen in the target detection area according to the first proportion value, the second proportion value and the third proportion value, it is specifically used to: obtain a first proportion change amount of the first proportion value relative to the second proportion value and a second proportion change amount of the third proportion value relative to the first proportion value; determine whether there is a flower screen in the target detection area according to the first proportion change amount and the second proportion change amount. For example, when the first proportion change amount is greater than or equal to the first change amount threshold and the second proportion change amount is greater than or equal to the second change amount threshold, it is determined that there is a flower screen in the target detection area; the first change amount threshold is a value greater than 1; the second change amount threshold is a value greater than 0 and less than 1; when the first proportion change amount is greater than or equal to the first change amount threshold and the second proportion change amount is less than the second change amount threshold, it is determined that there is no flower screen in the target detection area.

[0109] Optionally, the first proportion change is: the ratio of the first proportion value to the second proportion value; the second proportion change is: the ratio of the third proportion value to the first proportion value.

[0110] In one implementation, the device in this embodiment may further include the following units: Figure 8 As shown in:

[0111] The detection determination unit 705 is used to extract multiple frames of candidate image frames from the target image frame; perform region division on each frame of the candidate image frame to obtain multiple grid areas in each frame of the candidate image frame, and the grid areas divided into each frame of the candidate image frame correspond to each other in terms of grid positions; for each frame of the candidate image frame, screen out candidate areas whose proportion of second-type macroblocks meets the first condition; the second type is the SKIP macroblock type; in the multiple frames of the candidate image frames, obtain the number of candidate areas at the same grid position; and determine the grid areas whose number of candidate areas at the same grid position meets the second condition as target detection areas.

[0112] Among them, the first condition includes: the proportion of macroblocks of the second type is greater than or equal to a first threshold; the second condition includes: the number of candidate areas at the same grid position is greater than or equal to a second threshold.

[0113] It should be noted that the specific implementation method of each unit in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0114] An embodiment of the present application further provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0115] The memory is used to store computer programs;

[0116] The processor is used to execute the computer program so that the electronic device can implement the method for detecting the screen distortion of the media file described in the embodiment of the present application.

[0117] refer to Fig. 9 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, cloud servers, etc. Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0118] like Fig. 9 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0119] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a memory card, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0120] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the methods for detecting screen distortion of a media file provided in the embodiments of the present application.

[0121] A computer storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any one of the media file flower screen detection methods provided in this embodiment.

[0122] Taking the media files of a movie that needs to be published to a video server as an example, the technical solution of the present application is illustrated below:

[0123] The present application proposes a method for detecting screen noise in media files. The method for detecting screen noise in the present application is a method for detecting screen noise in the bitstream layer without reference. The method obtains the target detection area, and for media files that meet the detection conditions (the number of target detection areas is 0), the method solves the problem of missed detection of screen noise caused by transcoding of such media files by using the distribution characteristics of SKIP macroblocks and intra-frame coded macroblocks in the bitstream layer. Compared with the existing detection at the bitstream layer without reference, the method has a lower missed detection rate of screen noise, that is, the reliability and accuracy of screen noise detection are higher. The method in the present application is performed at the bitstream layer, and has the characteristics of low computational overhead, fast detection speed, and compatibility with the existing bitstream layer detection.

[0124] This application is mainly divided into the following steps:

[0125] 1. After reading the bitstream, the coded image frame is divided into M non-overlapping sub-regions (i.e., the grid regions mentioned above), where M is a positive integer greater than 0. For the forward reference video frame (P frame) or the bidirectional reference video frame (B frame) in the bitstream, i.e., the target image frame mentioned above, when the ratio of the number of SKIP macroblocks in a sub-region to the total number of macroblocks in the sub-region (i.e., the proportion value mentioned above) reaches a threshold value H0, the sub-region is defined as a candidate sub-region (i.e., the candidate region mentioned above), otherwise it is a non-candidate sub-region.

[0126] Second, starting from the beginning of the bitstream of the media file to be detected, N video frames with the coding type of forward reference or bidirectional reference are cumulatively selected, and the M sub-regions of each frame are judged in turn according to step 1 whether they are candidate sub-regions. The N video frames can be continuous in time or can be selected arbitrarily from the timeline of the media file.

[0127] 3. Count the number of candidate sub-regions in each of the N video frames identified as a candidate sub-region by spatial position (i.e., the grid position in the previous text). Candidate sub-regions at the same spatial position are counted in the same count. Based on this, traverse each candidate sub-region. When the count of a candidate sub-region reaches H1, the candidate sub-region is identified as a target detection sub-region (i.e., the grid region where the number of candidate regions at the same grid position in the previous text meets the second condition). The set of all target detection sub-regions can be called a target detection region, such as Fig.10 as shown in .

[0128] 4. If the number of target detection sub-areas of the media file is 0, that is, the detection condition is not met, the method of the present application is not used for subsequent flower screen detection, and image layer flower screen detection or other methods are used instead for flower screen detection.

[0129] 5. Traverse each forward reference video frame and bidirectional reference video frame of the media file. Calculate the ratio R(n) of the number of intra-frame coded macroblocks in the target detection area of ​​the current frame n to the total number of macroblocks in the area. Calculate the above ratio R(n-1) of the previous forward reference video frame or bidirectional reference video frame on the time axis of the current frame. Calculate the above ratio R(n+1) of the next forward reference video frame or bidirectional reference video frame on the time axis of the current frame. When the ratio of R(n) to R(n-1) is greater than H2, and the ratio of R(n+1) to R(n) is greater than H3, it is determined that the current frame n has a distorted screen.

[0130] In summary, the present application can first determine whether there is a long-term static area in the picture (ie, video frame) through the distribution of SKIP and use this area as the target detection area. When such areas are compressed and encoded using forward reference or bidirectional reference, the encoder will choose inter-frame encoding rather than intra-frame encoding to obtain higher compression efficiency. When a flower screen appears in the target detection area (directly or diffused from other areas to the target detection area), since the flower screen picture has a certain randomness, it destroys the coherence of the original picture content, so that the encoder often uses intra-frame coding for compression during transcoding. By judging whether the increase in the proportion of intra-frame coding in the target detection area occurs in two frames that are consecutive in time, a suspicious flower screen picture can be obtained in the present application. Further, by analyzing the proportion of intra-frame coding in the target detection area of ​​the third frame of video frame that is consecutive in time, the present application can avoid identifying normal scene switching as a flower screen, such as the target detection area of ​​the first frame is the bottom black border area of ​​the picture, and the second frame has subtitles in this area. The appearance of subtitles increases the number of intra-frame codes in this area. Considering that subtitles often last for a certain period of time, the third frame will still be a subtitle. At this time, the third frame has a high correlation with the second frame, and the amount of intra-frame coding will be significantly reduced. The above judgment method will not be judged as a flower screen.

[0131] It can be seen that when a media file is transcoded due to business needs (decoding and then encoding), although the screen is distorted due to syntax errors in the decoding stage, the encoder regards the distorted screen as normal content in the encoding stage, and the media file generated after transcoding will not have syntax errors, and it will only appear as a distorted screen perceived by the human eye. This feature makes it impossible for the code stream layer distorted screen detection based on syntax error detection to identify the distorted screen after transcoding, resulting in missed reports. The method of the present application utilizes the destruction of the continuity of the original picture by the distorted screen and the encoding characteristics of the distorted screen by the encoder in the transcoding stage to establish a distorted screen detection method based on the intra-frame coded macroblock type, which has a lower distorted screen missed report rate than the existing non-reference code stream layer detection method. At the same time, the method of the present application is carried out at the code stream layer, and has the characteristics of low computational overhead, fast detection speed, and compatibility with existing code stream layer detection.

[0132] An embodiment of the present application is described below:

[0133] This embodiment can be divided into two major steps: the first step is to obtain the target detection area, and the second step is to perform screen distortion detection based on the target detection sub-area when the target detection sub-area is not zero.

[0134] The first step, reference Fig.10 As shown:

[0135] A1. Divide the coded image frame into M sub-regions in units of 32x32, where M is related to the video resolution. When the video width and height are not multiples of 32 and the sub-region size at the right or bottom border is less than 32x32, the width and height of the sub-region can be expanded.

[0136] B1. For the forward reference video frame or bidirectional reference video frame in the bitstream, when the ratio of the number of SKIP macroblocks in a sub-region to the total number of macroblocks in the region reaches H0, the sub-region is identified as a candidate sub-region of the video frame. Here, H0 takes the value of 1.0. For media files in H264 format, the macroblock here is defined as a 16*16 macroblock size. For media files in H265 format, the macroblock is defined as CTU (coding tree unit).

[0137] C1. Estimate the total number of frames in the file frame_num as seconds*fps based on the duration seconds and frame rate fps of the media file to be detected. Select N frames from the beginning of the media file as forward reference video frames or bidirectional reference video frames, where N is frame_num / 10.

[0138] D1. Traverse N frames and determine whether the M sub-regions of each frame are candidate sub-regions according to step A1 and step B1. Each frame forms a two-dimensional array of M elements consisting of 0 and 1. Each element in the array corresponds to a sub-region. 1 indicates that the sub-region is a candidate sub-region, and 0 indicates that the sub-region is not a candidate sub-region.

[0139] E1. Traverse the two-dimensional array of the above N frames, and count the count of each sub-region identified as a candidate sub-region. Sub-regions in the same spatial position are counted in the same count. When the count of a sub-region reaches H1, the sub-region is deemed to belong to the target detection sub-region. H1 is N multiplied by 0.98. The maximum value of H1 is N. The closer to N, the stricter the selection of the target detection area, and the lower the false alarm rate, but at the same time, the fewer film sources are eligible for detection using the method of this application.

[0140] F1. If the number of target detection sub-regions of the media file is 0, this method is not used for subsequent screen distortion detection, and image layer screen distortion detection is used instead. The collection of all target detection sub-regions is called a target detection region.

[0141] After the target detection area is obtained in the above steps A1 to F1, the formal detection process begins. Fig.11 as shown in .

[0142] A2. Traverse each forward reference video frame (P frame) and bidirectional reference video frame (B frame) of the media file until the media file ends or a distorted screen occurs, and then end the detection. Calculate the ratio R(n) of the number of intra-frame coded macroblocks in the target detection area of ​​the current frame n to the total number of macroblocks in the area. Calculate the above ratio R(n-1) of the previous forward reference video frame or bidirectional reference video frame on the time axis of the current frame. Calculate the above ratio R(n+1) of the next forward reference video frame or bidirectional reference video frame on the time axis of the current frame.

[0143] B2. When R(n) / R(n-1) is greater than the threshold H2, and R(n+1) / R(n) is greater than the threshold H3, it is determined that the current frame n has a distorted screen and the traversal ends. Here, H2 takes the value of 2.0 and H3 takes the value of 0.3. When calculating the above ratios, it is necessary to ensure that the denominator value is not 0, and H2 and H3 are taken based on experience.

[0144] In summary, this application proposes a method for detecting screen distortion in media files. This method has the ability to detect screen distortion media files after transcoding. At the same time, this method does not require decoding or time-consuming image layer calculations, and has obvious performance advantages. Moreover, this application uses stream layer SKIP and intra-frame coding distribution features as the basis for judging screen distortion.

[0145] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0146] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0147] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0148] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for detecting a distorted screen of a media file, characterized in that: The method comprises: Obtaining the encoded image frame corresponding to the media file; Filtering target image frames whose encoding type is forward reference or bidirectional reference from the encoded image frames; Obtaining a proportion value of a first type of macroblock corresponding to a target detection area of ​​the target image frame; the first type is an intra-frame coded macroblock type; the target detection area is a grid area in which the number of candidate areas at the same grid position in the target image frame meets a second condition; the candidate area is an image area in which a proportion value of a second type of macroblock contained in the target image frame meets the first condition; the second type is a SKIP macroblock type; According to the proportion change state represented by the proportion value corresponding to at least three consecutive frames of the target image frames, it is determined whether there is a distorted screen in the target detection area, and whether there is a distorted screen in the target detection area represents whether there is a distorted screen in the media file.

2. The method according to claim 1, characterized in that: Determining whether there is a screen distortion in the target detection area according to a change state of the proportion represented by the proportion values ​​corresponding to at least three consecutive target image frames includes: From the proportion values, a proportion set corresponding to at least three consecutive target image frames is obtained, wherein the proportion set at least includes: a first proportion value corresponding to a current target image frame, a second proportion value corresponding to a target image frame of a frame before the current target image frame, and a third proportion value corresponding to a target image frame of a frame after the current target image frame; It is determined whether a screen distortion exists in the target detection area according to the first proportion value, the second proportion value and the third proportion value.

3. The method according to claim 2, characterized in that Determining whether there is a screen distortion in the target detection area according to the first proportion value, the second proportion value, and the third proportion value includes: Obtaining a first proportion change amount of the first proportion value relative to the second proportion value and a second proportion change amount of the third proportion value relative to the first proportion value; It is determined whether a screen distortion exists in the target detection area according to the first proportion change amount and the second proportion change amount.

4. The method according to claim 3, characterized in that Determining whether there is a screen distortion in the target detection area according to the first proportion change amount and the second proportion change amount includes: When the first proportion change is greater than or equal to a first change threshold and the second proportion change is greater than or equal to a second change threshold, it is determined that there is a screen distortion in the target detection area; the first change threshold is a value greater than 1; and the second change threshold is a value greater than 0 and less than 1; When the first proportion change is greater than or equal to the first change threshold and the second proportion change is less than the second change threshold, it is determined that there is no screen distortion in the target detection area.

5. The method according to claim 3, characterized in that: The first proportion change is: the ratio of the first proportion value to the second proportion value; The second proportion change is: the ratio of the third proportion value to the first proportion value.

6. The method according to claim 1, characterized in that The target detection area is determined by: Extracting multiple candidate image frames from the target image frame; Performing region division on each candidate image frame to obtain a plurality of grid regions in each candidate image frame, wherein the grid regions divided into each candidate image frame correspond to each other in terms of grid positions; For each of the candidate image frames, respectively, a candidate area whose proportion of macroblocks of the second type meets the first condition is screened out; The second type is a SKIP macroblock type; In the multiple candidate image frames, obtaining the number of candidate regions at the same grid position; The grid area whose number of candidate areas at the same grid position meets the second condition is determined as the target detection area.

7. The method according to claim 6, characterized in that The first condition: the proportion of macroblocks of the second type is greater than or equal to a first threshold; The second condition includes: the number of candidate areas at the same grid position is greater than or equal to a second threshold.

8. A device for detecting a distorted screen of a media file, characterized in that: The device comprises: A frame obtaining unit, used to obtain a coded image frame corresponding to a media file; A frame screening unit, used for screening target image frames whose coding type is forward reference or bidirectional reference from the coded image frames; A proportion obtaining unit is used to obtain a proportion value of a first type of macroblock corresponding to a target detection area of ​​the target image frame; the first type is an intra-frame coded macroblock type; the target detection area is a grid area in which the number of candidate areas at the same grid position in the target image frame meets a second condition; the candidate area is an image area in which a proportion value of a second type of macroblock contained in the target image frame meets the first condition; the second type is a SKIP macroblock type; The area detection unit is used to determine whether there is a distorted screen in the target detection area according to the proportion change state represented by the proportion value corresponding to at least three consecutive frames of the target image frames, and whether there is a distorted screen in the target detection area indicates whether there is a distorted screen in the media file.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the method for detecting screen distortion of a media file as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the method for detecting a distorted screen of a media file as described in any one of claims 1 to 7.

Images