Fancy screen detection method, device and equipment, medium and program product

Through real-time transmission parameter comparison and image block similarity matching, video frames are quickly filtered, which solves the problem of video screen detection too long and realizes efficient screen detection.

CN119094733APending Publication Date: 2024-12-06MIGU VIDEO TECH CO LTD +2
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411146764.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing video screen detection method requires full detection of the entire video content, resulting in too long detection time, a single frame processing time exceeding 1s, and the entire video frame traversal calculation is much more than 3 hours.

Method used

By obtaining the real-time transmission parameters of the target video and comparing the preset parameters, the distorted video frames are filtered out, and the image blocks are selected in the target area of ​​each distorted frame to compare the similarity with the pre-stored flower screen image to determine whether the frame is Bloom the screen to avoid traversal calculations of all video frames.

Benefits of technology

Quickly collecting video frames that may cause distortion, saving calculation time and cost, improving the efficiency of screen detection, and solving the problem of excessive screen detection time in the prior art.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094733A_ABST
    Figure CN119094733A_ABST
Patent Text Reader

Abstract

The invention provides a blurred screen detection method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring a real-time transmission parameter of a target video; comparing the real-time transmission parameter with a preset target transmission parameter, and screening distorted video frames according to a comparison result; selecting a target image block from a preset number of target areas of each screened distorted video frame; and when the similarity between one target image block and a pre-stored blurred screen image exceeds a preset similarity threshold, judging that the distorted video frame corresponding to the target image block is a blurred screen. According to the method and the device, the video frames which possibly generate video distortion are quickly acquired through analysis of the transmission parameters, whether a blurred screen exists or not does not need to be traversed and calculated for all the video frames, the image blocks in the distorted video frames are compared with the blurred screen images, and the whole video frames are not compared, so that the calculation time is shortened, and the calculation cost is saved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video detection technology, and in particular to a method, device, equipment, medium and program product for detecting a screen distortion. Background Art

[0002] Video distorted screen usually refers to the interference or distortion of the video signal, resulting in messy images or colors on the screen. Video distorted screen is the most important and most common problem faced by major video companies.

[0003] During the video transmission process, problems such as encoding, network, video stream encoding parameter changes, hardware compatibility, etc. may cause video distorted screen. In order to ensure the quality of video signal transmission and display, improve user experience, and ensure device stability, distorted screen detection is required to prevent the expansion of faults.

[0004] Existing screen noise detection technology usually requires full detection of the entire video content, and uses edge detection or gradient detection methods when analyzing each frame in the video. The processing time of a single frame exceeds 1 second, and the complete traversal and calculation of the entire video frame exceeds 3 hours, so the detection takes a lot of time. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment, medium and program product for detecting video noise screen to solve the problem that the existing video noise screen detection takes a lot of time.

[0006] In order to solve the above technical problems, this application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for detecting a screen distortion, including:

[0008] Obtain the real-time transmission parameters of the target video;

[0009] Comparing the real-time transmission parameters with preset target transmission parameters of the target video, and filtering out distorted video frames from the target video according to the comparison result;

[0010] A target image block is selected from each preset number of target areas of each filtered distorted video frame. When the similarity between one of the target image blocks and a pre-stored flower screen image exceeds a preset similarity threshold, the distorted video frame corresponding to the target image block is judged to be a flower screen.

[0011] Optionally, the real-time transmission parameter includes a real-time bit rate, and the real-time transmission parameter of the target video includes:

[0012] Obtaining preset target frame rate information of each frame of the target video;

[0013] The data sizes of all frames within one second are added together according to the preset target frame rate information to obtain the real-time bit rate.

[0014] Optionally, the real-time transmission parameter includes a real-time frame number, and the real-time transmission parameter of the target video is obtained, including:

[0015] Obtaining preset target frame rate information and a timestamp of each frame of the target video;

[0016] Obtaining the inter-frame duration according to the timestamps of the two adjacent frames before and after the target video;

[0017] According to the preset target frame rate information and the inter-frame duration, a counter is used to calculate the number of frames received per second to obtain the real-time frame number.

[0018] Optionally, comparing the real-time transmission parameter with a preset target transmission parameter, and filtering the distorted video frame according to the comparison result, includes:

[0019] Comparing the real-time transmission parameter with a preset target transmission parameter, and when the real-time transmission parameter does not match at least one of the preset target transmission parameters, recording a video frame corresponding to the current real-time transmission parameter as a distorted video frame;

[0020] Counting all the distorted video frames to obtain a distorted point array;

[0021] The data frames of a preset time period before and after each distorted point video frame in the distorted point array are used as distorted video frames.

[0022] Optionally, selecting a target image block from each of the preset number of target areas of each filtered distorted video frame includes:

[0023] Dividing each of the distorted video frames into a preset number of equal parts with a constant width and a constant length, to obtain at least one image block arranged in sequence;

[0024] Each of the distorted video frames is divided into three regions, namely, upper, middle and lower regions, according to the sequence, and an image block is selected from each of the three regions, namely, upper, middle and lower regions, as a target image block.

[0025] Optionally, before determining the similarity between the target image block and the flower screen image in the pre-built flower screen model, the method further includes:

[0026] The target image block is expanded using an expansion kernel factor.

[0027] In a second aspect, an embodiment of the present application provides a screen distortion detection device, including:

[0028] An acquisition module is used to acquire real-time transmission parameters of a target video;

[0029] A processing module, used for comparing the real-time transmission parameters with preset target transmission parameters of the target video, and filtering out distorted video frames from the target video according to the comparison result;

[0030] A judgment module is used to select a target image block from a preset number of target areas of each filtered distorted video frame. When the similarity between one of the target image blocks and a pre-stored distorted screen image exceeds a preset similarity threshold, the distorted video frame corresponding to the target image block is judged to be a distorted screen.

[0031] Optionally, the real-time transmission parameter includes a real-time bit rate, and the acquisition module includes:

[0032] A first acquisition submodule is used to acquire preset target frame rate information of each frame of the target video;

[0033] The first processing submodule is used to add the data sizes of all frames within one second according to the preset target frame rate information to obtain the real-time bit rate.

[0034] Optionally, the real-time transmission parameter includes a real-time frame number, and the acquisition module includes:

[0035] A second acquisition submodule is used to acquire preset target frame rate information of each frame of the target video and a timestamp of each frame;

[0036] A second processing submodule is used to obtain the inter-frame duration according to the timestamps of the two adjacent frames before and after the target video;

[0037] The third processing submodule is used to calculate the number of frames received per second using a counter according to the preset target frame rate information and the inter-frame time length to obtain the real-time frame number.

[0038] Optionally, the processing module includes:

[0039] The distortion processing submodule is used to compare the real-time transmission parameters with the preset target transmission parameters. When the real-time transmission parameters do not match at least one of the preset target transmission parameters, the video frame corresponding to the current real-time transmission parameter is recorded as a distortion video frame; all the distortion video frames are counted to obtain a distortion array; and the data frames of the preset time period before and after each distortion video frame in the distortion array are taken as distorted video frames.

[0040] Optionally, the judging module includes:

[0041] a fourth processing submodule, configured to divide the distorted video frame into a preset number of equal parts with unchanged width and unchanged length, to obtain at least one image block arranged in sequence;

[0042] The fifth processing submodule is used to divide each of the distorted video frames into three regions of upper, middle and lower regions according to the sequence, and select an image block from each of the three regions of upper, middle and lower regions of the distorted video frame as a target image block.

[0043] Optionally, also include:

[0044] The expansion module is used to perform expansion processing on the target image block using an expansion kernel factor.

[0045] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the flower screen detection method as described in any one of the first aspects.

[0046] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the flower screen detection method as described in any one of the first aspects are implemented.

[0047] In a fifth aspect, an embodiment of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the flower screen detection method as described in any one of the first aspects.

[0048] In the present application, by analyzing the transmission parameters, video frames that may be distorted are quickly collected without traversing all video frames to calculate whether they are flower screens. By comparing the image blocks in the distorted video frames with the flower screen images, the entire video frames are not compared, which improves the calculation time, saves the calculation cost, and solves the problem that the existing video flower screen detection takes a lot of time. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0050] Figure 1 This is a flow chart of a method for detecting a screen distortion provided by an embodiment of the present application;

[0051] Figure 2It is an overall flow chart of a screen distortion detection method provided in an embodiment of the present application;

[0052] Figure 3 It is a structural schematic diagram of a screen flower detection device provided in an embodiment of the present application;

[0053] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] Please refer to Figure 1 , the embodiment of the present application provides a method for detecting a screen distortion, including:

[0056] Step 11: Obtain the real-time transmission parameters of the target video;

[0057] In an embodiment of the present application, optionally, the real-time transmission parameter includes a real-time bit rate, where the bit rate refers to the size of data transmitted per unit time. In this embodiment, the real-time bit rate refers to the sum of the data sizes of all video frames transmitted within the current second;

[0058] The step of obtaining the real-time transmission parameters of the target video includes:

[0059] Obtaining preset target frame rate information of each frame of the target video;

[0060] The data sizes of all frames within one second are added together according to the preset target frame rate information to obtain the real-time bit rate.

[0061] In an embodiment of the present application, optionally, the real-time transmission parameter includes a real-time frame number, where the frame number is the average number of video frames displayed per second, and the real-time frame number refers to the number of frames collected within the current second.

[0062] The step of obtaining the real-time transmission parameters of the target video includes:

[0063] Obtaining preset target frame rate information and a timestamp of each frame of the target video;

[0064] Obtaining the inter-frame duration according to the timestamps of the two adjacent frames before and after the target video;

[0065] According to the preset target frame rate information and the inter-frame duration, a counter is used to calculate the number of frames received in the current second to obtain the real-time frame number.

[0066] In the embodiment of the present application, the target video is a video on demand or live broadcast video. The video processing tool FFprobe-show_frames can be used to collect frame information of each frame in the target video in real time, wherein the frame information includes: the data packet size (pkt_size) of the current frame, and the real-time bit rate and / or real-time frame number of the target video are calculated according to the frame information in the following manner.

[0067] In one embodiment of the present application, according to the video processing tool FFmpeg, the corresponding preset target frame rate (recorded as F) and the preset target bit rate (recorded as M) of live and on-demand video can be collected. In the embodiment of the present application, based on the pkt_size of each frame, in all frames in the target video, according to the target frame rate F, F is the number of frames displayed per second, and the pkt_size of each F frame is added to obtain the real data size of all frames within one second (unit duration), so as to obtain the real-time bit rate of the target video (recorded as M1), and store the real-time bit rate in a newly created bit rate array K in order, wherein each position in the bit rate array K corresponds to the bit rate of a time point, such as: the second number of the bit rate array K is the real-time bit rate corresponding to the second second of the target video, the tenth number of the bit rate array K is the real-time bit rate corresponding to the tenth second of the target video, and so on.

[0068] In a specific embodiment, if the target frame rate F is 25, FFprobe can count the size of each frame, that is, the pkt_size value of each frame, such as 1K for the first frame, 1.5K for the second frame, ..., 1.1K for the 25th frame. By adding the pkt_size values ​​of the 25 frames transmitted within one second, the real-time size of the data transmitted in one second is obtained, that is, the real-time bit rate M1. Every 25 data are added and put into the bit rate array K, which is the real-time bit rate corresponding to each second in the target video.

[0069] In another embodiment of the present application, the inter-frame duration of the target video is processed using a python-cv2 library tool. When each frame is captured using the cv2 function package in the python-cv2 library, the timestamp of the frame is recorded, and the timestamps before and after each frame are subtracted to obtain the inter-frame duration T of the frame; according to the inter-frame duration T, the real-time frame number within one second is counted to obtain the real-time frame number F1, and a new frame number array L is created to store the real-time frame number F1 in order, wherein each position of the array corresponds to the real-time frame number at the current time point, such as the second number of the frame number array L is the real-time frame number corresponding to the second second of the video, the tenth number of the frame number array L is the real-time frame number corresponding to the tenth second of the video, and so on.

[0070] In a specific embodiment, there may be a scenario where frames are lost during video transmission. For example, the target frame rate F is 25, that is, 25 frames are expected to be transmitted per second. However, during transmission, due to screen distortion and freezes, the encoder discards some frames, and may only output 23 frames per second, discarding 2 frames. Therefore, by calculating the timestamp of each frame in the transmission, a counter is set according to the inter-frame duration T of the frame obtained by subtracting the adjacent timestamps before and after each frame, and the real-time frame number F1 received in the second is calculated. Each second corresponds to a frame number, which is stored in the frame number array L.

[0071] In an embodiment of the present application, the real-time bit rate and / or the real-time frame number are obtained so that the real-time bit rate and / or the real-time frame number can be specifically analyzed later to quickly capture video frames that may produce distortion in the target video, thereby eliminating the need to traverse and calculate whether all video frames have a flower screen, thereby reducing the flower screen detection time.

[0072] Step 12: Compare the real-time transmission parameters with preset target transmission parameters, and filter out distorted video frames according to the comparison results;

[0073] In the embodiment of the present application, optionally, comparing the real-time transmission parameter with a preset target transmission parameter, and filtering the distorted video frame according to the comparison result, includes:

[0074] Comparing the real-time transmission parameter with a preset target transmission parameter, and when the real-time transmission parameter does not match at least one of the preset target transmission parameters, recording a video frame corresponding to the current real-time transmission parameter as a distorted video frame;

[0075] Counting all the distorted video frames to obtain a distorted point array;

[0076] The data frames of a preset time period before and after each distorted point video frame in the distorted point array are used as distorted video frames.

[0077] In the embodiment of the present application, according to the comparison between each M1 value in the above-mentioned bit rate array K and the target bit rate value M value preset for the video, if |M / M1|>the first threshold value or |M / M1|<the second threshold value, the video end may insert frames or drop frames, and the introduction of the distorted screen is likely to be caused by these two factors, wherein the first threshold value and the second threshold value can determine the corresponding values ​​according to the actual encoding scheme, such as the first threshold value is 1.5, and the second threshold value is 0.5. The two thresholds can be flexibly changed for different encoding methods; a new distortion point array J is created, and the value recorded in the distortion point array J is the number of seconds corresponding to the distortion point introduced by the comparison between the M1 value in the bit rate array K and the target bit rate value M value preset for the video exceeding the threshold, so as to find the time point corresponding to the distortion point, and there may be a probability of a distorted screen at this time point, such as at the 15th second, the ratio of the M1 value in the bit rate array K to the target bit rate value M value specified for the video is lower than 0.5, then the value 15 is recorded in the distortion point array J, and so on.

[0078] According to the above frame number array L, the difference between each value and the target frame rate F specified by the video is counted. When the difference is not 0, it indicates an abnormal situation. The position of the value in the frame number array L is recorded. For example, at the 20th position, the difference between the real-time frame number F1 stored in the array and the specified target frame rate F is 1, which means that at the 20th second, the actual number of frames received by the player is one less or one more. It may be that the player is skipping or inserting frames or there are objective factors such as network delay and packet loss. Therefore, the current position 20 is recorded at last, and the value 20 is also stored in the distortion point array J. If there is a duplicate value in the distortion point array J, it does not need to be recorded. And so on to complete the position statistics of all distortion points.

[0079] In the embodiment of the present invention, data frames of a time period of 1 second (3 seconds in total) before and after each distortion point value in the distortion point array J are further obtained and stored in the picture array P to ensure that the sudden change of the distortion point may cause screen distortion and ensure the accuracy of the data.

[0080] In an embodiment of the present application, by performing fluctuation analysis on the real-time bit rate and / or the real-time frame number, video frames that may cause video distortion are quickly acquired, thereby eliminating the need to traverse and calculate whether all video frames have a flowery screen, thereby reducing the flowery screen detection time.

[0081] Step 13: Select a target image block from each preset number of target areas of each filtered distorted video frame; when the similarity between one of the target image blocks and the pre-stored flower screen image exceeds a preset similarity threshold, the distorted video frame corresponding to the target image block is judged to be a flower screen.

[0082] In an embodiment of the present application, on each frame of the picture in the traversal array P, the upper, middle and lower areas are cropped as preset target areas. The size of the cropped preset target area is adjusted according to the resolution of the picture. The amount of comparison data is reduced by segmenting the picture, and the detection time is reduced. The top, middle, bottom, half-flowered screen, green screen, snow screen, etc. can all be covered and detected.

[0083] In the embodiment of the present application, optionally, the target image block is matched with the flower screen picture in the pre-constructed flower screen model for structural similarity; wherein, the SSIM algorithm is used for structural similarity matching, because natural images have extremely high structurality, which is manifested in the strong correlation between the pixels of the image, and these correlations carry important information about the structure of the object in the visual scene. The SSIM algorithm detects whether the structural information has changed to perceive approximate information such as image distortion, and measures the similarity of the two images.

[0084] In an embodiment of the present application, when the similarity between at least one of the target image blocks and the distorted screen image exceeds a preset similarity threshold, the distorted video frame corresponding to the target image block is judged to be a distorted screen, thereby reducing the amount of calculation for each video frame.

[0085] In an embodiment of the present application, the flower screen picture in the pre-constructed flower screen model includes at least one of the following: green screen, snow screen and black spot screen, etc., that is, it includes various flower screen related pictures caused by flower screen, frame loss, network packet loss and delay problems caused by the encoding process; by traversing the cropped video in P1 in the array and matching the constructed flower screen picture for structural similarity, a single cropped picture and the constructed flower screen picture are matched for SSIM structural similarity, and the higher the similarity value, the higher the similarity with the flower screen; the structural similarity matching time of the cropped picture processed by python-cv2 is optimized, which is much better than the SSIM calculation of all images in the entire video frame; wherein, 1 / 10 of the upper area, 1 / 10 of the middle area and 1 / 10 of the bottom area of ​​the picture are cropped, and SSIM structural similarity calculation is performed; the pictures in P1 are traversed in a loop for SSIM, and when it is higher than the similarity threshold, it is judged as a flower screen, and the current screenshot is saved for subsequent human objective evaluation and saved in the flower screen model.

[0086] In the embodiment of the present application, by analyzing the fluctuations of real-time transmission parameters, video frames that may cause video distortion are quickly collected. There is no need to traverse all video frames to calculate whether they are flower screens. This improves calculation time, saves calculation costs, and accurately performs flower screen detection to prevent the expansion of faults. This solves the problem that the existing video flower screen detection requires a lot of time and may produce video frames with video distortion.

[0087] In the embodiment of the present application, optionally, selecting a target image block from each of at least one preset target area of ​​the filtered distorted video frame includes:

[0088] Dividing the distorted video frame into a preset number of equal parts with unchanged width and unchanged length, to obtain at least one image block arranged in sequence;

[0089] Each of the distorted video frames is divided into three regions, namely, upper, middle and lower regions, according to the sequence, and an image block is selected from each of the three regions, namely, upper, middle and lower regions, as a target image block.

[0090] In the embodiment of the present application, on each frame of the picture in the traversal array P, the upper, middle and lower regions are cropped as the preset target regions, and the size of the cropped preset target regions is adjusted according to the resolution of the picture, wherein the rule height of the cropping can be set to 1 / 10 of the picture height according to the actual situation, and the length is not cropped; for example: if the resolution of the picture is 1920*1080, the sizes of the upper, middle and lower regions are all set to 1920*108, that is, they are cropped to 1 / 10 of the original picture to reduce the amount of calculation. The upper region of the position can be set to be intercepted from the top according to the actual situation, the middle region starts from the 540 position, and the lower region is the bottom region. The image segmentation interception reduces the amount of comparison data, reduces the detection time, and the top, middle, bottom, half-flowered screen, green screen, snow screen, etc. of the flower screen position can be covered and detected, and the cropped pictures are placed in the corresponding new picture array P1 in sequence, and the video frames that may cause video distortion are quickly collected, without traversing all video frames to calculate whether it is a flower screen, improving the calculation time and saving the calculation cost.

[0091] In the embodiment of the present application, optionally, before determining the similarity between the target image block and the flower screen image in the pre-constructed flower screen model, the method further includes:

[0092] The target image block is expanded using an expansion kernel factor.

[0093] In the embodiment of the present application, it is possible but not limited to define a dilation kernel factor for matching; and use the dilation kernel factor to perform dilation processing on the target image block. In one embodiment of the present invention, in the picture array P, when traversing the first picture, the optimal expansion kernel factor in multiple experiments is used to expand the target image block. For example, the matrix of the expansion kernel factor can be set to a 5x5 inverted triangle expansion matrix, that is, np.array([[1,0,0,0,0],[1,1,0,0,0],[1,1,1,0,0],[1,1,1,1,0],[1,1,1,1,1]]; the inverted triangle expansion matrix is ​​used to match the upper, middle and lower regions of the crop, and the maximum value of the pixels covered by the inverted triangle expansion matrix in the upper, middle and lower regions of the crop replaces the original image pixel value covered by the inverted triangle expansion matrix, thereby achieving expansion. After the expansion, the efficiency of matching the target image block with the flower screen picture in the pre-constructed flower screen model is improved, and the flower screen area of ​​the cropped picture can be copied to the maximum extent, the calculation time is improved, the calculation cost is saved, and the flower screen detection is accurately performed to prevent the expansion of the fault. Please refer to Figure 2 In the embodiment of the present application, specifically, the screen flower detection method includes:

[0094] 1. Collect the real-time video bit rate and real-time video frame number;

[0095] For example: FFprobe can collect the frame information of each frame in the target video in real time, wherein the frame information includes: the data packet size (pkt_size) of the current frame, such as 1K for the first frame, 1.5K for the second frame, ..., 1.1K for the 25th frame, by adding these 25 numbers, the size of the real-time one-second transmission data is obtained, that is, the real-time bit rate M1, and every 25 data are added and put into the bit rate array K, which is the real-time bit rate corresponding to each second in the target video; and when each frame is captured, the timestamp of the frame is recorded, and the adjacent timestamps before and after each frame are subtracted to obtain the inter-frame duration T of the frame; according to the inter-frame duration T, the real-time frame number within one second is counted to obtain the real-time frame number F1, and a new frame number array L is created to store the real-time F1 in order, wherein each position of the array corresponds to the frame number at the current time point, such as the second number of the frame number array L is the frame number corresponding to the second second of the video, and the tenth number of the frame number array L is the frame number corresponding to the tenth second of the video;

[0096] 2. Match and judge the time point of obtaining the suspected abnormal point;

[0097] Create a new distortion array J. The value recorded in the distortion array J is the number of seconds corresponding to the distortion introduced when the value of M1 in the bitrate array K exceeds the target bitrate value M preset by the video and exceeds the threshold, so as to find the time point corresponding to the distortion. At this time point, there may be a probability of screen distortion. For example, at the 15th second, if the ratio of the value of M1 in the bitrate array K to the target bitrate value M specified by the video is less than 0.5, then the value 15 is recorded in the distortion array J, and so on.

[0098] According to the above frame number array L, the difference between each value and the target frame rate F specified by the video is counted. When the difference is not 0, it indicates an abnormal situation. The position of the value in the frame number array L is recorded. For example, at the 20th position, the difference between the real-time frame number F1 stored in the array and the specified target frame rate F is 1, which means that at the 20th second, the actual number of frames received by the player is one less or one more. It may be that the player is skipping or inserting frames or there are objective factors such as network delay and packet loss. Therefore, the current position 20 is recorded at last, and the value 20 is also stored in the distortion point array J. If there is a duplicate value in the distortion point array J, it does not need to be recorded. And so on to complete the position statistics of all distortion points.

[0099] 3. Collect all frames within 3 seconds before and after the current distortion point, that is, obtain the data frames of 1 second (3 seconds in total) before and after each distortion point value in the distortion point array J, and store them in the picture array P to ensure the screen distortion caused by the sudden change of the distortion point and ensure the accuracy of the data.

[0100] 4. Set the image expansion kernel;

[0101] Performing dilation processing on the target image block using a dilation kernel factor;

[0102] And crop each frame data of all current frames;

[0103] On each frame of the picture in the traversal array P, the upper, middle and lower regions are cropped as the preset target regions. The cropping size of the preset target regions is adjusted according to the resolution of the picture. The cropping rule height can be set to 1 / 10 of the picture height according to the actual situation, and the length is not cropped;

[0104] 5. Match with the image model;

[0105] Performing structural similarity matching between the target image block and a flower screen picture in a pre-constructed flower screen model;

[0106] If the match is successful, it is determined that the distorted video frame corresponding to the target image block is a distorted screen;

[0107] If the matching is unsuccessful, repeat the above steps to continue matching.

[0108] In the embodiment of the present application, by analyzing the transmission parameters, video frames that may cause distortion are quickly collected without traversing all video frames to calculate whether they are flower screens. By comparing the image blocks in the distorted video frames with the flower screen images, the entire video frames are not compared, which improves the calculation time, saves the calculation cost, and solves the problem that the existing video flower screen detection requires a lot of time.

[0109] Please refer to Figure 3 , the embodiment of the present application provides a screen distortion detection device, comprising:

[0110] The acquisition module 31 is used to acquire the real-time transmission parameters of the target video, wherein the real-time transmission parameters include at least one of the following: real-time bit rate and real-time frame number;

[0111] A processing module 32, configured to compare the real-time transmission parameters with preset target transmission parameters, and filter out distorted video frames according to the comparison result;

[0112] The judgment module 33 is used to select a target image block from each preset number of target areas of each distorted video frame that has been screened out; when the similarity between one of the target image blocks and a pre-stored distorted screen image exceeds a preset similarity threshold, the distorted video frame corresponding to the target image block is judged to be a distorted screen.

[0113] In the embodiment of the present application, optionally, the acquisition module 31 includes:

[0114] A first acquisition submodule is used to acquire preset target frame rate information of each frame of the target video;

[0115] The first processing submodule is used to add the data sizes of all frames within one second according to the preset target frame rate information to obtain the real-time bit rate.

[0116] In the embodiment of the present application, optionally, the acquisition module 31 includes: a second acquisition submodule, configured to acquire preset target frame rate information of each frame of the target video and a timestamp of each frame;

[0117] A second processing submodule is used to obtain the inter-frame duration according to the timestamps of the two adjacent frames before and after the target video;

[0118] The third processing submodule is used to calculate the number of frames received per second using a counter according to the preset target frame rate information and the inter-frame time length to obtain the real-time frame number.

[0119] In the embodiment of the present application, optionally, the processing module 32 includes:

[0120] The distortion point processing submodule is used to compare the real-time transmission parameters with the preset target transmission parameters. When the real-time transmission parameters do not completely correspond to the preset target transmission parameters, the video frame corresponding to the current real-time transmission parameters is recorded as a distortion point; all the distortion points are counted to obtain a distortion point array; and the data frames of the preset time period before and after each distortion point in the distortion point array are used as distorted video frames.

[0121] In the embodiment of the present application, optionally, the judging module 33 includes:

[0122] a fourth processing submodule, configured to divide the distorted video frame into a preset number of equal parts with unchanged width and unchanged length, to obtain at least one image block arranged in sequence;

[0123] The fifth processing submodule is used to divide each of the distorted video frames into three regions of upper, middle and lower regions according to the sequence, and select an image block from each of the three regions of upper, middle and lower regions of the distorted video frame as a target image block.

[0124] In the embodiment of the present application, optionally, it also includes:

[0125] The expansion module is used to perform expansion processing on the target image block using an expansion kernel factor.

[0126] The flower screen detection device provided in the embodiment of the present application can achieve Figure 1 The various processes implemented by the method embodiment and achieving the same technical effect are not described here to avoid repetition.

[0127] The present application embodiment provides an electronic device 40, see Figure 4 As shown, Figure 4 This is a principle block diagram of an electronic device 40 of an embodiment of the present application, including a processor 41, a memory 42, and a program or instruction stored in the memory 42 and executable on the processor 41. When the program or instruction is executed by the processor, the steps in any one of the flower screen detection methods of the present application are implemented.

[0128] An embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of an embodiment of a flower screen detection method such as any one of the above-mentioned ones is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0129] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0130] Computer readable media include permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0131] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0132] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0133] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0135] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting a distorted screen, characterized in that: include: Obtain the real-time transmission parameters of the target video; Comparing the real-time transmission parameters with preset target transmission parameters of the target video, and filtering out distorted video frames from the target video according to the comparison result; A target image block is selected from each preset number of target areas of each filtered distorted video frame. When the similarity between one of the target image blocks and a pre-stored flower screen image exceeds a preset similarity threshold, the distorted video frame corresponding to the target image block is judged to be a flower screen.

2. The method for detecting a distorted screen according to claim 1, characterized in that: The real-time transmission parameters include the real-time bit rate, and the real-time transmission parameters of the target video are obtained, including: Obtaining preset target frame rate information of each frame of the target video; The data sizes of all frames within one second are added together according to the preset target frame rate information to obtain the real-time bit rate.

3. The method for detecting a distorted screen according to claim 1 or 2, characterized in that: The real-time transmission parameter includes a real-time frame number, and the real-time transmission parameter of the target video is obtained, including: Obtaining preset target frame rate information and a timestamp of each frame of the target video; Obtaining the inter-frame duration according to the timestamps of the two adjacent frames before and after the target video; According to the preset target frame rate information and the inter-frame duration, a counter is used to calculate the number of frames received per second to obtain the real-time frame number.

4. The method for detecting a distorted screen according to claim 1, characterized in that: The comparing the real-time transmission parameters with preset target transmission parameters and filtering the distorted video frames according to the comparison results includes: Comparing the real-time transmission parameter with a preset target transmission parameter, and when the real-time transmission parameter does not match at least one of the preset target transmission parameters, recording a video frame corresponding to the current real-time transmission parameter as a distorted video frame; Counting all the distorted video frames to obtain a distorted point array; The data frames of a preset time period before and after each distorted point video frame in the distorted point array are used as distorted video frames.

5. The method for detecting a distorted screen according to claim 1, characterized in that: The step of selecting a target image block from each of the preset number of target areas of each distorted video frame selected includes: Dividing each of the distorted video frames into a preset number of equal parts with a constant width and a constant length, to obtain at least one image block arranged in sequence; Each of the distorted video frames is divided into three regions, namely, upper, middle and lower regions, according to the sequence, and an image block is selected from each of the three regions, namely, upper, middle and lower regions, as a target image block.

6. The method for detecting a distorted screen according to claim 1, characterized in that: Before determining the similarity between the target image block and the flower screen image in the pre-built flower screen model, the method further includes: The target image block is expanded using an expansion kernel factor.

7. A screen flower detection device, characterized in that: include: An acquisition module is used to acquire real-time transmission parameters of a target video; A processing module, used for comparing the real-time transmission parameters with preset target transmission parameters of the target video, and filtering out distorted video frames from the target video according to the comparison result; A judgment module is used to select a target image block from a preset number of target areas of each filtered distorted video frame. When the similarity between one of the target image blocks and a pre-stored distorted screen image exceeds a preset similarity threshold, the distorted video frame corresponding to the target image block is judged to be a distorted screen.

8. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the flower screen detection method as described in any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps in the method for detecting a distorted screen as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that It comprises computer instructions, which, when executed by a processor, implement the steps in the flower screen detection method as described in any one of claims 1 to 6.

Citation Information

Cited By

  • Communication video quality real-time automatic detection method and system based on AI technology

    CN121056683A