A bad frame detection method for rendering sequential frames

Abstract

The application discloses a bad frame detection method for rendering sequence frames. By utilizing an SSIM algorithm and an image prediction technology, a set of sequence frame bad frame detection system is built on a cloud rendering platform, so that the bad frames in output can be detected in time, the rendering system is informed in time, the system reminds the user in time or automatically re-renders the bad frames, and therefore, the efficiency of the rendering platform is effectively improved, and the time of the user is greatly saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video software rendering, in particular to a bad frame detection method for rendering sequence frames. BACKGROUND

[0002] In the production process of film and television special effects, there is a very key step, which is rendering. Rendering is the process of converting 3D models into 2D images. The output result of animation rendering is a group of sequence images, and the images have strong correlation. That is, the output result of rendering is the frame in the video, and the similarity between frames is very high. Only a small part of the pixels are different, and the difference is caused by motion. In the conventional rendering, due to some reasons, there may be bad frames in the output sequence frames. Cloud rendering is generally unattended. When a bad frame is found, the customer needs to use the picture, which will delay the user's work progress and affect the user experience.

[0003] In the prior art, the user needs to download the rendering task and check whether there is a bad frame by visual inspection with the naked eye, or whether there is a bad frame in the next production process. This way is very time-consuming. The user needs to re-render on the platform or re-render locally to obtain normal frame pictures. The rendering process is very time-consuming.

[0004] There is also a way in the rendering process. The user checks whether there is a bad frame by outputting a thumbnail from the rendering platform. However, this way requires the user to constantly monitor the rendering job and check each frame of picture, which is too low in efficiency.

[0005] Therefore, the prior art has defects and needs to be improved. SUMMARY

[0006] The purpose of the present application is to overcome the shortcomings of the prior art and provide a bad frame detection method for rendering sequence frames.

[0007] The technical solution of the present application is as follows: a bad frame detection method for rendering sequence frames is provided, which comprises the following steps:

[0008] Step 1: Determine whether the first frame is a bad frame. If not, take this frame picture as a prediction base point. If yes, detect whether the next frame is a bad frame, and take the frame picture of the first non-bad frame as a prediction base point. Record the prediction base point as the base frame I;

[0009] Step 2: According to the base frame I, obtain the subsequent frame of this frame through the steps of motion estimation, motion compensation, error coding and decoding, and record it as the prediction frame P;

[0010] Step 3: After obtaining the prediction frame P, use the SSIM algorithm to obtain the similarity value of the prediction frame P and the next frame N of the base frame I;

[0011] Step 4: judging the similarity value in step 3, if the similarity value is greater than a set value, it is determined that the N frame is a bad frame, otherwise it is a normal frame;

[0012] Step 5: if the N frame in step 4 is a bad frame, marking the N frame as a bad frame, using the predicted frame P to predict the next frame N+1 of the N frame as an I frame, obtaining a predicted image P+1;

[0013] Step 6: if the N frame in step 4 is a normal frame, using the N frame as an I frame to predict the predicted image P+1 of the next frame N+1 of the N frame;

[0014] Step 7: after completing step 5 or step 6, calculating the similarity value between the predicted P+1 frame and the N+1 frame using the SSIM algorithm;

[0015] Step 8: judging the similarity value result in step 7;

[0016] Step 9: repeating steps 4 to 8 until the inspection of all frame pictures is completed;

[0017] Step 10: after completing the inspection of all frame pictures, automatically retrieving the bad frame by the rendering platform, or notifying the user to manually retrieve.

[0018] Further, the specific steps of step 1 are:

[0019] Step 1.1: waiting for the task rendering to exceed 5 frames;

[0020] Step 1.2: after the rendering exceeds 5 frames, using the SSIM algorithm to obtain the similarity data A of the 1st frame and the 2nd frame, the similarity data B of the 2nd frame and the 3rd frame, the similarity data C of the 3rd frame and the 4th frame, and the similarity data D of the 4th frame and the 5th frame;

[0021] Step 1.3: comparing the differences between data A, B, C, and D;

[0022] Step 1.4: if the differences between data A and data B, C, and D are small, it indicates that the 1st frame picture is normal, and the bad frame detection starts from the 1st frame, and the 1st frame is recorded as a base frame I;

[0023] Step 1.5: if the difference between data A and data B is large, exceeding a set value, and the difference between data B and data CD is small, it indicates that the 1st frame is a bad frame, and the bad frame detection starts from the 2nd frame, and the 2nd frame is recorded as a base frame I;

[0024] Step 1.6: after completing step 1.4 or step 1.5, obtaining the detected base frame I, and starting the bad frame detection of the sequence frame.

[0025] By using the SSIM algorithm and image prediction technology, the cloud rendering platform is built with a sequence frame bad frame detection system, so that the bad frame in the output can be detected in time, and the rendering system is informed in time, so that the system can remind the user or automatically re-render the bad frame, thereby effectively improving the efficiency of the rendering platform and greatly saving the time of the user. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A flowchart for first frame bad frame detection.

[0027] Figure 2 A flowchart for sequence frame bad frame detection. DETAILED DESCRIPTION

[0028] The application will be described in detail below in combination with the drawings and specific embodiments.

[0029] The application provides a bad frame detection method for rendering sequence frames, comprising the following steps:

[0030] Step 1: refer to Figure 1 determine whether the first frame is a bad frame, if not, the frame picture is taken as a prediction base point; if yes, detect whether the next frame is a bad frame, and take the frame picture of the first non-bad frame as a prediction base point; the prediction base point is recorded as a base frame I. The specific steps are as follows:

[0031] Step 1.1: wait for more than 5 frames of task rendering.

[0032] Step 1.2: after more than 5 frames of rendering, use the SSIM algorithm to obtain the similarity data A of the 1st frame and the 2nd frame, the similarity data B of the 2nd frame and the 3rd frame, the similarity data C of the 3rd frame and the 4th frame, and the similarity data D of the 4th frame and the 5th frame.

[0033] Step 1.3: compare the differences between data A, B, C and D.

[0034] Step 1.4: if the differences between data A and data B, C and D are small, it indicates that the 1st frame picture is normal, and the bad frame detection starts from the 1st frame, and the 1st frame is recorded as the base frame I.

[0035] Step 1.5: if the difference between data A and data B is large and exceeds the set value, and the difference between data B and data CD is small, it indicates that the 1st frame is a bad frame, and the bad frame detection starts from the 2nd frame, and the 2nd frame is recorded as the base frame I.

[0036] Step 1.6: after completing step 1.4 or step 1.5, the detected base frame I is obtained, and the bad frame detection of the sequence frame is started.

[0037] Step 2: refer to Figure 2, according to the base frame I, the subsequent frame of the frame is obtained through the steps of motion estimation, motion compensation, error coding and decoding, and is recorded as a predicted frame P.

[0038] Step 3: After obtaining the predicted frame P, the similarity value of the predicted frame P and the next frame N of the base frame I is obtained using the SSIM algorithm.

[0039] Step 4: Determine the similarity value in step 3, if the similarity value is greater than a set value, determine that the N frame is a bad frame, otherwise it is a normal frame.

[0040] Step 5: If the N frame in step 4 is a bad frame, mark the N frame as a bad frame, and use the predicted frame P as the I frame to predict the next frame N+1 of the N frame, to obtain the predicted image P+1.

[0041] Step 6: If the N frame in step 4 is a normal frame, use the N frame as the I frame to predict the next frame N+1 of the N frame.

[0042] Step 7: After completing step 5 or step 6, calculate the similarity value of the predicted P+1 frame and N+1 frame using the SSIM algorithm.

[0043] Step 8: Determine the similarity value result in step 7.

[0044] Step 9: Repeat steps 4 to 8 until the inspection of all frame pictures is completed.

[0045] Step 10: After completing the inspection of all frame pictures, automatically retrieve the bad frame by the rendering platform, or notify the user to manually retrieve.

[0046] In the field of image, SSIM algorithm is commonly used, SSIM algorithm is an algorithm for evaluating image quality, which can calculate the structural similarity between two images, and further evaluate the similarity between two images. SSIM algorithm has certain advantages in evaluating image quality, not only considering the brightness and contrast of image, but also considering the structural similarity of image, therefore, it has been widely used in image coding, image processing and other fields.

[0047] In the field of video processing, image prediction technology will be used, image prediction technology is a technology that predicts the content of the next frame of image through existing image information, which is commonly used in video coding, video transmission and other fields. Common prediction techniques include motion estimation prediction, intra-frame prediction, residual prediction, etc.

[0048] In the technical solution provided by the application, the SSIM algorithm and image prediction technology are used for bad frame detection, which can well solve the low efficiency problem of relying on naked eyes to check bad frames. The application mainly has two processes: 1. determining whether the first frame is a bad frame, if not, the frame is used as a prediction base point, otherwise, the next frame is started; 2. bad frame detection of sequence frames.

[0049] Specifically, when the first frame is detected, more than 5 frame pictures are waited for task rendering, then the SSIM algorithm is used to obtain the similarity data between the adjacent two frames from the 1st frame to the 5th frame, which are respectively recorded as data A, B, C and D. The differences between the data A, B, C and D are compared, and according to the size of the differences, it is judged whether the 1st frame is a bad frame, if not, the 1st frame is used as a base frame I; if yes, the 2nd frame is used as the base frame I.

[0050] When the bad frame detection of sequence frames is performed, the subsequent frame P of the base frame I is obtained through the steps of motion estimation, motion compensation, error encoding and decoding, the similarity value of the P frame and the next frame N of the I frame is obtained by using the SSIM algorithm, and then it is judged whether the difference of the similarity value is greater than a set value (generally 10%). If the difference of the similarity value is greater than 10%, the N frame is determined to be a bad frame, and the N frame is marked. Because the N frame is a bad frame, the N frame cannot be used as the I frame to predict the next frame N+1 frame of the P frame, therefore, the P frame is used as the I frame to predict the next frame N+1 frame of the N frame, and the predicted image P+1 is obtained. If the difference of the similarity value is not greater than 10%, the N frame is determined to be a normal frame, and the predicted image P+1 frame of the next frame N+1 frame of the N frame is predicted by using the N frame as the I frame. The similarity result of all frame pictures is repeatedly judged, the checking of all frames is completed, and the bad frame is marked, and the bad frame is automatically retrieved by the rendering platform or the user is notified to manually retrieve. Through the method provided by the application, the bad frame file appearing in the rendering can be quickly detected, the bad frame is quickly re-rendered through different strategies, so that the efficiency of the rendering platform is greatly improved, and the time of the user is saved.

[0051] The above is only a preferred embodiment of the application, and is not used to limit the application, any modification, equivalent replacement and improvement made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A bad frame detection method of rendering a sequence of frames, the method comprising: The method comprises the following steps: Step 1: determining whether the first frame is a bad frame, if not, taking the frame picture as a prediction base point; if yes, detecting whether the next frame is a bad frame, and taking the frame picture of the first non-bad frame as a prediction base point; recording the prediction base point as a base frame I; Step 2: obtaining the subsequent frame of the frame according to the base frame I through the steps of motion estimation, motion compensation, error coding and decoding, and recording the prediction frame as P; Step 3: after obtaining the prediction frame P, using the SSIM algorithm to obtain the similarity value of the prediction frame P and the next frame N of the base frame I; Step 4: judging the similarity value in step 3, if the similarity value is greater than a set value, determining that the N frame is a normal frame, otherwise, a bad frame; Step 5: if the N frame in step 4 is a bad frame, marking the N frame as a bad frame, and using the prediction frame P as an I frame to predict the next frame N+1 of the N frame to obtain a prediction image P+1; Step 6: if the N frame in step 4 is a normal frame, using the N frame as an I frame to predict the prediction image P+1 of the next frame N+1 of the N frame; Step 7: after completing step 5 or step 6, using the SSIM algorithm to calculate the similarity value of the prediction obtained P+1 frame and N+1 frame; Step 8: judging the similarity value result in step 7; Step 9: repeating steps 4 to 8 until the checking of all frame pictures is completed; Step 10: after completing the checking of all frame pictures, automatically retrieving the bad frame by the rendering platform, or notifying the user to manually retrieve.

2. The bad frame detection method of rendering a sequence frame according to claim 1, wherein, The specific steps of step 1 are as follows: Step 1.1: waiting for more than 5 frames of task rendering; Step 1.2: after more than 5 frames of rendering, using the SSIM algorithm to obtain the similarity data A of the 1st frame and the 2nd frame, the similarity data B of the 2nd frame and the 3rd frame, the similarity data C of the 3rd frame and the 4th frame, and the similarity data D of the 4th frame and the 5th frame; Step 1.3: comparing the differences among data A, B, C and D; Step 1.4: if the difference between data A and data B, C and D is small, it indicates that the 1st frame picture is normal, and the bad frame detection starts from the 1st frame, and the 1st frame is recorded as a base frame I; Step 1.5: if the difference between data A and data B is large and exceeds a set value, while the difference between data B and data C and D is small, it indicates that the 1st frame is a bad frame, and the bad frame detection starts from the 2nd frame, and the 2nd frame is recorded as a base frame I; Step 1.6: after completing step 1.4 or step 1.5, obtaining the detected base frame I, and starting the bad frame detection of the sequence frame.

Images