Scene change detection method and device, electronic equipment and storage medium

Abstract

The invention provides a scene change detection method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The scene change detection method comprises the following steps: acquiring coding prediction mode information of a to-be-detected video; for a to-be-detected video frame which is not a key frame in the to-be-detected video, determining whether the to-be-detected video frame is a scene mutation video frame based on the coding prediction mode information and a preset intra-frame prediction pixel proportion threshold; wherein if the intra-frame prediction pixel proportion included in the coding prediction mode information is larger than the intra-frame prediction pixel proportion threshold value, it is determined that the video frame to be detected is the scene mutation video frame with scene change. Aiming at to-be-detected video frames which are key frames in the to-be-detected video, acquiring histogram information of each video frame in the to-be-detected video; and based on the obtained histogram information of each video frame in the to-be-detected video, determining whether the to-be-detected video frame is a scene mutation video frame or not based on a preset mode.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and specifically provides a scene change detection method, device, electronic device and storage medium. Background Art

[0002] Scene change detection identifies the moment a scene in a video changes—that is, the point at which one scene ends and a new one begins. Scene changes can be categorized as sudden, fade-in, fade-out, and gradual. Scene change detection has a wide range of applications, encompassing video editing, content analysis, display parameter optimization, and more. Its accuracy directly impacts subsequent video processing.

[0003] Existing detection methods based on feature information such as feature points and image semantics, as well as detection methods based on machine learning, have high algorithm complexity and excessive hardware implementation overhead. Summary of the Invention

[0004] The present application provides a scene change detection method, device, electronic device and storage medium to solve the problems of high algorithm complexity and excessive hardware implementation overhead in the prior art.

[0005] In a first aspect, the present application provides a scene change detection method, comprising: obtaining coding prediction mode information of a video to be detected; for a video frame to be detected that is not a key frame in the video to be detected, determining whether the video frame to be detected is a scene mutation video frame based on the coding prediction mode information and a preset intra-frame prediction pixel ratio threshold; wherein, if the intra-frame prediction pixel ratio included in the coding prediction mode information is greater than the intra-frame prediction pixel ratio threshold, it is determined that the video frame to be detected is a scene mutation video frame in which a scene change has occurred.

[0006] In the embodiments of the present application, if a video frame is a scene change frame, the difference between this video frame and the preceding video frame is often significant. Therefore, the proportion of pixels predicted using the inter-frame prediction mode in a scene change video frame is very small, while the proportion of pixels predicted using the intra-frame prediction mode is relatively large. Therefore, by judging the pixel ratio in the intra-frame prediction mode, it is possible to determine whether a scene change has occurred in a video frame. This approach is also more concise than detection methods based on feature information such as feature points and image semantics, thereby reducing algorithm complexity and hardware overhead.

[0007] In combination with the technical solution provided in the first aspect above, in some possible implementations, when it is determined that the video frame to be detected is not a scene mutation video frame, the method further includes: calculating first histogram difference information between the histogram information of the video frame to be detected and the historical histogram information of the video frame to be detected; based on the first histogram difference information and a preset historical histogram difference threshold, determining whether the video frame to be detected is a scene gradual change video frame; wherein, if the first histogram difference information is greater than the historical histogram difference threshold, it is determined that the video frame to be detected is a scene gradual change video frame.

[0008] In the embodiment of the present application, the difference between the video frame to be detected and its corresponding historical histogram information can be determined by using the first histogram difference information between the video frame to be detected and the historical histogram information of the video frame to be detected. Since the historical histogram information can reflect the characteristics of the video frames before the video frame to be detected, it can be determined whether the scene has gradually changed.

[0009] In combination with the technical solution provided in the first aspect above, in some possible implementations, for any video frame in the video to be detected, the historical histogram information of the video frame is calculated by using the histogram information of the previous video frame of the video frame and the historical histogram information of the previous video frame of the video frame; wherein, the historical histogram information of the first video frame in the video to be detected is a preset initial value.

[0010] In an embodiment of the present application, since the historical histogram information is calculated by using the histogram information of the previous video frame and the historical histogram information of the previous video frame, that is, the characteristics of multiple video frames before the video frame for which the historical histogram information needs to be calculated are taken into account, the historical histogram information can reflect the scene characteristics of multiple video frames before the video frame for which the historical histogram information needs to be calculated.

[0011] In combination with the technical solution provided in the first aspect above, in some possible implementations, for any video frame in the video to be detected, the historical histogram information of the video frame is calculated by the following steps: obtaining the histogram information and historical histogram information corresponding to the previous video frame of the video frame; and performing weighted summation processing on the histogram information corresponding to the previous video frame and the historical histogram information of the previous video frame using a preset weight value to obtain the historical histogram information of the video frame.

[0012] In combination with the technical solution provided in the first aspect above, in some possible implementations, when it is determined that the video frame to be detected is a scene mutation video frame, the method further includes: resetting the historical histogram information of the video frame to be detected to the initial value.

[0013] In the embodiment of the present application, since a scene mutation video frame indicates a scene mutation, the scene of the video frame after the scene mutation frame should be different from that of the video frame before the scene mutation frame. Therefore, when determining whether the video frame after the scene mutation frame is a scene gradual change frame, the scene in the video frame before the scene mutation frame no longer has reference value. Therefore, the historical histogram information of the video frame to be detected can be reset to the initial value to reduce the interference of the video frame before the scene mutation frame on subsequent judgments.

[0014] In combination with the technical solution provided in the first aspect above, in some possible implementations, the method further includes: for the video frame to be detected that is a key frame in the video to be detected, obtaining histogram information of each frame of the video to be detected; based on the obtained histogram information of each frame of the video to be detected, determining whether the video frame to be detected is a scene mutation video frame based on a preset method.

[0015] In the embodiment of the present application, the key frame is completely generated by the intra-frame prediction mode, and the pixel ratio of the intra-frame prediction mode is constant at 100%. Therefore, it is possible to determine whether the key frame is a scene mutation frame through histogram information.

[0016] In combination with the technical solution provided in the first aspect above, in some possible implementations, determining whether the video frame to be detected is a scene mutation video frame based on a preset method includes: calculating second histogram difference information between the histogram information of the video frame to be detected and the histogram information of the previous video frame of the video frame to be detected; determining whether the video frame to be detected is a scene mutation video frame based on the second histogram difference information and a preset histogram difference threshold; wherein, if the second histogram difference information is greater than or equal to the histogram difference threshold, it is determined that the video frame to be detected is a scene mutation video frame.

[0017] In combination with the technical solution provided in the first aspect above, in some possible implementations, the histogram information includes at least one of luminance histogram information and chrominance histogram information.

[0018] In a second aspect, the present application provides a scene change detection device, comprising: an acquisition module and a processing module, the acquisition module being used to acquire coding prediction mode information of a video to be detected; the processing module being used to determine, for a video frame to be detected that is not a key frame in the video to be detected, whether the video frame to be detected is a scene mutation video frame based on the coding prediction mode information and a preset intra-frame prediction pixel ratio threshold; wherein, if the intra-frame prediction pixel ratio included in the coding prediction mode information is greater than the intra-frame prediction pixel ratio threshold, it is determined that the video frame to be detected is a scene mutation video frame in which a scene change has occurred.

[0019] In combination with the technical solution provided in the second aspect above, in some possible implementations, the processing module is also used to obtain histogram information of each frame of the video to be detected for the video frame to be detected that is a key frame in the video to be detected; based on the obtained histogram information of each frame of the video to be detected, based on a preset method, determine whether the video frame to be detected is a scene mutation video frame.

[0020] In a third aspect, the present application provides an electronic device comprising: a memory and a processor, the memory and the processor being connected; the memory being used to store programs; and the processor being used to call the programs stored in the memory to execute the method described in the first aspect above and / or in combination with any possible implementation of the first aspect above.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer, it executes the method described in the first aspect and / or in combination with any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 A schematic diagram of a flow chart of a first scene change detection method provided in an embodiment of the present application; Figure 2 A schematic diagram of an image processing process provided in an embodiment of the present application; Figure 3 A schematic diagram of a flow chart of a second scene change detection method provided in an embodiment of the present application; Figure 4 A structural block diagram of a scene change detection device provided in an embodiment of the present application; Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0025] See also Figure 1, Figure 1 This is a scene change detection method shown in the embodiment of the present application. Figure 1 Describe the steps it involves.

[0026] S100: Obtain coding prediction mode information of a video to be detected.

[0027] Optionally, the coding prediction mode information of the video to be detected may be acquired in advance and stored in a local storage medium, and can be directly called when needed.

[0028] Alternatively, the scene change detection method can be placed after the video decoding unit in the video processing process, and the coding prediction mode information collected in the video decoding unit can be directly obtained, and then the present solution can be executed to determine whether the video frame is a scene change frame. The result of the determination can be provided to the subsequent video processing unit for use in the subsequent video processing process. Figure 2 As shown, the position of this solution in the video processing flow is the scene change detection unit.

[0029] The video decoding unit may be a video decoder supporting common video formats, including but not limited to H.264 / AVC (Advanced Video Coding), H.265 / HEVC (High Efficiency Video Coding), H.266 / VVC (Versatile Video Coding), VP9 (an open, royalty-free video compression standard developed by Google), AV1 (AOMedia Video 1, an open, royalty-free video coding standard developed by the Alliance for Open Media), MPEG-2 (an international standard developed by the Moving Picture Experts Group, primarily for the compression and transmission of audio and video data), MPEG-4, AVS (Digital Audio and Video Codec Standard), AVS2, and AVS3. Statistics of coding prediction mode information need to be performed within the video decoder of the corresponding format. The information that the video decoding unit needs to provide to the scene change detection unit includes: reconstructed frame video data and coding prediction mode information; the coding prediction mode information includes the video frame coding type and the pixel ratio of the video frame's intra-frame prediction mode.

[0030] The reconstructed frame video data is the data of the original video frame restored from the received compressed data and the reference frame during the decoding process. The specific method of obtaining the reconstructed frame video data is well known to those skilled in the art and will not be repeated here for the sake of brevity.

[0031] The video processing unit that receives the judgment result representing whether the video frame is a scene change frame after the video decoding unit can be a digital video display unit. The information of whether the video frame has a scene change can be used to adjust and dynamically optimize parameters such as brightness, contrast, color saturation, color temperature, sharpness, dynamic range, backlight, gamma and color gamut to enhance the viewing experience.

[0032] In one embodiment, the specific method of obtaining the coding prediction mode information of the video to be detected may be: first, receiving the original data of the video to be detected, and then decoding the original data to obtain the coding prediction mode information of the video to be detected.

[0033] The original data includes but is not limited to video coding formats such as H.264 / AVC, H.265 / HEVC, H.266 / VVC, VP9, ​​AV1, MPEG-2, MPEG-4, AVS, AVS2, AVS3, and video data in upper-layer container formats such as MP4, MKV, and AVI.

[0034] Optionally, the coding prediction mode information includes the video frame coding type and the pixel ratio of the video frame intra-frame prediction mode. The video frame coding type is used to determine whether the video frame is a key frame. If the video frame is an I frame (intra-coded frame), the video frame coding type is a key frame. If the video frame is a P frame (a P frame is predicted from a preceding B frame or I frame) or a B frame (bidirectionally interpolated frame), the video frame coding type is a non-key frame.

[0035] Optionally, the calculation method of the pixel ratio of the intra prediction mode of the video frame can be expressed as , where i=[0, IntraCuNum-1].

[0036] IntraCuNum represents the number of coding units (CUs) in intra prediction mode in the video frame being tested. IntraCuSize represents the size of the coding unit (CU) in intra prediction mode. For an 8x8 CU, IntraCuSize is 64. PicWidth represents the width of the video frame; PicHeight represents the height of the video frame; and IntraAreaProb represents the pixel ratio of the video frame in intra prediction mode.

[0037] S200: For a video frame to be detected that is not a key frame in the video to be detected, determining whether the video frame to be detected is a scene mutation video frame based on encoding prediction mode information and a preset intra-frame prediction pixel ratio threshold.

[0038] If the intra-frame prediction pixel ratio included in the coding prediction mode information is greater than the intra-frame prediction pixel ratio threshold, the video frame to be detected is determined to be a scene change video frame in which a scene change has occurred. If the intra-frame prediction pixel ratio included in the coding prediction mode information is less than or equal to the intra-frame prediction pixel ratio threshold, the video frame to be detected is determined to have not undergone a scene change.

[0039] If the video frame to be detected is not a keyframe, the pixel ratio in the intra-frame prediction mode can be used as a criterion for scene change detection. If the video frame is a scene change frame, it is often significantly different from the previous frame. Under this condition, the pixel ratio predicted by the inter-frame prediction mode is small, while the pixel ratio predicted by the intra-frame prediction mode is large. Therefore, the pixel ratio in the intra-frame prediction mode can be used to determine whether it is a scene change frame.

[0040] Optionally, the specific value of the intra-frame prediction pixel ratio threshold can be set according to actual conditions, and its specific value is not limited here.

[0041] In one embodiment, for a key frame in a video to be detected, histogram information of each video frame in the video to be detected is obtained. Based on the obtained histogram information of each video frame in the video to be detected, a determination is made based on a preset method as to whether the video frame to be detected is a scene change video frame.

[0042] The histogram information of each video frame in the video to be detected can be pre-acquired and stored in a local storage medium, and can be directly called when needed. Alternatively, the reconstructed frame video data can be sorted according to the display order, and the histogram information of each video frame in the video to be detected can be counted in order.

[0043] In one implementation, the histogram information includes at least one of luminance histogram information and chrominance histogram information.

[0044] In one embodiment, the method of obtaining the histogram information of each frame of the video to be detected can be: first obtain the reconstructed frame video data, then sort the reconstructed frame video data according to the display order, and count the histogram information of the video frames in order.

[0045] The reconstructed frame video data may be acquired in advance and stored in a local storage medium, and can be directly called when needed.

[0046] Alternatively, the scene change detection method may be implemented after the video decoding unit in the video processing flow. In this manner, before executing the scene change detection method, reconstructed frame video data may be directly obtained from the video decoding unit, and the present solution may be executed based on the reconstructed frame video data.

[0047] The order in which the video decoding unit outputs the reconstructed frame video data can be selected according to the decoding order or the display order. Since scene change detection determines whether the scene has changed according to the display order, here, if the reconstructed frame video data is not output in the display order, it needs to be re-sorted according to the display order.

[0048] In one embodiment, the specific method of obtaining the reconstructed frame video data and the coding prediction mode information of the video to be detected may be: first, receiving the original data of the video to be detected, and then decoding the original data to obtain the reconstructed frame video data of the video to be detected.

[0049] Optionally, the color space of the reconstructed frame video data includes but is not limited to one or a combination of YUV (a color encoding method), YCbCr (a color space), and RGB (a color space).

[0050] Optionally, the histogram information includes brightness histogram information. The specific method of obtaining the histogram information of each frame of the video to be detected can be: first obtain the brightness values ​​of all pixels of each frame of the video to be detected; then based on the preset brightness histogram statistical dimension, count the brightness values ​​of all pixels of each frame of the video to obtain brightness histogram information that characterizes the brightness characteristics of the video frame; the histogram information includes brightness histogram information.

[0051] Optionally, the histogram information includes chromaticity histogram information. A specific method for obtaining the histogram information for each frame of the video to be detected may be: first, obtaining the color values ​​of all pixels in each frame of the video to be detected. Then, based on a preset color histogram statistical dimension, the color values ​​of all pixels in each frame of the video to be detected are counted to obtain chromaticity histogram information representing color characteristics of the video frame; the histogram information includes chromaticity histogram information.

[0052] Optionally, the histogram information includes brightness histogram information and chromaticity histogram information. The specific method of obtaining the histogram information of each frame of the video to be detected can be: first, obtain the brightness value and color value of all pixels of each frame of the video to be detected. Then, based on the preset brightness histogram statistical dimension, count the brightness values ​​of all pixels of each frame of the video to obtain brightness histogram information that characterizes the brightness characteristics of the video frame. And based on the preset color histogram statistical dimension, count the color values ​​of all pixels of each frame of the video to obtain chromaticity histogram information that characterizes the color characteristics of the video frame. The histogram information includes brightness histogram information and chromaticity histogram information.

[0053] Optionally, each type of histogram information may be divided into several levels, and the maximum number of levels is determined by the pixel bit depth.

[0054] For example, if only the brightness histogram information is counted and the bit depth is 8 bits, the maximum number of levels for the histogram division is 256, that is, each level contains one pixel value.

[0055] However, if a level in the histogram contains multiple pixel values, the number of histogram levels will be reduced. For example, if a level in the histogram contains 4 pixel values, the number of histogram levels for a pixel value of 8-bit depth is 64.

[0056] In one implementation manner, based on a preset method, a specific method for determining whether the video frame to be detected is a scene mutation video frame may also be: detecting whether the video frame to be detected is a scene mutation frame based on machine learning.

[0057] Optionally, a specific method for detecting whether a video frame to be detected is a scene change frame based on machine learning may include: using histogram information of the video frame to be detected and histogram information of n frames preceding the video frame to be detected as input data to a pre-trained scene change detection model, and obtaining an output result of the scene change detection model. The output result indicates whether the video frame to be detected is a scene change frame.

[0058] Wherein, n is a positive integer greater than or equal to 1.

[0059] The scene change detection model can be trained using a training dataset. The training dataset includes multiple training data. Each training data includes: histogram information of a target video frame, histogram information of n frames preceding the target video frame, and a label indicating whether the target video frame is a scene change frame.

[0060] The specific method and principle of training the model are well known to those skilled in the art and will not be described here for the sake of brevity.

[0061] In one embodiment, based on a preset method, determining whether a video frame to be detected is a scene change video frame may be performed by calculating a second histogram difference between histogram information of the video frame to be detected and histogram information of a video frame immediately preceding the video frame to be detected. Then, based on the second histogram difference information and a preset histogram difference threshold, determining whether the video frame to be detected is a scene change video frame.

[0062] If the second histogram difference information is greater than or equal to the histogram difference threshold, it is determined that the video frame to be detected is a scene mutation video frame.

[0063] The video frame preceding the video frame to be detected is the video frame that is displayed before the video frame to be detected in the display order of the video to be detected. For example, if the video frame to be detected is the kth video frame in the video to be detected, the video frame preceding the video frame to be detected is the k-1th video frame in the video to be detected.

[0064] Alternatively, the preceding video frame of the video frame to be detected may be a video frame that is detected before the video frame to be detected in the display order of the video to be detected. For example, if the video frames to be detected in the video to be detected are, in display order, the a-th video frame, the b-th video frame, the c-th video frame, and the d-th video frame, then the preceding video frame of the c-th video frame is the b-th video frame.

[0065] In one embodiment, the specific method of calculating the second histogram difference information between the histogram information of the video frame to be detected and the histogram information of the previous video frame of the video frame to be detected can be: performing difference calculation on the histogram information of the video frame to be detected and the histogram information of the previous video frame of the video frame to be detected, and normalizing the result of the difference calculation to obtain the second histogram difference information.

[0066] Optionally, a specific method for calculating the second histogram difference information between the histogram information of the video frame to be detected and the histogram information of the previous video frame to be detected can be: first, for each dimension of the histogram division, obtain the absolute value of the difference between the numerical value corresponding to the level in the histogram information of the video frame to be detected and the numerical value corresponding to the level in the histogram information of the previous video frame, and then calculate the sum of the absolute values ​​of each level difference in the histogram. Finally, the obtained sum is normalized, and the obtained value is the second histogram difference information. Wherein, the denominator of the normalization process is twice the total number of pixels counted in the histogram information.

[0067] Optionally, a formula for calculating second histogram difference information between the histogram information of the video frame to be detected and the histogram information of the video frame preceding the video frame to be detected may be: , where i=[0, levelNum-1].

[0068] Abs is the function for calculating the absolute value, Hist n Represents the histogram information of the video frame to be detected; Hist n-1 Indicates the histogram information of the video frame before the video frame to be detected; levelNum indicates the level number of the division in the current histogram information. PicWidth indicates the width of the video frame; PicHeight indicates the height of the video frame.

[0069] For example, if the histogram information includes three levels A, B, and C, and in the histogram information of the video frame to be detected, the value corresponding to level A is 100, the value corresponding to level B is 120, and the value of level C is 80. In the histogram information of the video frame before the video frame to be detected, the value corresponding to level A is 90, the value corresponding to level B is 100, and the value of level C is 110.

[0070] First, the absolute value of the difference corresponding to series A is calculated as |100-90|, which is 10. And the absolute value of the difference corresponding to series B is calculated as |120-100|, which is 20. The absolute value of the difference corresponding to series C is calculated as |80-110|, which is 30. Then the sum of the differences of the three series is calculated as 10+20+30=60. Finally, 60 is normalized, that is, 60÷600=0.1, that is, the second histogram difference information is 0.1. 600 is twice the total number of pixels counted in the histogram information. The examples here are only for ease of understanding and should not be used as limitations on this application.

[0071] Optionally, the calculation method for the second histogram difference information is not limited to the above-mentioned calculation method. Other methods for calculating histogram correlation, such as the Pearson coefficient method, the chi-square distribution method, and the Bhattacharyya distance method, may also be used. The results calculated using these calculation methods, after normalization, can also be used as the second histogram difference information.

[0072] In one embodiment, a specific method for determining whether the video frame to be detected is a scene change video frame based on the second histogram difference information and a preset histogram difference threshold may be: when the second histogram difference information is greater than or equal to the histogram difference threshold, it indicates that the video frame to be detected is a scene change video frame. In other words, a scene change has occurred in the video frame to be detected.

[0073] When the second histogram difference information is less than the histogram difference threshold, it indicates that the video frame to be detected is not a scene mutation video frame.

[0074] Since the key frame is completely generated by the intra-frame prediction mode, the pixel ratio of the intra-frame prediction mode is constant at 100%. The pixel ratio of the intra-frame prediction mode cannot be used as a judgment condition for scene change detection. Therefore, judgment is made only by histogram difference information.

[0075] The preset histogram difference threshold can be set according to actual needs, and its specific value is not limited here.

[0076] A lower histogram difference threshold results in more sensitive detection but is more prone to false positives. A higher histogram difference threshold results in less sensitive detection and is more prone to missed positives. The specific value of the histogram difference threshold can be set based on actual circumstances and is not limited here.

[0077] In one implementation, the histogram difference threshold may be set to any value between 0.15 and 0.18.

[0078] In one embodiment, if it is determined that the video frame to be detected is not a scene-change frame, first histogram difference information between the histogram information of the video frame to be detected and previously acquired historical histogram information may be calculated. Based on the first histogram difference information and a preset historical histogram difference threshold, it is determined whether the video frame to be detected is a scene-change frame.

[0079] If the first histogram difference information is greater than or equal to the historical histogram difference threshold, the video frame to be detected is determined to be a scene-gradual-change video frame. A scene-gradual-change frame indicates that a scene change occurs in the video frame to be detected.

[0080] Optionally, the historical histogram difference threshold may be smaller than the aforementioned histogram difference threshold for determining frames with sudden scene changes, so as to improve the sensitivity of detecting frames with gradual scene changes.

[0081] Historical histogram information can be obtained from a third party or calculated when needed.

[0082] In one implementation manner, the historical histogram information of the video frame to be detected is calculated by using the histogram information of a previous video frame of the video frame to be detected and the historical histogram information of the previous video frame.

[0083] Optionally, the historical histogram information is calculated by the following steps: first, obtaining histogram information and historical histogram information corresponding to a video frame preceding the video frame to be detected; wherein the historical histogram information corresponding to the starting video frame of the video to be detected is 0. Then, the histogram information corresponding to the previous video frame and the historical histogram information are weighted and summed using a preset weight value to obtain the historical histogram information of the video frame to be detected.

[0084] The weight value of the weighted summation process is preset, and the sum of the weight values ​​corresponding to the histogram information corresponding to the previous video frame and the historical histogram information is 1.

[0085] For ease of understanding, the formula for calculating historical histogram information can be expressed as: , where i=[0, levelNum-1].

[0086] Among them HistHistory n-1 Represents the historical histogram information of the previous video frame, Hist n-1 Indicates the histogram information of the previous video frame, w1 is the preset weight. HistHistory n Indicates the historical histogram information of the video frame to be detected.

[0087] Optionally, if it is determined that the video frame to be detected is not a scene-change frame, historical histogram information corresponding to the video frame to be detected may be first calculated. A first histogram difference between the histogram information of the video frame to be detected and the historical histogram information may then be calculated. Subsequently, based on the first histogram difference information and a preset historical histogram difference threshold, a determination may be made as to whether the video frame to be detected is a scene-change frame.

[0088] The method of calculating the historical histogram information corresponding to the video frame to be detected has been clearly described in the previous text and will not be repeated here for the sake of simplicity.

[0089] Optionally, after a scene change video frame appears, the historical histogram information may be reset, wherein the historical histogram information corresponding to the scene change video frame is 0, and the historical histogram information corresponding to the next video frame after the scene change video frame is the histogram information of the scene change video frame.

[0090] For example, if the n-1th video frame is a scene change frame, the historical histogram information corresponding to the n-1th video frame is reset to 0. The historical histogram information corresponding to the nth video frame is the histogram information corresponding to the n-1th video frame. That is, . Where i=[0, levelNum-1].

[0091] Among them, if the video frame is the starting frame of the video to be detected (that is, the video frame that is first displayed in the video to be detected), its corresponding historical histogram information is 0, and the historical histogram information corresponding to the next video frame of the starting frame is the histogram information of the starting frame.

[0092] In one embodiment, the first histogram difference information between the histogram information of the video frame to be detected and the previously acquired historical histogram information can be calculated by first obtaining the absolute value of the difference between the value corresponding to the level in the histogram information of the video frame to be detected and the value corresponding to the level in the historical histogram information for each dimension of the histogram division, and then calculating the sum of the absolute values ​​of the differences for each level in the histogram. Finally, the obtained sum is normalized, and the obtained value is the first histogram difference information. The denominator of the normalization process is twice the total number of pixels counted in the histogram information.

[0093] For ease of understanding, the formula for calculating the first histogram difference information between the histogram information of the video frame to be detected and the pre-acquired historical histogram information can be expressed as: , where i=[0, levelNum-1].

[0094] Abs is the function for calculating the absolute value, Hist n Represents the histogram information of the video frame to be detected; HistHistory n Indicates the historical histogram information corresponding to the video frame to be detected; levelNum indicates the level of division in the current histogram information. PicWidth indicates the width of the video frame; PicHeight indicates the height of the video frame.

[0095] For example, if the histogram information includes three levels A, B, and C, and the histogram information of the video frame to be detected has a value of 100 for level A, 120 for level B, and 80 for level C, then the historical histogram information of the video frame to be detected has a value of 90 for level A, 100 for level B, and 110 for level C.

[0096] First, the absolute value of the difference corresponding to series A is calculated as |100-90|, which is 10. And the absolute value of the difference corresponding to series B is calculated as |120-100|, which is 20. The absolute value of the difference corresponding to series C is calculated as |80-110|, which is 30. Then the sum of the differences of the three series is calculated as 10+20+30=60. Finally, 60 is normalized, that is, 60÷600=0.1, that is, the first histogram difference information is 0.1. 600 is twice the total number of pixels counted in the histogram information. The examples here are only for ease of understanding and should not be used as limitations on this application.

[0097] Optionally, the calculation method for the first histogram difference information is not limited to the above-mentioned calculation method. Other methods for calculating histogram correlation, such as the Pearson coefficient method, the chi-square distribution method, and the Bhattacharyya distance method, may also be used. The results calculated using these calculation methods, after normalization, can also be used as the first histogram difference information.

[0098] To facilitate understanding of the above-mentioned scene change detection method, an example is given below to illustrate.

[0099] like Figure 3As shown, the original data of the video to be detected is first received. The original data is then decoded to obtain reconstructed frame video data and coding prediction mode information of the video to be detected. The reconstructed frame video data is then sorted according to the display order, and histogram information of each video frame in the video to be detected is counted in order. The second histogram difference information between the histogram information of the video frame to be detected and the histogram information of the video frame before the video frame to be detected is calculated.

[0100] Determine whether the video frame to be detected is a key frame. If the video frame to be detected is a key frame, determine whether the video frame to be detected is a scene change video frame based on the second histogram difference information and a preset histogram difference threshold.

[0101] If the video frame to be detected is not a key frame, it is determined whether the video frame to be detected is a scene mutation video frame based on the encoding prediction mode information and a preset intra-frame prediction pixel ratio threshold.

[0102] If it is determined that the video frame to be detected is a scene mutation video frame, it is determined that a scene change has occurred, and the detection of the video frame to be detected is completed.

[0103] If it is determined that the video frame to be detected is not a scene change frame, historical histogram information is calculated based on the histogram information of the previous video frame of the video frame to be detected and the historical histogram information of the previous video frame. Then, first histogram difference information between the histogram information of the video frame to be detected and the previously acquired historical histogram information is calculated. Thereafter, based on the first histogram difference information and a preset historical histogram difference threshold, it is determined whether the video frame to be detected is a scene change video frame. If it is determined that the video frame to be detected is a scene change video frame, it is determined that a scene change has occurred, and the detection of the video frame to be detected is completed.

[0104] If it is determined that the video frame to be detected is not a scene-gradual-change video frame, it is determined that no scene change occurs, and the detection of the video frame to be detected is completed.

[0105] Figure 3 The specific methods and principles of each step shown have been clearly described in the previous text and will not be repeated here for the sake of brevity.

[0106] Based on the same technical concept, the present application also provides a scene change detection device, such as Figure 4 As shown, the scene change detection device 100 includes an acquisition module 110 and a processing module 120 .

[0107] The acquisition module 110 is configured to acquire coding prediction mode information of the video to be detected.

[0108] The processing module 120 is used to determine, for a video frame to be detected that is not a key frame in the video to be detected, whether the video frame to be detected is a scene mutation video frame based on the encoding prediction mode information and a preset intra-frame prediction pixel ratio threshold; wherein, if the intra-frame prediction pixel ratio included in the encoding prediction mode information is greater than the intra-frame prediction pixel ratio threshold, the video frame to be detected is determined to be a scene mutation video frame in which a scene change has occurred.

[0109] The processing module 120 is further configured to, when it is determined that the video frame to be detected is not a scene mutation video frame, calculate first histogram difference information between the histogram information of the video frame to be detected and the historical histogram information of the video frame to be detected; and determine whether the video frame to be detected is a scene gradual change video frame based on the first histogram difference information and a preset historical histogram difference threshold; wherein, if the first histogram difference information is greater than the historical histogram difference threshold, it is determined that the video frame to be detected is a scene gradual change video frame.

[0110] In one embodiment, for any video frame in the video to be detected, the historical histogram information of the video frame is calculated by using the histogram information of the previous video frame of the video frame and the historical histogram information of the previous video frame of the video frame; wherein the historical histogram information of the first video frame in the video to be detected is a preset initial value.

[0111] For any video frame in the video to be detected, the processing module 120 is further used to obtain the histogram information and historical histogram information corresponding to the previous video frame of the video frame; the histogram information corresponding to the previous video frame and the historical histogram information of the previous video frame are weighted and summed using a preset weight value to obtain the historical histogram information of the video frame.

[0112] In the case where it is determined that the video frame to be detected is a scene mutation video frame, the processing module 120 is further configured to reset the historical histogram information of the video frame to be detected to the initial value.

[0113] The processing module 120 is further used to obtain histogram information of each frame of the video to be detected for the video frame to be detected that is a key frame in the video to be detected; based on the obtained histogram information of each frame of the video to be detected, based on a preset method, determine whether the video frame to be detected is a scene mutation video frame.

[0114] The processing module 120 is specifically configured to calculate second histogram difference information between the histogram information of the video frame to be detected and the histogram information of the video frame preceding the video frame to be detected; based on the second histogram difference information and a preset histogram difference threshold, determine whether the video frame to be detected is a scene mutation video frame; wherein, if the second histogram difference information is greater than or equal to the histogram difference threshold, determine that the video frame to be detected is a scene mutation video frame.

[0115] In one embodiment, the histogram information includes at least one of luminance histogram information and chrominance histogram information.

[0116] The scene change detection device 100 provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned scene change detection method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding content in the aforementioned scene change detection method embodiment.

[0117] See also Figure 5 , which is an electronic device 200 provided in an embodiment of the present application. The electronic device 200 includes: a processor 210 and a memory 220.

[0118] The memory 220 and the processor 210 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 220 is used to store computer programs, such as Figure 4 The software functional module shown in the figure is the scene change detection device 100. The scene change detection device 100 includes at least one software functional module that can be stored in the memory 220 in the form of software or firmware or embedded in the operating system (OS) of the electronic device 200. The processor 210 is used to execute the executable module stored in the memory 220, such as the software functional module or computer program included in the scene change detection device 100. At this time, the processor 210 is used to obtain the coding prediction mode information of the video to be detected; for the video frame to be detected that is not a key frame in the video to be detected, based on the coding prediction mode information and a preset intra-frame prediction pixel ratio threshold, determine whether the video frame to be detected is a scene mutation video frame; wherein, if the intra-frame prediction pixel ratio included in the coding prediction mode information is greater than the intra-frame prediction pixel ratio threshold, it is determined that the video frame to be detected is a scene mutation video frame in which a scene change has occurred.

[0119] The memory 220 may be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0120] Processor 210 may be an integrated circuit chip with signal processing capabilities. Such processors may be general-purpose processors, including CPUs (Central Processing Units) and NPs (Network Processors). They may also be ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. They may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor, or processor 210 may be any conventional processor.

[0121] Alternatively, the processor 210 may be a data processing core of a GPU (Graphics Processing Unit), AI (Artificial Intelligence), NPU (Neural Network Processing Unit), ISP (Image Signal Processor), DPU (Display Processing Unit), VPU (Video Processing Unit), DSP (Digital Signal Processor), or the like, or a processor chip used in scenarios such as large-scale data computing. The above is merely an example and should not be construed as limiting the present application.

[0122] The electronic device 200 mentioned above includes but is not limited to a personal computer, a server, etc.

[0123] The present application also provides a computer-readable storage medium (hereinafter referred to as storage medium) having a computer program stored thereon. When the computer program is executed by a computer, such as the electronic device 200 described above, the computer program executes the scene change detection method described above. The computer-readable storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0124] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A scene change detection method, characterized in that: include: Obtaining the coding prediction mode information of the video to be detected; For a video frame to be detected that is not a key frame in the video to be detected, determine whether the video frame to be detected is a scene mutation video frame based on the encoding prediction mode information and a preset intra-frame prediction pixel ratio threshold; wherein, if the intra-frame prediction pixel ratio included in the encoding prediction mode information is greater than the intra-frame prediction pixel ratio threshold, determine that the video frame to be detected is a scene mutation video frame in which a scene change has occurred.

2. The method according to claim 1, characterized in that When it is determined that the video frame to be detected is not a scene mutation video frame, the method further includes: Calculating first histogram difference information between the histogram information of the video frame to be detected and the historical histogram information of the video frame to be detected; Based on the first histogram difference information and a preset historical histogram difference threshold, it is determined whether the video frame to be detected is a scene gradual change video frame; wherein, if the first histogram difference information is greater than the historical histogram difference threshold, it is determined that the video frame to be detected is a scene gradual change video frame.

3. The method according to claim 2, characterized in that For any video frame in the video to be detected, the historical histogram information of the video frame is calculated by using the histogram information of the previous video frame of the video frame and the historical histogram information of the previous video frame of the video frame; wherein the historical histogram information of the first video frame in the video to be detected is a preset initial value.

4. The method according to claim 3, characterized in that For any video frame in the video to be detected, the historical histogram information of the video frame is calculated by the following steps: Obtaining histogram information and historical histogram information corresponding to a previous video frame of the video frame; The histogram information corresponding to the previous video frame and the historical histogram information of the previous video frame are weighted and summed using a preset weight value to obtain the historical histogram information of the video frame.

5. The method according to claim 4, characterized in that When it is determined that the video frame to be detected is a scene mutation video frame, the method further includes: The historical histogram information of the video frame to be detected is reset to the initial value.

6. The method according to claim 1, characterized in that The method further comprises: For the video frame to be detected that is a key frame in the video to be detected, obtaining histogram information of each video frame in the video to be detected; Based on the acquired histogram information of each video frame in the video to be detected, it is determined based on a preset method whether the video frame to be detected is a scene mutation video frame.

7. The method according to claim 6, characterized in that The determining, based on a preset method, whether the video frame to be detected is a scene mutation video frame includes: Calculating second histogram difference information between the histogram information of the video frame to be detected and the histogram information of the video frame before the video frame to be detected; Based on the second histogram difference information and a preset histogram difference threshold, determine whether the video frame to be detected is a scene mutation video frame; wherein, if the second histogram difference information is greater than or equal to the histogram difference threshold, determine that the video frame to be detected is a scene mutation video frame.

8. The method according to claim 6, characterized in that The histogram information includes at least one of luminance histogram information and chrominance histogram information.

9. A scene change detection device, characterized in that: include: An acquisition module, used to obtain the coding prediction mode information of the video to be detected; A processing module is used to determine, for a video frame to be detected that is not a key frame in the video to be detected, whether the video frame to be detected is a scene mutation video frame based on the encoding prediction mode information and a preset intra-frame prediction pixel ratio threshold; wherein, if the intra-frame prediction pixel ratio included in the encoding prediction mode information is greater than the intra-frame prediction pixel ratio threshold, it is determined that the video frame to be detected is a scene mutation video frame in which a scene change has occurred.

10. The scene change detection device according to claim 9, wherein: The processing module is further configured to obtain, for each key frame of the video to be detected, histogram information of the video to be detected; and determine, based on the obtained histogram information of each frame of the video to be detected and a preset method, whether the video frame to be detected is a scene mutation video frame.

11. An electronic device, characterized in that: include: a memory and a processor, the memory and the processor being connected; The memory is used to store programs; The processor is configured to call a program stored in the memory to execute the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is run by a computer, the method according to any one of claims 1 to 8 is executed.

Images