Flicker analysis method and device of video, electronic equipment and storage medium

Abstract

The application discloses a flicker analysis method and device of a video, an electronic device and a storage medium. The method comprises the following steps: sampling a plurality of video frames from a to-be-detected video as sampling video frames; determining a corresponding depth image for each sampling video frame; for each sampling video frame, determining a candidate pixel from the sampling video frame based on the depth image of the sampling video frame; determining a matching region matched with the candidate pixel from a related sampling video frame of the sampling video frame based on the depth image of the sampling video frame; the related sampling video frame is a sampling video frame adjacent to and located after the sampling video frame; and determining a flicker analysis result of the to-be-detected video based on the difference between each candidate pixel and the matching region matched with the candidate pixel in each sampling video frame. According to the method, the flicker analysis accuracy of the video is high, and the flicker analysis cost is low.

Description

Technical Field

[0001] This application relates to the field of electronic information technology, and more specifically, to a method, apparatus, electronic device, and storage medium for analyzing video flicker. Background Technology

[0002] During video capture, changes in lighting or shooting angle can cause flickering in the captured video. Currently, flicker analysis models can be used to detect flickering in videos to determine its presence. However, this method is costly and has low accuracy. Summary of the Invention

[0003] In view of this, embodiments of this application propose a method, apparatus, electronic device, and storage medium for analyzing video flicker.

[0004] In a first aspect, embodiments of this application provide a method for flicker analysis of a video. The method includes: sampling multiple video frames from a video to be detected as sampled video frames; determining a corresponding depth image for each sampled video frame; the depth image including the depth value of each pixel in the corresponding sampled video frame; for each sampled video frame, determining candidate pixels from the sampled video frames based on the depth image of the sampled video frames; determining a matching region matching the candidate pixels from related sampled video frames based on the depth image of the sampled video frames; the related sampled video frames being adjacent to and following the sampled video frames; and determining the flicker analysis result of the video to be detected based on the difference between each candidate pixel and its matching region in each sampled video frame.

[0005] Secondly, embodiments of this application provide a video flicker analysis apparatus, comprising: a sampling module for sampling multiple video frames from a video to be detected as sampled video frames; a first determining module for determining a corresponding depth image for each sampled video frame; the depth image including the depth value of each pixel in the corresponding sampled video frame; a second determining module for determining candidate pixels from the sampled video frames based on the depth image of the sampled video frames for each sampled video frame; a matching module for determining a matching region matching the candidate pixels from related sampled video frames based on the depth image of the sampled video frames; the related sampled video frames are sampled video frames adjacent to and following the sampled video frames; and a result determining module for determining the flicker analysis result of the video to be detected based on the difference between each candidate pixel and its matching region in each sampled video frame.

[0006] Optionally, the matching module is further configured to determine a first matching distance based on the depth value of the candidate pixel; determine the matching region corresponding to the candidate pixel from the relevant sampled video frames based on the position of the candidate pixel in the sampled video frame and the matching distance; and determine the size of the matching region based on the first matching distance.

[0007] Optionally, the video to be detected is a video viewed from the perspective of the target object; the matching module is further used to determine the offset distance, offset direction, and second matching distance of the candidate pixel based on the depth value of the candidate pixel and the motion information of the target object; and to determine the matching region corresponding to the candidate pixel from the relevant sampled video frames based on the position of the candidate pixel in the sampled video frame, the second matching distance, the offset direction, and the offset distance; the size of the matching region is determined based on the second matching distance.

[0008] Optionally, the second determining module is further configured to, for each pixel in the sampled video frame, determine the pixel as a candidate pixel if the pixel's depth value is within a specified depth value range.

[0009] Optionally, the result determination module is further configured to, for each candidate pixel, determine the pixel flicker analysis result corresponding to each candidate pixel based on the difference between the candidate pixel and its matching region; determine the video frame flicker analysis result corresponding to each sampled video frame based on the pixel flicker analysis result of each candidate pixel in each sampled video frame; and determine the flicker analysis result of the video to be detected based on the video frame flicker analysis result corresponding to each sampled video frame.

[0010] Optionally, the result determination module is further configured to identify target pixels exhibiting flickering from each sampled video frame; and to perform video frame flicker analysis based on the distribution information of each target pixel in each sampled video frame, thereby obtaining the video frame flicker analysis result for each sampled video frame.

[0011] Optionally, the result determination module is further configured to, for each sampled video frame, determine a binary image indicating each target pixel in the sampled video frame based on the distribution information of each target pixel in the sampled video frame; perform denoising processing on the binary image to obtain a denoised binary image; and perform video frame flicker analysis based on the denoised binary image to obtain the video frame flicker analysis result of the sampled video frame.

[0012] Optionally, the result determination module is also used to perform connected component analysis on the binary image to obtain an intermediate binary image including multiple connected components; the binary pixels in the binary image that indicate each target pixel are located in each connected component; and to delete connected components in the intermediate binary image whose area is smaller than a preset area to obtain a denoised binary image.

[0013] Optionally, the result determination module is also used to determine the candidate pixels and the pixel differences between each pixel in the matching region of the candidate pixels; based on the pixel differences, determine the target pixel differences corresponding to the candidate pixels; if the target pixel differences are less than the pixel difference threshold, obtain the pixel flicker analysis result that the candidate pixels do not have flickering phenomenon.

[0014] Optionally, the video to be detected is a video taken from the perspective of the target object when the target object moves in the target scene; the device also includes a video acquisition module for determining at least one target point from the target scene; for each target point, a video taken from the perspective of the target object when the target object moves from the target point as the starting point of movement and according to the target movement mode is acquired, which is used as a video to be detected for the target point and the target movement mode.

[0015] Optionally, the video acquisition module is also used to acquire the movement path of the target object during the movement of the target object in the target scene; and to sample path points from the movement path at specified distance intervals to obtain at least one target point.

[0016] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory; the memory stores computer-readable instructions, which, when executed by the processor, implement the above-described method.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the above-described method.

[0018] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the method described above.

[0019] This application provides a video flicker analysis method, apparatus, electronic device, and storage medium. In this application, multiple sampled video frames are sampled from the video to be detected. Then, a depth image is determined for each sampled video frame. Based on the sampled video frames and their corresponding depth images, candidate pixels and their corresponding matching regions are determined. Finally, based on the differences between each candidate pixel and its matching region in each sampled video frame, the flicker analysis result of the video to be detected is determined. On the one hand, compared to schemes that analyze the video to be detected using a flicker analysis model, this application reduces the reliance on flicker analysis models and lowers the cost of flicker analysis of the video to be detected. On the other hand, during the training of the flicker analysis model, different flickering phenomena are actually learned using the flicker analysis model. However, the collected samples are difficult to cover the diverse flickering phenomena, making it difficult for the trained flicker analysis model to accurately analyze the flickering of the video to be detected. In this application, candidate pixels and matching regions of candidate pixels are determined based on the depth image of the sampled video frame. Flicker analysis is performed based on the difference between the candidate pixels and the matching regions of candidate pixels. The difference between the candidate pixels and the matching regions of candidate pixels can accurately indicate whether the candidate pixels have flickered, thereby making the flicker analysis results of the video to be detected more accurate. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram is shown illustrating the application scenarios applicable to the embodiments of this application;

[0022] Figure 2 A flowchart of a video flicker analysis method according to an embodiment of this application is shown;

[0023] Figure 3 A schematic diagram of a sampled video frame is shown in an embodiment of this application;

[0024] Figure 4 It shows Figure 3 A schematic diagram of the depth image corresponding to the sampled video frame;

[0025] Figure 5 It shows Figure 2 A flowchart of step S150 in one embodiment is shown in the corresponding example;

[0026] Figure 6It shows Figure 2 A flowchart of the steps preceding step S110 in the corresponding embodiment is shown in one embodiment;

[0027] Figure 7 A flowchart of a video flicker analysis method according to another embodiment of this application is shown;

[0028] Figure 8 A schematic diagram of a video flicker analysis process in an embodiment of this application is shown;

[0029] Figure 9 A block diagram of a video flicker analysis apparatus according to an embodiment of this application is shown;

[0030] Figure 10 A structural block diagram of an electronic device for performing a video flicker analysis method according to an embodiment of this application is shown. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0032] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit the application. It should be noted that "multiple" as used herein refers to two or more. "And / or" describes the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0034] Please refer to Figure 1 The diagram illustrates an application scenario applicable to the embodiments of this application. This application scenario includes a terminal 110 and a server 120.

[0035] Terminal 110 can be a smartphone, tablet, e-book reader, music player, wearable device, smart home device, in-vehicle terminal, etc. Terminal 110 has a video flicker analysis client installed. This client can analyze the video to be detected to obtain flicker analysis results, or it can send a video flicker analysis request so that server 120 can analyze the video to be detected based on the request and obtain flicker analysis results.

[0036] The server 120 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0037] Terminal 110 can generate and send a video flicker analysis request to server 120 (via a video flicker analysis client). Server 120 obtains the video to be detected based on the video flicker analysis request, samples multiple video frames from the video to be detected as sampled video frames, determines the corresponding depth image for each sampled video frame, performs flicker analysis on the video to be detected based on the depth images of each sampled video frame, and obtains the flicker analysis result of the video to be detected. Finally, server 120 returns the flicker analysis result of the video to be detected to terminal 110 so that terminal 110 can display the flicker analysis result of the video to be detected.

[0038] In some implementations, the server 120 can directly acquire the video to be detected and sample multiple video frames from the video to be detected as sampled video frames. Then, it can determine the corresponding depth image for each sampled video frame. After that, it can perform flicker analysis on the video to be detected based on the depth images of each sampled video frame to obtain the flicker analysis result of the video to be detected.

[0039] Alternatively, the terminal 110 can directly acquire the video to be detected and sample multiple video frames from the video to be detected as sampled video frames. Then, it can determine the corresponding depth image for each sampled video frame. After that, it can perform flicker analysis on the video to be detected based on the depth images of each sampled video frame to obtain the flicker analysis result of the video to be detected.

[0040] To more clearly explain the solution of this application, the following embodiments all use electronic devices as the execution subject of the video flicker analysis method.

[0041] Please see Figure 2 , Figure 2 This application illustrates a flowchart of a video flicker analysis method according to an embodiment of the present application. The method is applied to an electronic device, which may be... Figure 1 The terminal 110 or server 120 in the middle, the method may include:

[0042] S110. Sample multiple video frames from the video to be detected as sampled video frames.

[0043] The video to be tested refers to the video to be subjected to flicker analysis. The video to be tested can be a video obtained by capturing a real scene through a real camera in a real scene. For example, the video to be tested can be a video taken by user a1 using a mobile phone in office b1.

[0044] The video to be tested can also be a video viewed from the perspective of a virtual object in a virtual scene. The perspective of the virtual object can include both first-person and third-person perspectives. For example, the video to be tested could be a video observed from the third-person perspective of a character a2 in a game.

[0045] The video to be detected can also be a synthetic video generated by a video generation model or a large language model based on given instruction information. For example, "a red car driving at a constant speed on a country road on a rainy day" can be used as the prompt information to input into a large language model for video generation, and the resulting video can be used as the video to be detected.

[0046] In the video to be tested, each video frame can be used as a sampled video frame, thus enabling analysis of each video frame. Alternatively, a portion of the video frames can be sampled from the video to be tested, for example, sampling one video frame every four or five frames from the video to be tested.

[0047] Of course, when sampling video frames from the video to be detected, the interval between two adjacent sampled video frames can be equal, for example, 5 frames apart. The interval between two adjacent sampled video frames can also be unequal, for example, the first sampled video frame is 4 frames apart from the second sampled video frame, and the fifth sampled video frame is 6 frames apart from the sixth sampled video frame.

[0048] S120. Determine the corresponding depth image for each sampled video frame.

[0049] The depth image includes the depth value of each pixel in the corresponding sampled video frame.

[0050] That is, for each sampled video frame, the depth value of each pixel in the sampled video frame is determined to determine the depth value of each pixel. Then, based on the arrangement position of each pixel in the sampled video frame, the depth value of each pixel is used as the pixel value and synthesized into an image indicating the depth value of each pixel, which is used as the depth image of the sampled video frame.

[0051] In some implementations, the user can manually determine the depth value of each pixel in the sampled video frame, and then, based on the arrangement position of each pixel in the sampled video frame, use the depth value of each pixel as the pixel value to synthesize an image indicating the depth value of each pixel, which is then used as the depth image of the sampled video frame.

[0052] In some implementations, a depth image can be generated based on the sampled video frame using a depth estimation model to obtain the depth image corresponding to the sampled video frame. The depth estimation model can be Stable Diffusion or Depth Anything, etc.

[0053] Of course, depth estimation models can also be trained using deep neural networks, convolutional neural networks (CNNs), and graph neural networks with parameters initialized based on sample images and their corresponding depth images. The sample depth image includes the depth value of each pixel in the sample image. For example, a depth estimation model initialized with sample images can be used to generate a predicted depth image. Then, based on the difference between the predicted depth image and the sample depth image, the depth estimation model can be trained to generate depth images with high accuracy.

[0054] For example Figure 3 The sampled video frames shown, the determined depth images are as follows: Figure 4 As shown, for Figure 3 For each pixel in the image, the larger the depth value of the pixel, the better. Figure 4 The higher the pixel value of a pixel (the darker the color, the higher the pixel value), the better.

[0055] S130. For each sampled video frame, determine candidate pixels from the sampled video frame based on the depth image of the sampled video frame.

[0056] In this application, flicker analysis of the sampled video frame is performed based on the depth image of the sampled video frame. First, the depth value of each pixel in the sampled video frame is obtained from the depth image of the sampled video frame. Then, based on the depth value of each pixel, the pixels that change less in the next sampled video frame are determined as candidate pixels.

[0057] In some implementations, for each pixel in a sampled video frame, if the pixel's depth value falls within a specified depth value range, the pixel is identified as a candidate pixel. The specified depth value range can refer to a range of depth values ​​in the video to be detected that are not prone to sudden changes. However, the specified depth value range may differ for different scenarios involving the video to be detected.

[0058] For example, when the video to be detected is a video captured by a real camera of urban buildings, the specified depth value range is a depth value greater than 5. As another example, when the video to be detected is a video captured by a real camera of terraced fields in a rural area, the specified depth value range is a depth value greater than 10. Furthermore, when the video to be detected is also a video viewed from the third-person perspective of a gunman in a virtual shooting scene, the specified depth value range is a depth value greater than 20.

[0059] For each pixel in a sampled video frame, if the pixel's depth value is within a specified depth range, it means that the indicated position of that pixel is far from the observation position of the sampled video frame. Compared to that pixel, the change in the pixel indicating that position in the next sampled video frame is small, thus identifying that pixel as a candidate pixel. If the pixel's depth value is not within the specified depth range, it means that the indicated position of that pixel is close to the observation position of the sampled video frame. Compared to that pixel, the change in the pixel indicating that position in the next sampled video frame is large, thus identifying that pixel as not a candidate pixel. Here, the indicated position of a pixel refers to a location indicated by the pixel. For videos captured in a real environment, the indicated position is a real location; for videos captured in a virtual environment, the indicated position is a virtual location.

[0060] S140. Based on the depth image of the sampled video frame, determine the matching region that matches the candidate pixel from the related sampled video frames.

[0061] Among them, the relevant sampled video frame is the sampled video frame that is adjacent to the sampled video frame and located after the sampled video frame.

[0062] In the video to be tested, the video frames are arranged in chronological order of playback time. Therefore, the obtained sampled video frames are also arranged in chronological order of playback time. This means that the (n+1)th sampled video frame is a related sampled video frame of the nth sampled video frame. For example, if the test video consists of 600 consecutive video frames, and the video to be tested has 100 sampled video frames, then for the 60th sampled video frame, the corresponding related sampled video frame is the 61st sampled video frame.

[0063] For each candidate pixel hx1, based on the depth value of the candidate pixel hx1, a region f1 is determined from the related sampled video frames of the sampled video frame as the matching region of the candidate pixel hx1. The indication region indicated by region f1 (including the indication position indicated by each pixel within range f1) includes the indication position indicated by the candidate pixel hx1. In other words, the matching region of each candidate pixel is determined based on the depth value of each candidate pixel.

[0064] In some implementations, S140 may further include: determining a first matching distance based on the depth value of the candidate pixel; determining a matching region corresponding to the candidate pixel from the relevant sampled video frames based on the position of the candidate pixel in the sampled video frames and the matching distance; and determining the size of the matching region based on the first matching distance.

[0065] The first matching distance, determined for different depth values, can be set based on requirements. Generally, more distant indicator positions show less variation in adjacent sampled video frames; therefore, candidate pixels with larger depth values ​​have smaller first matching distances. Conversely, more nearby indicator positions show greater variation in adjacent sampled video frames; therefore, candidate pixels with smaller depth values ​​have larger first matching distances. For example, when the depth value is 20-100, the corresponding first matching distance is 50 pixels; and when the depth value is greater than 100, the corresponding first matching distance is 20 pixels.

[0066] In this application, a preset shape can be set for determining the matching region for candidate pixels. The preset shape can be, for example, a rectangle, a square, or a circle. Therefore, for each candidate pixel, a first matching distance is determined based on the candidate pixel's depth value. Based on the target size of the first matching distance, and then with the target position in the relevant sampled video frame corresponding to the candidate pixel as the center, the area enclosed by a graphic with the target size and the preset shape is determined as the matching region for the candidate pixel. Here, the target position refers to the pixel position in the relevant sampled video frame whose image coordinates are consistent with the image coordinates of the candidate pixel in the sampled video frame.

[0067] For example, the preset shape is a rectangle with an aspect ratio of 1:1.2, the image coordinates of the candidate pixel are z1, the image coordinates of the target position corresponding to the candidate pixel are also z1, the depth value of the candidate pixel is d1, and the corresponding first matching distance is determined to be 50 pixels. At this time, the target size is determined to be 50 pixels wide and 60 pixels high. Then, the area selected by the rectangle with z1 as the center, 50 pixels wide and 60 pixels high in the relevant sampled image is determined as the matching area of ​​the candidate pixel.

[0068] In some other embodiments, the video to be detected is a video viewed from the perspective of the target object; accordingly, S140 may include: determining the offset distance, offset direction, and second matching distance of the candidate pixel based on the depth value of the candidate pixel and the motion information of the target object; determining the matching region corresponding to the candidate pixel from the relevant sampled video frames based on the position of the candidate pixel in the sampled video frames, the second matching distance, the offset direction, and the offset distance; the size of the matching region is determined based on the second matching distance. Here, the target object can refer to an object that observes the video to be detected; for example, in a game scene, the target object can be a virtual shooter, or in a real scene, the target object can be a real camera; the motion information of the target object refers to information used to indicate the motion state of the target object.

[0069] First, the offset direction of the candidate pixel indicator's position in the relevant sampled video frame varies depending on the target object's motion state. For example, when the target object is moving straight, the candidate pixel indicator's position does not shift in the relevant sampled video frame (at this time, the corresponding offset distance is also zero). As another example, when the target object turns left, the candidate pixel indicator's position shifts to the right in the relevant sampled video frame. Yet another example is when the target object turns right, the candidate pixel indicator's position shifts to the left in the relevant sampled video frame.

[0070] Secondly, for different candidate pixels, the offset distance and second matching distance may differ depending on the depth value. Generally, farther indicator positions change less in adjacent sampled video frames; therefore, candidate pixels with larger depth values ​​have smaller offset distances and smaller second matching distances. Conversely, closer indicator positions change more in adjacent sampled video frames; therefore, candidate pixels with smaller depth values ​​have larger offset distances and larger second matching distances. For example, when the depth value is 20-100, the corresponding second matching distance is 50 pixels and the offset distance is 30 pixels. Similarly, when the depth value is greater than 100, the corresponding second matching distance is 20 pixels and the offset distance is 10 pixels.

[0071] For candidate pixels with the same depth value, the second matching distance varies depending on the motion state of the target object. For example, when the target object is moving straight, the second matching distance is 42 pixels when the depth value is 20-100. When the target object rotates its viewpoint, the second matching distance is 52 pixels when the depth value is 20-100.

[0072] Similarly, for candidate pixels with the same depth value, the offset distance varies depending on the motion state of the target object. For example, when the target object is viewed from the left, the offset distance is 52 pixels when the depth value is 20-100. When the target object is viewed from the top, the offset distance is 60 pixels when the depth value is 20-100.

[0073] Therefore, for each candidate pixel, a second matching distance is determined based on the depth value of the candidate pixel. Based on the target size of the second matching distance, and then taking the target position corresponding to the candidate pixel in the relevant sampled video frame as the starting point, and the position after offsetting the offset distance along the offset direction as the center, the area selected by the graphic with the target size and the preset shape is determined as the matching area of ​​the candidate pixel.

[0074] S150. Based on the differences between each candidate pixel and its matching region in each sampled video frame, determine the flicker analysis result of the video to be detected.

[0075] For each sampled video frame, the presence of flickering can be determined based on the differences between candidate pixels and their matching regions. This process is repeated for all sampled video frames. If none of the sampled video frames exhibit flickering, the flicker analysis result for the video to be detected is that the video to be detected does not exhibit flickering. If at least one sampled video frame exhibits flickering, the flicker analysis result for the video to be detected is that the video to be detected exhibits flickering. In this case, the flicker analysis result may also include the location of the sampled video frame exhibiting flickering within the video to be detected (e.g., indicated by frame number or frame identifier).

[0076] In this embodiment, the mean of the pixel value differences between the candidate pixel and each pixel in its matching region can be determined as the difference between the candidate pixel and its matching region; alternatively, the mean of the pixel values ​​of each pixel in the matching region corresponding to the candidate pixel can be determined, and then the difference between the pixel value of the candidate pixel and the mean of the pixel values ​​can be calculated as the difference between the candidate pixel and its matching region; alternatively, pixel value features of the candidate pixel and the pixel value features of the matching region can be extracted, and the similarity of the pixel value features of the candidate pixel and the pixel value features of each pixel in the matching region can be determined (which may be cosine similarity, Euclidean distance, or Manhattan distance, etc.), and the mean of the pixel value feature similarity can be calculated as the difference between the candidate pixel and its matching region; or alternatively, pixel value features of the candidate pixel and the pixel value features of the matching region can be extracted, the mean of the pixel value features of each pixel in the matching region can be determined as the mean pixel value feature, and then the similarity between the pixel value features of the candidate pixel and the mean pixel value feature can be determined (which may be cosine similarity, Euclidean distance, or Manhattan distance, etc.) as the difference between the candidate pixel and its matching region.

[0077] For each candidate pixel, if the difference between the candidate pixel and its matching region is less than a preset difference, the pixel flicker analysis result for the candidate pixel is determined to be that the candidate pixel does not flicker; if the difference between the candidate pixel and its matching region is not less than the preset difference, the pixel flicker analysis result for the candidate pixel is determined to be that the candidate pixel does flicker. The preset difference can be set based on requirements and is not limited in this application.

[0078] Correspondingly, if the pixel flicker analysis results of each candidate pixel in the sampled video frame are all negative, it is determined that the sampled video frame does not have flicker; if the sampled video frame includes candidate pixels that exhibit flicker, it is determined that the sampled video frame exhibits flicker.

[0079] Of course, when there is flickering in the sampled video frame, the analysis results of the sampled video frame can also include the location of the candidate pixels where flickering occurs, so as to determine the location of the flickering phenomenon based on the analysis results of the sampled video frame.

[0080] It is easy to understand that in this embodiment, the flicker analysis result is determined by the difference between candidate pixels and their corresponding matching pixel regions. Therefore, the scene appearing in a video to be detected is unchanged, or it switches between similar scenes. This is because when the scene changes or the difference between the changing scenes is large, the difference between video frames will inevitably increase, leading to inaccurate analysis results.

[0081] In this embodiment, multiple sampled video frames are sampled from the video to be detected. Then, a depth image is determined for each sampled video frame. Based on the sampled video frames and their corresponding depth images, candidate pixels and their corresponding matching regions are determined. Finally, based on the differences between each candidate pixel and its matching region in each sampled video frame, the flicker analysis result of the video to be detected is determined. On the one hand, compared to schemes that analyze the video to be detected using a flicker analysis model, this application reduces the reliance on the flicker analysis model and lowers the cost of flicker analysis for the video to be detected. On the other hand, during the training process of the flicker analysis model, different flicker phenomena are actually learned using the model. However, the collected samples are insufficient to cover diverse flicker phenomena, making it difficult for the trained flicker analysis model to accurately analyze the flicker of the video to be detected. In this application, candidate pixels and their matching regions are determined based on the depth images of the sampled video frames. Flicker analysis is performed based on the differences between the candidate pixels and their matching regions. The differences between the candidate pixels and their matching regions accurately indicate whether the candidate pixels have flickered, resulting in a higher accuracy rate for the flicker analysis result of the video to be detected.

[0082] In some embodiments, such as Figure 5 As shown, S150 includes:

[0083] S151. For each candidate pixel, based on the difference between the candidate pixel and its matching region, determine the pixel flicker analysis result corresponding to each candidate pixel.

[0084] In some embodiments, the process of determining the difference between a candidate pixel and its matching region is as described in S150 of the preceding embodiments. At this time, for each candidate pixel, if the difference between the candidate pixel and its matching region is less than a preset difference, the pixel flicker analysis result for the candidate pixel is determined to be that the candidate pixel does not flicker; if the difference between the candidate pixel and its matching region is not less than the preset difference, the pixel flicker analysis result for the candidate pixel is determined to be that the candidate pixel does flicker. The preset difference here can be set based on requirements and is not limited in this application.

[0085] In some other embodiments, S151 may also include: determining candidate pixels and the pixel differences between each pixel within the matching region of the candidate pixels; determining the target pixel difference corresponding to the candidate pixels based on the pixel differences; and obtaining a pixel flicker analysis result that the candidate pixels do not exhibit flickering if the target pixel difference is less than a pixel difference threshold. Here, pixel difference can refer to the absolute value of the difference between pixel values, and the pixel difference threshold can be set based on requirements, such as 60.

[0086] For example, when the pixel values ​​include red, green, and blue pixel values, the process of determining the pixel difference can be: Pixel difference = |Rnext-Rprev| + |Gnext-Gprev| + |Bnext-Bprev|, where Rnext, Gnext, and Bnext are the red, green, and blue pixel values ​​of any pixel in the relevant sampled video frame corresponding to the candidate pixel, respectively, and Rprev, Gprev, and Bprev are the red, green, and blue pixel values ​​of the candidate pixel, respectively.

[0087] For each candidate pixel, iterate through each pixel within its corresponding matching region to obtain multiple pixel differences. Each pixel in the matching region corresponds to one pixel difference; for example, if the matching region includes 100 pixels, then the candidate pixel corresponds to 100 pixel differences. Then, select the maximum, mode, mean, or minimum value among these multiple pixel differences as the target pixel difference. If the target pixel difference is less than the pixel difference threshold, it means the difference between the candidate pixel and its corresponding matching region is small, resulting in a pixel flicker analysis result indicating no flickering in the candidate pixel. If the target pixel difference is not less than the pixel difference threshold, it means the difference between the candidate pixel and its corresponding matching region is large, resulting in a pixel flicker analysis result indicating flickering in the candidate pixel.

[0088] S152. Based on the pixel flicker analysis results of each candidate pixel in each sampled video frame, determine the video frame flicker analysis results corresponding to each sampled video frame.

[0089] For each sampled video frame, if the pixel flicker analysis results of each candidate pixel in the sampled video frame are all negative, then the video frame flicker analysis result with no flicker is obtained. If the pixel flicker analysis results of each candidate pixel in the sampled video frame show at least one flickering phenomenon, then the video frame flicker analysis result with flicker is obtained. Of course, the video frame flicker analysis result can also include the position of the candidate pixel with flickering phenomenon in the sampled video frame.

[0090] In some implementations, S152 may include: determining target pixels exhibiting flickering from each sampled video frame; performing video frame flicker analysis based on the distribution information of each target pixel in each sampled video frame to obtain video frame flicker analysis results for each sampled video frame, wherein the pixel flicker analysis results indicate whether the corresponding candidate pixel exhibits flickering.

[0091] For each sampled video frame, the density of target pixels (the ratio of the number of target pixels to the size of the sampled video frame) can be determined as the distribution information of the target pixels. Then, based on the density of target pixels in the sampled video frame, the video frame flicker analysis result of the sampled video frame is determined. For example, if the density of target pixels in the sampled video frame is less than a specified density, the video frame flicker analysis result of the sampled video frame is determined to be that there is no flicker; if the density of target pixels in the sampled video frame is not less than the specified density, the video frame flicker analysis result of the sampled video frame is determined to be that there is flicker. Here, the specified density can be, for example, 0.2.

[0092] Alternatively, the maximum area (the maximum area formed by adjacent target pixels) and density of the target pixels in the sampled video frame can be determined as the distribution information of the target pixels. Then, based on the density of the target pixels in the sampled video frame, the video frame flicker analysis result of the sampled video frame can be determined. For example, if the density of the target pixels in the sampled video frame is less than a specified density and the maximum area is less than an area threshold, the video frame flicker analysis result of the sampled video frame is determined to be flicker-free. If the density of the target pixels in the sampled video frame is not less than a specified density or the maximum area is not less than an area threshold, the video frame flicker analysis result of the sampled video frame is determined to be flicker-free. The area threshold can be, for example, 50.

[0093] Of course, the maximum area here can also be replaced by the average area or minimum area of ​​the target pixels in the sampled video frame. The average area refers to the average area of ​​the region formed by adjacent target pixels, and the minimum area refers to the minimum area of ​​the region formed by adjacent target pixels.

[0094] Alternatively, for each sampled video frame, based on the distribution information of each target pixel in the sampled video frame, a binary image indicating each target pixel in the sampled video frame is determined; the binary image is denoised to obtain a denoised binary image; and video frame flicker analysis is performed based on the denoised binary image to obtain the video frame flicker analysis result of the sampled video frame.

[0095] In a binary image, the pixel value of the pixel corresponding to the target pixel in the sampled video frame can be 1, and the pixel value of the pixel corresponding to the pixel other than the target pixel in the sampled video frame can be 0. Alternatively, the pixel value of the pixel corresponding to the target pixel in the sampled video frame can be 0, and the pixel value of the number of pixels corresponding to the pixel other than the target pixel in the sampled video frame can be 1. Thus, the distribution of the target pixel can be more clearly indicated by the binary image.

[0096] Denoising of binary images can be achieved through filtering or by using a denoising model. The denoising model can be obtained by training a neural network with parameters initialized using binary images of samples containing noise and denoised binary images of samples without noise. The denoising model can be a convolutional neural network, a deep neural network, or a graph neural network, etc.

[0097] In some implementations, connected component analysis can be performed on the binary image to obtain an intermediate binary image comprising multiple connected components; binary pixels in the binary image indicating each target pixel are located in each connected component; connected components in the intermediate binary image with an area smaller than a preset area are deleted to obtain a denoised binary image. Here, a binary pixel refers to any pixel in the binary image, and the preset area can be, for example, 40.

[0098] If the area of ​​a connected component is smaller than a preset area, it means the connected component is small, contains fewer target pixels, and exhibits less noticeable flickering; therefore, the connected component can be deleted. If the area of ​​a connected component is not smaller than the preset area, it means the connected component is large, contains more target pixels, and exhibits noticeable flickering; therefore, the connected component needs to be retained. The process involves traversing all connected components in the binary image to obtain a binary image with connected components whose areas are smaller than the preset area removed; this image is then used as the denoised binary image.

[0099] Therefore, by using the above method to denoise binary images, connected components with inconspicuous flickering are ignored, thus improving the accuracy of flicker analysis.

[0100] After obtaining the denoised binary image, video frame flicker analysis can be performed based on the denoised binary image to obtain the video frame flicker analysis results of the sampled video frame. Specifically, the video frame flicker analysis results of the sampled video frame can be determined based on the density and / or maximum binary area of ​​the target binary image indicating the target pixel in the denoised binary image.

[0101] For example, the density of the target binary pixel (the binary pixel indicating the target pixel is used as the target binary pixel) in the denoised binary image is used as the distribution information of the binary pixels. If the density of the target binary pixel is less than the preset density, the video frame flicker analysis result of the sampled video frame is determined to be that there is no flicker phenomenon. If the density of the target binary pixel is not less than the preset density, the video frame flicker analysis result of the sampled video frame is determined to be that there is flicker phenomenon. The preset density can be, for example, 0.15.

[0102] For example, in a denoised binary image, the maximum binary area (the maximum area formed by adjacent binary pixels) and density of the target binary pixels are used to determine the flicker analysis result of the sampled video frame. If the density of the target binary pixels is less than a preset density and the maximum binary area is less than a binary area threshold, the flicker analysis result of the sampled video frame is determined to be that there is no flicker. If the density of the target binary pixels is not less than a preset density or the maximum binary area is not less than a binary area threshold, the flicker analysis result of the sampled video frame is determined to be that there is flicker. The binary area threshold can be, for example, 50.

[0103] Of course, the maximum binary area here can also be replaced with the average binary area or the minimum binary area. The average binary area refers to the average area of ​​the region formed by adjacent target binary pixels, and the minimum binary area refers to the minimum area of ​​the region formed by adjacent target binary pixels.

[0104] As mentioned above, when the denoised binary image is the image obtained by deleting connected components with an area smaller than a preset area from a binary image containing multiple connected components, if the denoised binary image does not include connected components, the video frame flicker analysis result of the sampled video frame is determined to be that there is no flickering phenomenon. If the denoised binary image includes at least one connected component, the video frame flicker analysis result of the sampled video frame is determined to be that there is flickering phenomenon. In this case, the video frame flicker analysis result can also include the flickering area indicated by each connected component in the sampled video frame, so as to view the flickering area based on the video frame flicker analysis result.

[0105] It's easy to understand that the blinking area can be indicated using specific prompts, such as drawing an outline of the blinking area in a specified color, covering the blinking area with a transparent layer of a specified color, or blinking the blinking area.

[0106] S153. Based on the flicker analysis results of each sampled video frame, determine the flicker analysis results of the video to be detected.

[0107] If all sampled video frames are free of flicker (the flicker analysis results for all sampled video frames are all free of flicker), the flicker analysis result for the video to be tested is that the video to be tested is free of flicker. If at least one sampled video frame is flicker (the flicker analysis results for at least one sampled video frame are all flicker), the flicker analysis result for the video to be tested is that the video to be tested is flicker. In this case, the flicker analysis result may also include the position of the sampled video frame with flicker in the video to be tested.

[0108] In this embodiment, flicker analysis is performed on each candidate pixel, which enables a comprehensive analysis of the sampled video frames and results in a high accuracy of the flicker analysis results.

[0109] Secondly, flicker analysis is performed based on the distribution information of target pixels exhibiting flickering phenomena, so as to determine from an observational perspective whether the flickering phenomenon is obvious, resulting in a higher accuracy of the flicker analysis results.

[0110] Furthermore, the binary image indicating the target pixel is also denoised to filter out inconspicuous connected regions, making the flicker analysis results more in line with the user's viewing needs and resulting in a higher accuracy rate for the flicker analysis results.

[0111] In some embodiments, the video to be detected is a video taken from the perspective of the target object as it moves within the target scene, such as... Figure 6 As shown, before S110, the method further includes:

[0112] S210. Determine at least one target point from the target scene.

[0113] The target scene can be a real or virtual scene, and the target object can be any object that can move within the target scene. The target object's movement methods can include moving forward, jumping, vaulting over walls, moving backward, and rotating the viewpoint. For example, the target scene could be a shooting range in a virtual game scene, and the target object could be a virtual shooter moving within the shooting scene.

[0114] In some implementations, at least one target point can be determined from the target scene based on its size and the distribution of object trajectories of moving objects within it. For example, the target scene can be divided into multiple grid regions based on its size and the distribution of object trajectories of moving objects within it, with each vertex of each grid region serving as a target point.

[0115] For example, a target scene of 1000×1000 square area can be divided into grid areas with a unit grid size of 20×20, and each vertex of each grid can be used as a target point. Similarly, a target scene of 10000×10000 square area can be divided into grid areas with a unit grid size of 100×100, and each vertex of each grid can be used as a target point.

[0116] For example, if the target scene is a 1000×1000 square field, the distribution of the object trajectories of moving objects in the square field is as follows: the distribution density of object trajectories is high in the central area and low in the surrounding areas. Therefore, for the central 500×500 area, it is divided into multiple grids with a unit grid size of 10×10, and each vertex of each grid is used as a target point. For other areas, it is divided into multiple grids with a unit grid size of 20×20, and each vertex of each grid is used as a target point.

[0117] In some other implementations, the movement path of the target object can be obtained as the target object moves in the target scene; path points can be sampled from the movement path at specified distance intervals to obtain at least one target point.

[0118] For example, when the target scene is a virtual scene in a game, the game interface can be used to obtain video of the game character moving in the target scene, and the movement path of the game character in the target scene can be obtained. A point on the movement path can be sampled every 5 meters as the target point.

[0119] Of course, in order to easily distinguish and store these sampled target points, the target points can be arranged according to their order in the movement path to obtain a target point sequence. The order of the target points in the target point sequence is the order in which the game character arrives at the target points when the game character moves.

[0120] S220. For each target point, acquire a video of the target object moving from the target point as the starting point and according to the target movement method, using the target object's perspective as the observation perspective, as a video to be detected for the target point and the target movement method.

[0121] In this application, the target can move forward or move forward after rotating the viewpoint, etc., wherein the rotation direction of the viewpoint can be leftward, rightward, or backward.

[0122] In this way, multiple target movement methods can be set. For each target point, the target object is obtained by moving from the target point as the starting point according to multiple target movement methods. This yields the video to be detected for each target point according to each of the multiple target movement methods. Thus, the number of videos to be detected for each target point is the same as the number of target movement methods.

[0123] For example, given multiple target points in a game scene, the system can iterate sequentially (according to their order in the target point sequence). For the currently visited target point *di*, the game interface can be called to teleport to *di*. First, the game character moves forward at a fixed speed for 1 second, obtaining the video observed by the game character as a target point *di* and a video to be detected during the forward movement. Then, the game interface can be used to teleport the game character back to the original target point *di*, rotating the game character's view to backward and moving forward at the same speed for 1 second, obtaining the video observed by the game character as a target point *di* and a video to be detected during the forward movement. First, a video of the character moving forward after a right turn is generated. Next, the game character can be controlled via the game interface to teleport back to the original target point di, rotate the game character's view to the right, and move forward at the same speed for 1 second. This video, observed by the game character, serves as a test video for target point di and the character moving forward after a right turn. Finally, the game character can be controlled via the game interface to teleport back to the original target point di, rotate the game character's view to the left, and move forward at the same speed for 1 second. This video, observed by the game character, serves as a test video for target point di and the character moving forward after a left turn. Thus, four test videos are generated for target point di based on four different movement methods.

[0124] In this embodiment, multiple target points are automatically sampled from the target scene, and the video of the target object moving from the target point is used as the video to be detected. This eliminates the need to manually collect the video to be detected, thus improving the efficiency of acquiring the video to be detected.

[0125] Please see Figure 7 , Figure 7 This illustration shows a flowchart of a video flicker analysis method according to another embodiment of this application. The method can be used in an electronic device, which may be... Figure 1 The methods include: (120 on the server side or 110 on the terminal side)

[0126] S310. Sample multiple video frames from the video to be detected as sampled video frames; determine the corresponding depth image for each sampled video frame.

[0127] The description of S310 is the same as that of S110-S120, and will not be repeated here.

[0128] S320. Scale each sampled video frame to obtain a scaled video frame corresponding to each sampled video frame; scale the depth image of each sampled video frame to obtain a scaled depth image corresponding to each sampled video frame.

[0129] The scaled video frame and the scaled depth image of the sampled video frame have the same size, and the scaled depth image includes the depth value of each pixel in the scaled video frame.

[0130] By scaling, the amount of data in video frames and depth maps is reduced, thereby reducing the amount of data that needs to be analyzed in subsequent flicker analysis and improving the efficiency of flicker analysis.

[0131] It is easy to understand that, in order to further improve the efficiency of analysis, the sampled video frames can be scaled first to obtain scaled video frames, and the corresponding depth image of the scaled video frames can be determined as the scaled depth image. At this time, it is not necessary to determine the depth image first and then scale it, which reduces the scaling process of the depth image and can further improve the efficiency of flicker analysis.

[0132] S330. For each scaled video frame, determine the first pixel from the scaled video frame based on the scaled depth image corresponding to the scaled video frame; determine the first region matching the first pixel from the related scaled video frames based on the scaled depth image corresponding to the scaled video frame.

[0133] In this process, a sampled video frame is scaled to obtain a corresponding scaled video frame, and a depth image of a sampled video frame is scaled to obtain a scaled depth image. Thus, the same sampled video frame corresponds to a scaled video frame and a scaled depth image. The scaled video frame and scaled depth image with a corresponding relationship refer to the scaled video frame and scaled depth image corresponding to the same sampled video frame.

[0134] The related zoomed video frame refers to the zoomed video frame that is adjacent to and follows the zoomed video frame.

[0135] In this embodiment, for each pixel in the scaled video frame, if the pixel's depth value is within a specified depth value range, the pixel is determined as a first pixel.

[0136] In some implementations, for each first pixel, a first distance can be determined based on the depth value of the first pixel (determined from a scaled depth image); a first region corresponding to the first pixel is determined from the relevant scaled video frame based on the position of the first pixel in the scaled video frame and the first distance; the size of the first region is determined based on the first distance.

[0137] The detailed description of determining the first region here can be found in the aforementioned description of determining the matching region based on the first matching distance, and will not be repeated here.

[0138] In some other implementations, the video to be detected is a video viewed from the perspective of the target object. In this case, the scaling offset distance, offset direction, and second distance of the first pixel can be determined based on the depth value of the first pixel and the motion information of the target object. Based on the position of the first pixel in the sampled video frame, the second distance, the offset direction, and the scaling offset distance, the first region corresponding to the first pixel can be determined from the relevant scaled video frames.

[0139] The detailed description of determining the first region here can be found in the aforementioned description of determining the matching region based on the second matching distance, and will not be repeated here.

[0140] S340. Based on the difference between each first pixel and its matching first region in each scaled video frame, determine the flicker analysis result of the video to be detected.

[0141] In other words, for each first pixel, based on the difference between the first pixel and its matching first region, the pixel flicker analysis result corresponding to each first pixel can be determined; based on the pixel flicker analysis result of each first pixel in each scaled video frame, the video frame flicker analysis result corresponding to each sampled video frame can be determined; then, for each candidate pixel, based on the difference between the candidate pixel and its matching region, the pixel flicker analysis result corresponding to each candidate pixel can be determined; based on the pixel flicker analysis result of each candidate pixel in each sampled video frame, the video frame flicker analysis result corresponding to each sampled video frame can be determined.

[0142] Specifically, for each scaled video frame, based on the difference between each first pixel and its matching first region in the scaled video frame, it is determined whether the scaled video frame has a flickering phenomenon. If the video frame has a flickering phenomenon, the video frame flickering analysis result of the sampled video frame corresponding to the scaled video frame is determined to have a flickering phenomenon. If the video frame does not have a flickering phenomenon, the video frame flickering analysis result of the sampled video frame corresponding to the scaled video frame is determined to have no flickering phenomenon.

[0143] In this embodiment, the sampled video frames and depth images are also scaled, and flicker analysis is performed based on the scaled depth images and scaled video frames after scaling, which reduces the amount of data for flicker analysis and improves the efficiency of flicker analysis.

[0144] To better understand the solution of this application, the flicker analysis method of this application will be explained below with an example.

[0145] like Figure 8 As shown, it includes:

[0146] S410. Record the game character's path. In this example, the game character's movement path in the game scene is obtained through the game interface, and target points are collected from the movement path to obtain a target point sequence.

[0147] S420. Replay the game character's path. Traverse the target points sequentially according to their order in the target point sequence, control the game character to teleport to the corresponding target point, and move according to the given target movement method, so as to determine a video to be detected for each target point and each target movement method.

[0148] S430. Obtain depth images. Multiple sampled video frames are obtained by sampling the video to be detected, and a depth image is determined for each sampled video frame. The depth image of each sampled video frame includes the depth value of each pixel in the sampled video frame.

[0149] S440. Determine the matching distance based on depth values. Based on the depth value of each pixel in the sampled video frame, determine candidate pixels and their matching distances. Alternatively, the offset distance and direction can be determined based on the movement of the game character, allowing for the combination of candidate pixel positions, matching distances, offset distances, and offset directions to determine the matching region.

[0150] S450. Calculate pixel differences. For each candidate pixel and its corresponding matching distance, determine the pixel difference, and use the maximum pixel difference as the target pixel difference.

[0151] S460. Denoising Processing. Based on the differences in target pixels, determine the pixel flicker analysis results of candidate pixels. Based on the pixel flicker analysis results, determine the distribution information of target pixels exhibiting flickering phenomena, and determine the binary image of the sampled video frame. Construct connected components in the binary image, and delete connected components with areas smaller than a preset area to obtain a denoised binary image. Finally, based on whether connected components exist in the denoised binary image and their locations, determine the video frame flicker analysis results of the sampled video frame corresponding to the denoised binary image.

[0152] If the denoised binary image no longer includes connected components, the video frame flicker analysis result of the sampled video frame is determined to be that there is no flickering phenomenon. If the denoised binary image includes connected components, the video frame flicker analysis result of the sampled video frame is determined to be that there is flickering phenomenon. In this case, the video frame flicker analysis result of the sampled video frame may also include the location of the flickering phenomenon (the location in the sampled video frame that corresponds to the location of the connected component in the denoised binary image).

[0153] S470. Determine the flicker analysis results. Summarize the flicker analysis results of each sampled video frame in the video frame to be detected to obtain the flicker analysis results of the video to be detected.

[0154] This example demonstrates depth value extraction and pixel matching analysis of continuous game images. By calculating the matching distance between each frame and the next, the algorithm can accurately identify the correspondence of pixels in consecutive frames, thereby effectively detecting pixel flickering. This method not only improves the accuracy and efficiency of detection but also maintains high robustness in complex game scenes, providing reliable technical support for optimizing game image quality.

[0155] Please see Figure 9 , Figure 9 This illustration shows a block diagram of a video flicker analysis apparatus according to an embodiment of this application. The apparatus 700 includes:

[0156] The sampling module 710 is used to sample multiple video frames from the video to be detected as sampled video frames;

[0157] The first determining module 720 is used to determine the corresponding depth image for each sampled video frame; the depth image includes the depth value of each pixel in the corresponding sampled video frame;

[0158] The second determining module 730 is used to determine candidate pixels from the sampled video frame based on the depth image of the sampled video frame for each sampled video frame.

[0159] The matching module 740 is used to determine the matching region that matches the candidate pixel from the related sampled video frames based on the depth image of the sampled video frame; the related sampled video frames are the sampled video frames that are adjacent to the sampled video frame and located after the sampled video frame.

[0160] The result determination module 750 is used to determine the flicker analysis result of the video to be detected based on the difference between each candidate pixel and its matching region in each sampled video frame.

[0161] Optionally, the matching module 740 is further configured to determine a first matching distance based on the depth value of the candidate pixel; determine the matching region corresponding to the candidate pixel from the relevant sampled video frame based on the position of the candidate pixel in the sampled video frame and the matching distance; and determine the size of the matching region based on the first matching distance.

[0162] Optionally, the video to be detected is a video viewed from the perspective of the target object; the matching module 740 is further configured to determine the offset distance, offset direction, and second matching distance of the candidate pixel based on the depth value of the candidate pixel and the motion information of the target object; and to determine the matching region corresponding to the candidate pixel from the relevant sampled video frames based on the position of the candidate pixel in the sampled video frame, the second matching distance, the offset direction, and the offset distance; the size of the matching region is determined based on the second matching distance.

[0163] Optionally, the second determining module 730 is further configured to determine a pixel as a candidate pixel if the pixel's depth value is within a specified depth value range for each pixel in the sampled video frame.

[0164] Optionally, the result determination module 750 is further configured to, for each candidate pixel, determine the pixel flicker analysis result corresponding to each candidate pixel based on the difference between the candidate pixel and its matching region; determine the video frame flicker analysis result corresponding to each sampled video frame based on the pixel flicker analysis result of each candidate pixel in each sampled video frame; and determine the flicker analysis result of the video to be detected based on the video frame flicker analysis result corresponding to each sampled video frame.

[0165] Optionally, the result determination module 750 is further configured to determine the target pixels exhibiting flickering from each sampled video frame; and to perform video frame flicker analysis based on the distribution information of each target pixel in each sampled video frame to obtain the video frame flicker analysis result for each sampled video frame.

[0166] Optionally, the result determination module 750 is further configured to, for each sampled video frame, determine a binary image indicating each target pixel in the sampled video frame based on the distribution information of each target pixel in the sampled video frame; perform denoising processing on the binary image to obtain a denoised binary image; and perform video frame flicker analysis based on the denoised binary image to obtain the video frame flicker analysis result of the sampled video frame.

[0167] Optionally, the result determination module 750 is further configured to perform connected component analysis on the binary image to obtain an intermediate binary image including multiple connected components; the binary pixels in the binary image that indicate each target pixel are located in each connected component; and to delete connected components in the intermediate binary image whose area is smaller than a preset area to obtain a denoised binary image.

[0168] Optionally, the result determination module 750 is further configured to determine the candidate pixel and the pixel difference between each pixel in the matching region of the candidate pixel; based on the pixel difference, determine the target pixel difference corresponding to the candidate pixel; if the target pixel difference is less than the pixel difference threshold, obtain the pixel flicker analysis result that the candidate pixel does not have flickering phenomenon.

[0169] Optionally, the video to be detected is a video taken from the perspective of the target object when the target object moves in the target scene; the device also includes a video acquisition module for determining at least one target point from the target scene; for each target point, a video taken from the perspective of the target object when the target object moves from the target point as the starting point of movement and according to the target movement mode is acquired, which is used as a video to be detected for the target point and the target movement mode.

[0170] Optionally, the video acquisition module is also used to acquire the movement path of the target object during the movement of the target object in the target scene; and to sample path points from the movement path at specified distance intervals to obtain at least one target point.

[0171] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0172] Figure 10 A structural block diagram of an electronic device for performing a flicker analysis method for a video according to an embodiment of this application is shown. The electronic device may be... Figure 1 The terminal (110) or server (120), etc., should be noted. Figure 10 The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0173] like Figure 10 As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203. The RAM 1203 also stores various programs and data required for system operation. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An Input / Output (I / O) interface 1205 is also connected to the bus 1204.

[0174] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1210 as needed so that computer programs read from them can be installed into storage section 1208 as needed.

[0175] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by central processing unit (CPU) 1201, it performs various functions defined in the system of this application.

[0176] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0178] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0179] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0180] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the electronic device to perform the methods of any of the above embodiments.

[0181] In the embodiments of this application, the terms "module" or "unit" refer to a part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (e.g., processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that functions as a whole.

[0182] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0183] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause an electronic device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0184] Other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A scintillation analysis method, characterized in that, The method includes: Multiple video frames are sampled from the video to be detected as sampled video frames; For each of the sampled video frames, a corresponding depth image is determined; the depth image includes the depth value of each pixel in the corresponding sampled video frame. For each of the sampled video frames, candidate pixels are determined from the sampled video frames based on the depth image of the sampled video frames; Based on the depth image of the sampled video frame, a matching region matching the candidate pixel is determined from the related sampled video frames of the sampled video frame; the related sampled video frames are those adjacent to the sampled video frame and located after the sampled video frame. Based on the differences between each candidate pixel and its matching region in each sampled video frame, the flicker analysis result of the video to be detected is determined.

2. The method according to claim 1, characterized in that, The step of determining a matching region that matches the candidate pixel from related sampled video frames based on the depth image of the sampled video frame includes: Based on the depth values ​​of the candidate pixels, a first matching distance is determined; Based on the position of the candidate pixel in the sampled video frame and the matching distance, a matching region corresponding to the candidate pixel is determined from the relevant sampled video frame; the size of the matching region is determined based on the first matching distance.

3. The method according to claim 1, characterized in that, The video to be detected is a video viewed from the perspective of the target object. The step of determining a matching region that matches the candidate pixel from related sampled video frames based on the depth image of the sampled video frame includes: Based on the depth value of the candidate pixel and the motion information of the target object, the offset distance, offset direction, and second matching distance of the candidate pixel are determined; Based on the position of the candidate pixel in the sampled video frame, the second matching distance, the offset direction, and the offset distance, a matching region corresponding to the candidate pixel is determined from the relevant sampled video frame; the size of the matching region is determined based on the second matching distance.

4. The method according to any one of claims 1-3, characterized in that, For each of the sampled video frames, determining candidate pixels from the sampled video frames based on the depth image of the sampled video frames includes: For each pixel in the sampled video frame, if the depth value of the pixel is within a specified depth value range, the pixel is determined as a candidate pixel.

5. The method according to claim 1, characterized in that, The step of determining the flicker analysis result of the video to be detected based on the difference between each candidate pixel and its matched region in each sampled video frame includes: For each candidate pixel, based on the difference between the candidate pixel and its matching region, determine the pixel flicker analysis result corresponding to each candidate pixel; Based on the pixel flicker analysis results of each candidate pixel in each sampled video frame, the video frame flicker analysis results corresponding to each sampled video frame are determined. Based on the flicker analysis results of each of the sampled video frames, the flicker analysis results of the video to be detected are determined.

6. The method according to claim 5, characterized in that, The step of determining the video frame flicker analysis result corresponding to each sampled video frame based on the pixel flicker analysis results of each candidate pixel in each sampled video frame includes: Identify the target pixel exhibiting flickering from each of the sampled video frames; Based on the distribution information of each target pixel in each sampled video frame, video frame flicker analysis is performed to obtain the video frame flicker analysis result for each sampled video frame. The pixel flicker analysis result indicates whether there is flickering phenomenon in the corresponding candidate pixel.

7. The method according to claim 6, characterized in that, The step of performing video frame flicker analysis based on the distribution information of each target pixel in each sampled video frame to obtain the video frame flicker analysis result for each sampled video frame includes: For each of the sampled video frames, a binary image indicating each target pixel in the sampled video frame is determined based on the distribution information of each target pixel in the sampled video frame; The binary image is denoised to obtain a denoised binary image; Based on the denoised binary image, video frame flicker analysis is performed to obtain the video frame flicker analysis results of the sampled video frame.

8. The method according to claim 7, characterized in that, The step of denoising the binary image to obtain a denoised binary image includes: Connectivity analysis is performed on the binary image to obtain an intermediate binary image comprising multiple connected components; the binary pixels in the binary image that indicate each target pixel are located in each connected component. The connected components with areas smaller than a preset area in the intermediate binary image are deleted to obtain the denoised binary image.

9. The method according to claim 5, characterized in that, For each candidate pixel, determining the pixel flicker analysis result corresponding to each candidate pixel based on the difference between the candidate pixel and its matched region includes: Determine the candidate pixels and the pixel differences between each pixel within the matching region that the candidate pixels match; Based on the pixel differences, the target pixel differences corresponding to the candidate pixels are determined; If the difference between the target pixels is less than the pixel difference threshold, the pixel flicker analysis result is obtained, indicating that the candidate pixels do not exhibit flickering.

10. The method according to claim 1, characterized in that, The video to be detected is a video taken from the perspective of the target object as it moves within the target scene. Before constructing a video frame sequence by sampling multiple sampled video frames from the video to be detected, the method further includes: Determine at least one target point from the target scene; For each target point, a video is obtained showing the target object moving from the target point as the starting point and according to the target movement method, with the target object's perspective as the observation perspective. This video is used as a test video for the target point and the target movement method.

11. The method according to claim 10, characterized in that, Determining at least one target point from the target scene includes: During the movement of the target object in the target scene, the movement path of the target object is obtained; The path points are sampled from the movement path at specified distance intervals to obtain the at least one target point.

12. A video flicker analysis device, characterized in that, The device includes: The sampling module is used to sample multiple video frames from the video to be detected as sampled video frames. The first determining module is used to determine a corresponding depth image for each of the sampled video frames; the depth image includes the depth value of each pixel in the corresponding sampled video frame; The second determining module is used to determine candidate pixels from the sampled video frame based on the depth image of the sampled video frame for each sampled video frame; A matching module is used to determine a matching region that matches the candidate pixel from related sampled video frames based on the depth image of the sampled video frame; the related sampled video frames are sampled video frames that are adjacent to the sampled video frame and located after the sampled video frame; The result determination module is used to determine the flicker analysis result of the video to be detected based on the difference between each candidate pixel and its matching region in each sampled video frame.

13. An electronic device, characterized in that, include: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by a processor, implement the method as described in any one of claims 1-11.

15. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the method of any one of claims 1-11.

Images