Video frame detection method and device, electronic equipment and storage medium

Abstract

The invention provides a video frame detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-played first video on a to-be-detected player, the first video comprising a first region of first pixel information and a second region of second pixel information in a multi-frame image, the player is controlled to play the first video, the acquisition device is driven to acquire the multi-frame images in the played first video so as to obtain a target image, and the target image is acquired according to the pixel information of the plurality of first areas in the target image. According to the method, the frame loss and / or multi-frame condition of the first video played by the player is detected, and the video frame playing result of the player is detected to judge whether the abnormity of frame loss or multi-frame exists, so that the effectiveness of the video frame detection and processing method in the system of the player is identified.

Description

Technical Field

[0001] This application relates to the field of optical technologies, and in particular, to a video frame detection method, apparatus, electronic device, and storage medium. Background Art

[0002] During video transmission or processing, due to reasons such as network latency, bandwidth limitations, hardware failures, and requirements for smoothness, problems such as video frame loss or duplication may occur. In related technologies, through video frame detection within a video playback system, abnormal situations of video frames can be detected in a timely manner, and corresponding measures can be taken to correct the abnormalities of the video frames to ensure the quality of video transmission or processing.

[0003] However, the display effect after video frame abnormality detection and processing cannot be evaluated. Summary of the Invention

[0004] This application aims to solve at least one of the technical problems in the related technologies to some extent.

[0005] To this end, this application provides a video frame detection method, apparatus, electronic device, and storage medium. By detecting the result of a player playing video frames, it is determined whether there are abnormalities such as frame loss or duplicate frames, so as to identify the effectiveness of the video frame detection and processing method inside the player's system.

[0006] An embodiment of one aspect of this application provides a video frame detection method, including:

[0007] Obtain a first video to be played on a player to be detected; wherein, among the multiple frames of images included in the first video, there are a first region including first pixel information and a second region including second pixel information; there is no overlapping region among the first regions included in the multiple frames of images;

[0008] Control the player to play the first video, and drive an acquisition device to acquire multiple frames of images in the played first video to obtain a target image;

[0009] Detect the frame loss and / or multiple frames situation of the player playing the first video according to the pixel information of multiple first regions in the target image.

[0010] An embodiment of another aspect of this application provides a video frame detection apparatus, including:

[0011] An acquisition module, configured to obtain a first video to be played on a player to be detected; wherein, among the multiple frames of images included in the first video, there are a first region including first pixel information and a second region including second pixel information; there is no overlapping region among the first regions included in the multiple frames of images;

[0012] A control module, configured to control the player to play the first video and drive an acquisition device to acquire multiple frames of images in the played first video to obtain a target image;

[0013] A detection module, configured to detect frame loss and / or multiple frames of the player playing the first video according to pixel information of multiple first regions in the target image. Another embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the foregoing aspect is implemented.

[0014] Another embodiment of the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the foregoing aspect is implemented.

[0015] Another embodiment of the present application provides a computer program product, on which a computer program is stored. When the program is executed by a processor, the method described in the foregoing aspect is implemented.

[0016] The video frame detection method, device, electronic device, and storage medium provided by the present application obtain a first video to be played on a player to be detected. The first video includes a first region with first pixel information and a second region with second pixel information in multiple frames of images, and there is no overlapping region between the first regions included in the multiple frames of images. Control the player to play the first video, and drive an acquisition device to acquire multiple frames of images in the played first video to obtain a target image. Detect frame loss and / or multiple frames of the player playing the first video according to pixel information of multiple first regions in the target image. By detecting the result of the player playing video frames, it is determined whether there is an abnormality of frame loss or duplicate frames, so as to identify the effectiveness of the video frame detection and processing method inside the player's system.

[0017] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Description of the Drawings

[0018] The above and / or additional aspects and advantages of the present application will become apparent and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0019] Figure 1 is a flowchart of a video frame detection method provided by an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of a target image provided by an embodiment of the present application;

[0021] Figure 3 It is a schematic flowchart of another video frame detection method provided by an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the position of a first area provided by an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of a test system provided by an embodiment of the present application;

[0024] Figure 6 It is a schematic structural diagram of a video frame detection device provided by an embodiment of the present application;

[0025] Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0026] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation to the present application.

[0027] The video frame detection method, device, electronic device, and storage medium of the embodiments of the present application will be described below with reference to the accompanying drawings.

[0028] Figure 1 It is a schematic flowchart of a video frame detection method provided by an embodiment of the present application.

[0029] The execution subject of the video frame detection method in the embodiment of the present application is a video frame detection device, which can be set in an electronic device. The electronic device can be a smart phone, a laptop computer, a smart wearable device with a display screen, etc., which are not limited in this embodiment.

[0030] As Figure 1 shown, the method may include the following steps:

[0031] Step 101, obtain a first video to be played on a player to be detected.

[0032] Among them, the player to be detected may refer to software that can play video files stored in digital signal form, such as a playback software, or an electronic device product with the function of playing video files, such as a mobile phone, a computer, etc., which are not limited in this embodiment.

[0033] In an embodiment of the present application, the first video includes multiple frames of images, referred to as the first number of frames. Each frame of the images in the first video includes a first region with first pixel information of a set size and a second region with second pixel information, where the first pixel information and the second pixel information are different, and the pixel information includes pixel values. Among them, the positions of the first regions in different frames of images are different, and there is no overlapping region between the first regions in different frames of images.

[0034] It should be understood that since the positions of the first regions in each frame of the image are different, during the playback of the first video, the first regions in each frame present an animation effect of sliding on the screen.

[0035] Step 102: Control the player to play the first video, and drive the acquisition device to acquire multiple frames of images in the played first video to obtain a target image.

[0036] In an embodiment of the present application, control the player to play the first video, and during the playback of the first video, drive the acquisition device to capture each frame of the images in the played first video to obtain a target image. Among them, the acquisition device can acquire based on a long exposure duration or a short exposure duration. The following will be described separately.

[0037] In an implementation manner of an embodiment of the present application, the acquisition device uses a long exposure duration for shooting, that is, drives the acquisition device to acquire multiple frames of images in the played first video with a long exposure duration to obtain a frame of target image. Among them, within the long exposure duration, multiple frames of images in the first video can be played out, so that the sensor of the acquisition device superimposes the pixel values at the corresponding pixel positions based on the multiple frames of images and outputs a frame of target image. That is to say, the pixel information of each pixel point in the target image is obtained by the sensor through acquisition and superposition within the long exposure duration. Therefore, in the case of frame loss or multiple frames in the multiple frames of images, when the sensor performs pixel superposition based on the acquired images, the superposition results at different positions will also change. Thus, based on the pixel information in the output target image, it can be identified whether there is frame loss or multiple frames in the multiple frames of images in the first video. Among them, as an example, the first video includes 120 frames, and the frame rate of the first video is 120 frames per second. Therefore, setting the long exposure duration to be greater than or equal to 1 second can complete the acquisition of 120 frames of images within the long exposure duration.

[0038] In another implementation of the embodiments of the present application, the acquisition device captures images with a short exposure duration. Within the short exposure duration, a frame of image in the first video can be acquired. Through multiple captures, multiple candidate images are acquired. Then, the pixel values at the corresponding pixel positions in the multiple acquired candidate images are superimposed to obtain a composite image. The composite image is used as the target image, and then recognition is performed based on the target image. Therefore, the pixel information of each pixel point in the target image is also obtained by superimposition. Among them, multiple candidate images obtained by capturing are synthesized to obtain a frame of target image, that is, the multiple candidate images are aligned pixel by pixel, and for each pixel point, the pixel values of this pixel point are superimposed. The superimposed pixel value is used as the pixel value of the corresponding pixel point in the synthesized frame of target image. Similarly, the pixel values of all pixel points in the synthesized frame of target image can be obtained. Thus, based on the pixel information in the target image, it can be recognized whether there are missing frames or multiple frames in the multiple frames of images in the first video. It should be understood that when capturing based on a short exposure duration, the sampling frequency of the acquisition device needs to be greater than or equal to the frame rate of the first video to ensure that each frame played in the first video can be acquired. For example, if the first video includes 120 frames and the frame rate of the first video is 120 frames per second, then when the acquisition device acquires images based on a short exposure duration, the acquisition frequency needs to be greater than or equal to 120 Hz.

[0039] Among them, multiple candidate images are obtained by capturing. For the sake of distinction, it is called the second number of frames. Among them, the second number of frames and the first number of frames may be the same or different. In one scenario, during the playback of the first video, due to device or transmission anomalies, or to match the decoding ability, frame loss occurs, then the second number of frames is less than the first number of frames; in the second scenario, during the playback of the first video, when the video frame rate is low, for example, when it is lower than 24 frames per second, the human eye will feel that the video is not smooth. Therefore, frame duplication processing will be performed during playback to increase the video frame rate and improve the playback effect, then the second number of frames is greater than the first number of frames; in the third scenario, during the playback of the first video, the player does not perform frame loss or frame duplication processing, then the second number of frames is equal to the first number of frames.

[0040] Step 103, according to the pixel information of multiple first regions in the target image, detect the frame loss and / or frame duplication situation of the player playing the first video.

[0041] Among them, the pixel information of each pixel point in each first region is the same and is all the first pixel information; the pixel information of each pixel point in the second region is also the same and is all the second pixel information. Among them, the pixel information includes pixel values.

[0042] In the embodiments of the present application, based on the pixel information of multiple first regions in the target image, it can be used to indicate whether there is a frame loss or multiple frames. This is because the pixel information of each pixel point in the target image is obtained by superposition. After superposition, there are changes in YUV or RGB for the pixel points, resulting in changes in the pixel information.

[0043] For the target image, in the case where it is synthesized from multiple frames of images collected by short exposure, among them, for the pixel information of each first region in the target image, as an implementation manner, a reference image including this first region is determined, the target regions corresponding to the position of the first region in other frame images other than the reference image in multiple frames of images of the first video are determined, and pixel synthesis is performed on the first region in the reference image and the target regions in other frame images to obtain the pixel information of this first region in the target image. Similarly, the pixel information of other first regions can be obtained.

[0044] In one scenario, in response to the pixel information of at least one first region in the target image being the third pixel information, it is determined that there is a frame loss when the player plays the first video. Among them, the third pixel information is obtained by superposing multiple second pixel informations. That is to say, for the third pixel information (third pixel value) of each pixel point, the second pixel information corresponding to this pixel point in each candidate image for superposition is superposed. Since there is no overlap in the first regions in multiple frames of images in the first video, if any frame is lost, for the sake of convenience, the first region in the lost frame is called the reference region, the pixel information of the reference region in any lost frame is the first pixel information, and the pixel information of the reference region in the remaining image frames in the first video is the second pixel information. Therefore, after any such frame is lost, the pixel information of the reference region in the remaining image frames in the first video is the second pixel information, and thus the pixel information of each pixel point in the reference region is the third pixel information obtained by superposing the second pixel information and the second pixel information.

[0045] In another scenario, in response to the pixel information of at least one first region in the target image being the fourth pixel information, it is determined that there are multiple frames when the player plays the first video. Among them, the fourth pixel information is obtained by superposing multiple first pixel informations. That is to say, for the fourth pixel information (fourth pixel value) of each pixel point, the first pixel information corresponding to this pixel point in each candidate image for superposition is superposed. Since there is no overlap in the first regions in multiple frames of images in the first video, the pixel information in the first region is the first pixel information. If any frame is played repeatedly, the pixel information of the first region in the repeatedly played image is the fourth pixel information obtained by superposing multiple first pixel informations.

[0046] In yet another scenario, in response to the pixel information of at least one first region in the target image being the third pixel information and the pixel information of at least one first region being the fourth pixel information, it is determined that there are frame loss and multiple-frame situations when the player plays the first video.

[0047] As an example, the pixel information is the pixel value, and the image is a black-and-white image. Among them, the pixel value of each pixel point in the first region of the image is 1 and is displayed as white, and the pixel value of the pixel points outside the second region in the image is 0 and is displayed as black. If there is frame loss, then in the target image, there is at least one pixel point in the first region whose pixel value is not 0 or 1, but the third pixel value; if there is no frame loss and multiple-frame situation, in the target image, the pixel information of each pixel point is obtained by superimposing the first pixel value and the second pixel value, which is called the fifth pixel value; if there is a multiple-frame situation, in the synthesized frame image, there is at least one pixel point in the first region whose pixel value is the fourth pixel value obtained by superimposing multiple first pixel values. That is to say, based on the different pixel values of the pixel points in the target image, it can be recognized whether there is frame loss and / or multiple frames during the video playback process.

[0048] As an example, as Figure 2 shown, A in the figure indicates the display effect of each first region in the target image in the case of no frame loss and no multiple frames, and B in the figure indicates the display effect of each first region in the target image in the case of frame loss and multiple frames. Since in the scenario of multiple frames or frame loss, the pixel values of the pixel points for pixel synthesis are different, the pixel values of the synthesized pixel points are also different. For the pixel points with different synthesized pixel values, the displayed colors are different. Therefore, through the pixel values of the pixel points in the target image, it can be recognized whether there is an abnormal frame processing during the playback of the player, that is, there is frame loss or multiple frames, and further, the effectiveness of the video frame detection and processing method inside the player's system can be recognized.

[0049] For the target image, when the target image is obtained by pixel superposition inside the sensor of the acquisition device based on a long exposure time, the principle is the same as that of the synthesized image, and it will not be elaborated in this embodiment.

[0050] It should be understood that if the video frame detection and processing method inside the player's system is effective, then there will be no frame loss or multiple-frame situation. If there is frame loss or multiple-frame situation, it may be that the effectiveness of the video frame detection and processing method inside the player's system is poor.

[0051] In the video frame detection method according to the embodiments of the present application, a first video to be played on a player to be detected is obtained. The first video includes a first region with first pixel information and a second region with second pixel information in each frame image included therein, and there is no overlapping region between the first regions included in multiple frame images. The player is controlled to play the first video, and an acquisition device is driven to acquire multiple frame images in the played first video to obtain a target image. According to the pixel information of multiple first regions in the target image, the frame dropping and / or multiple frame situations of the player playing the first video are detected. By detecting the result of the player playing video frames, it is determined whether there are abnormal situations of frame dropping or frame duplication, so as to identify the effectiveness of the video frame detection and processing method inside the player system.

[0052] Based on the above embodiments, Figure 3 is a schematic flowchart of another video frame detection method provided by the embodiments of the present application, as Figure 3 shown. This method includes the following steps:

[0053] Step 301, obtain the refresh rate, resolution, and movement rule of the first region of the player.

[0054] Among them, the resolution refers to the number of pixels that the display corresponding to the player can display. For example, it is n*m, such as 1024*768. The refresh rate of the player refers to the number of times the player refreshes per second, that is, the number of frames of the picture played per second. The movement rule of the first region is used to determine the position of the first region in each frame image, including the movement direction and the movement distance.

[0055] Step 302, generate images with the first number of frames that matches the refresh rate of the screen.

[0056] In one implementation manner of the embodiments of the present application, the first number of frames is equal to the refresh rate. For example, if the refresh rate is R, then the first number of frames F = R.

[0057] Step 303, determine the pixel size of the first region and the position of the first region in each frame image of the first number of frames according to the movement rule, the first number of frames, and the resolution.

[0058] Among them, as one implementation manner, the first number of frames is the same as the refresh rate of the screen.

[0059] The first region can be of any shape. For example, it can be a rectangle, such as a rectangle or a square, or it can be a circle, etc. It is not limited in this embodiment.

[0060] In one implementation of the embodiment of the present application, the pixel size of the first region is determined according to the moving direction, the moving distance, the first number of frames, and the resolution. In one scenario, the first region is rectangular. As one implementation, the first region is rectangular, the moving direction includes horizontal sliding and / or vertical sliding, and a first expression corresponding to the moving rule is obtained, that is, a first relational expression corresponding to the moving direction and the moving distance is obtained. The first number of frames and the resolution are substituted into the first relational expression to determine the set size of the first region, where the first relational expression includes:

[0061] The product between the integer part of and the integer part of is greater than or equal to f;

[0062] wherein, the width of the first region is a, k1 is the distance of horizontal sliding of the first region, the length is b, and k2 is the distance of vertical sliding of the first region; the resolution of the screen of the player is n*m, n is the number of pixel points in the horizontal direction of the screen, m is the number of pixel points in the vertical direction of the screen, and the first number of frames is f.

[0063] In one scenario, the first region first slides vertically, and the vertical sliding distance is the length of the first region, and then slides horizontally, and the horizontal sliding distance is the width of the first region, or, the first region first slides horizontally, and the horizontal sliding distance is the width of the first region, and then slides vertically, and the vertical sliding distance is the length of the first region. Then the corresponding first expression is:

[0064] The product between the integer part of n / a and the integer part of m / b is greater than or equal to f, that is

[0065] wherein, the length of the first region is a and the width is b; the resolution of the screen of the player is n*m, n is the number of pixel points in the horizontal direction of the screen, m is the number of pixel points in the vertical direction of the screen, and the first number of frames included in the first video is f.

[0066] Among them, since the resolution of the screen indicates the display resolution of the playable video frames, that is, it can indicate the size of the image frames, the integer part of n / a indicates the first quantity of the first region included in the horizontal direction of the screen, the integer part of m / b indicates the second quantity of the first region included in the vertical direction of the screen, and the product of the integer part values of the first quantity and the second quantity needs to be greater than the first number of frames f to be played, so that the positions of the first region in each frame are different and there is no overlapping region.

[0067] As a second implementation method, set the rule that when the first video is played frame by frame, the display effect of the first area is to slide along the diagonal of the screen. First, calculate the size of the screen diagonal based on n and m, and then divide the rounded-down size of the diagonal into f equal parts. If the first area is a rectangle, the rounded-down size of each part is used as the size of the diagonal of the first area, and then the length and width of the rectangle are determined; if the first area is a circle, the rounded-down size of each part is used as the diameter of the first area, that is, the size of the first area is determined.

[0068] Furthermore, according to the moving direction, moving distance, and pixel size of the first area, determine the position of the first area with the pixel size in each frame of the first number of frames.

[0069] In the embodiment of the present application, each frame of the first video has a first area. The positions of the first area in each frame are different and do not overlap with each other. The position of the first area can be used to identify each frame for subsequent identification in the target image of which frame is lost or which frame is repeated.

[0070] As an example, Figure 4 is a schematic diagram of the position of a first area provided by the embodiment of the present application. As Figure 4 shown, the first video includes f frames, the first area is a rectangle, and the size of the first area is b*a, where the length is b pixel points and the width is a pixel points. Figure 4 shows the positions of the first area in each frame, and there is no overlapping area between them. Figure 4 schematically shows the positions of the first area in the first frame, the second frame, the third frame, and the fth frame. The slide is in the upper left corner of the video frame in the first frame. From the first frame to the second frame, the first area slides down by b pixel positions. That is, the display effect of the first area when played frame by frame is to slide first in the vertical direction, that is, vertically downward, and then in the horizontal direction, that is, horizontally to the right, until the last frame, when it appears in the lower right corner of the screen. According to the moving direction and moving distance of the first area, and the pixel size of the first area, the position of the first area with the pixel size can be determined in each frame image of the first number of frames.

[0071] Step 304, generate the first video according to the image of the first number of frames that determines the pixel size and position of the first area.

[0072] In the embodiment of the present application, in the image of the first number of frames, based on the moving direction and moving distance, the pixel size and position of the first area in each frame image are determined, so that each frame image has a display order corresponding to the moving direction. Based on the display order, the first video is generated.

[0073] For example, as Figure 4As shown, the first frame to the f-th frame are determined based on the moving direction. When generating the first video, the sorting of the frames in the first video is the first frame to the f-th frame.

[0074] Step 305: Obtain the first video to be played on the player to be detected.

[0075] As an example, Figure 5 is a schematic diagram of a test system provided by an embodiment of the present application, which is used to implement the video frame detection method of the present application. As Figure 5 shown, the system includes a player to be detected, a collection device, and a computer PC. A multi-dimensional adjustment base for the display to be detected and the detector is respectively set up. Among them, the player to be detected, that is, the display to be detected, and the collection device is the detector. The display to be detected and the detector are respectively connected to the computer and respectively connected to the synchronization device. Among them, the computer is used to control the display to be detected to display video data, and control the detector to perform image collection according to the set exposure duration and sampling frequency. The synchronization device is used to control the display to be detected and the detector to work synchronously, that is, when the display to be detected plays the first video, synchronously control the detector to perform collection, so as to make the play and collection synchronous, avoid missed frames in collection, and improve the accuracy of collection.

[0076] Step 306: Control the player to play the first video, and drive the collection device to collect multiple frames of images in the played first video to obtain the target image.

[0077] Step 307: Detect the frame loss and / or multiple frames of the player playing the first video according to the pixel information of multiple first regions in the target image.

[0078] Among them, steps 305 to 307 can refer to the explanations in the foregoing embodiments, with the same principle, which will not be elaborated here.

[0079] In the video frame detection method of the embodiment of the present application, images with the first number of frames matching the screen refresh rate of the generated and tested player are generated, so that when the screen refresh rate and the video frame number are at the same level, the image can be made more stable and the video can be made more fluent. Based on the screen refresh rate, resolution, and the moving rule of the first region, the pixel size of the first region is determined, and the position of the first region in each frame of the first number of frames of images is determined, so that the position of the first region in each frame is different and there is no overlapping region, so as to facilitate identifying whether there is frame loss or multiple frames after image collection and synthesis based on the collected images, thereby identifying the flow situation of the player when playing the first video, and the effectiveness of the video frame detection and processing method inside the player system.

[0080] To implement the above embodiments, the embodiment of the present application also proposes a video frame detection device.

[0081] Figure 6 The structural schematic diagram of a video frame detection device provided by an embodiment of the present application.

[0082] As Figure 6 shown, the device may include:

[0083] An acquisition module 61, configured to acquire a first video to be played on a player to be detected; wherein, among multiple frames of images included in the first video, there are a first region including first pixel information and a second region including second pixel information; there is no overlapping region among the first regions included in multiple frames of images;

[0084] A control module 62, configured to control the player to play the first video, and drive an acquisition device to acquire multiple frames of images in the played first video to obtain a target image;

[0085] A detection module 63, configured to detect frame loss and / or multiple frames of the player playing the first video according to the pixel information of multiple first regions in the target image.

[0086] Further, in an implementation manner of an embodiment of the present application, the device further includes a generation module, configured to:

[0087] Obtain the screen refresh rate, resolution of the player, and the movement rule of the first region;

[0088] Generate images with a first number of frames matching the screen refresh rate;

[0089] Determine the pixel size of the first region according to the movement rule, the first number of frames, and the resolution, and determine the position of the first region in each frame of the first number of frames of images;

[0090] Generate the first video according to the images of the first number of frames determining the pixel size and position of the first region.

[0091] In an implementation manner of an embodiment of the present application, the movement rule includes a movement direction and a movement distance, and the generation module is further configured to:

[0092] Determine the pixel size of the first region according to the movement direction, the movement distance, the first number of frames, and the resolution;

[0093] Determine the position of the first region with the pixel size in each frame of the first number of frames according to the movement direction, the movement distance, and the pixel size of the first region.

[0094] In an implementation manner of the embodiment of the present application, the first area is rectangular, and the moving direction includes horizontal sliding and / or vertical sliding. The generating module is further configured to:

[0095] Obtain a first relational expression corresponding to the moving direction and the moving distance;

[0096] Substitute the first number of frames and the resolution into the first relational expression to determine the pixel size of the first area;

[0097] Wherein, the first relational expression includes:

[0098] The product between the integer part of and the integer part of is greater than or equal to f;

[0099] Wherein, a is the width of the first area, k1 is the horizontal moving distance of the first area, the length is b, and k2 is the vertical moving distance of the first area; the resolution of the screen of the player is n*m, and f is the first number of frames.

[0100] In an implementation manner of the embodiment of the present application, the control module 62 is further configured to:

[0101] Drive the acquisition device to use a long exposure duration to capture multiple frames of images in the played first video to obtain the target image.

[0102] In an implementation manner of the embodiment of the present application, the control module 62 is further configured to:

[0103] Drive the acquisition device to use a short exposure duration to capture multiple frames of images in the played first video to obtain multiple candidate images;

[0104] Synthesize corresponding pixels in the multiple candidate images captured to obtain a synthesized image;

[0105] Use the synthesized image as the target image. In an implementation manner of the embodiment of the present application, the detection module 63 is further configured to:

[0106] In response to the pixel information of at least one first area in the target image being third pixel information, determine that there is a frame loss situation when the player plays the first video; wherein, the third pixel information is obtained by superimposing a plurality of the second pixel information.

[0107] In an implementation manner of the embodiment of the present application, the detection module 63 is further configured to:

[0108] In response to the pixel information of at least one first region in the target image being fourth pixel information, it is determined that there are multiple frames in the player playing the first video; wherein, the fourth pixel information is obtained by superimposing a plurality of the first pixel information.

[0109] It should be noted that the foregoing explanation of the method embodiment also applies to the device of this embodiment, and will not be repeated here.

[0110] In the video frame detection device of the embodiment of the present application, a first video to be played on a player to be detected is obtained, wherein each frame image included in the first video includes a first region with first pixel information and a second region with second pixel information, and there is no overlapping region between the first regions included in multiple frame images. The player is controlled to play the first video, and an acquisition device is driven to acquire multiple frame images in the played first video to obtain a target image. According to the pixel information of multiple first regions in the target image, the situation of frame loss and / or multiple frames in the player playing the first video is detected. By detecting the result of the player playing video frames, it is determined whether there is an abnormality of frame loss or repeated frames, so as to identify the effectiveness of the video frame detection and processing method inside the player's system.

[0111] To implement the above embodiment, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the foregoing method embodiment is implemented.

[0112] To implement the above embodiment, the present application also proposes a non-temporary computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in the foregoing method embodiment is implemented.

[0113] To implement the above embodiment, the present application also proposes a computer program product, on which a computer program is stored. When the computer program is executed by a processor, the method described in the foregoing method embodiment is implemented.

[0114] Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0115] Refer to Figure 7 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0116] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-described methods. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0117] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks.

[0118] The power component 806 provides power to the various components of the electronic device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0119] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each of the front camera and the rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0120] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0121] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include but are not limited to: a home button, a volume button, a power button, and a lock button.

[0122] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the electronic device 800. For example, the sensor component 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800. The sensor component 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0123] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0124] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0125] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by a processor 820 of the electronic device 800 to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0126] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0127] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0128] Any process or method description in the flowchart or described in other ways herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in an order opposite to that shown or discussed, according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0129] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0130] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with suitable combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0131] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0132] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0133] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A video frame detection method, characterized in that, Including: Obtain a first video to be played on a player to be detected; wherein, among multiple frames of images included in the first video, there are a first area with first pixel information and a second area with second pixel information; there is no overlapping area among the first areas included in the multiple frames of images; Control the player to play the first video, and drive an acquisition device to acquire multiple frames of images in the played first video to obtain a target image; Detect the frame dropping and / or multiple-frame situation of the player playing the first video according to the pixel information of multiple first areas in the target image.

2. The method according to claim 1, wherein Before obtaining the first video to be played on the player to be detected, it includes: Obtain the screen refresh rate, resolution of the player, and the movement rule of the first area; Generate images with a first number of frames that match the screen refresh rate; Determine the pixel size of the first area according to the movement rule, the first number of frames, and the resolution, and determine the position of the first area in each frame of the first number of frames of images; Generate the first video according to the images with the first number of frames for which the pixel size and position of the first area are determined.

3. The method according to claim 2, characterized in that The movement rule includes a movement direction and a movement distance. The determining the pixel size of the first area according to the movement rule, the first number of frames, and the resolution, and determining the position of the first area in each frame of the first number of frames of images includes: Determine the pixel size of the first area according to the movement direction, the movement distance, the first number of frames, and the resolution; Determine the position of the first area with the pixel size in each frame of the first number of frames of images according to the movement direction, the movement distance, and the pixel size of the first area.

4. The method according to claim 1, characterized in that The driving the acquisition device to acquire multiple frames of images in the played first video to obtain a target image includes: Drive the acquisition device to acquire multiple frames of images in the played first video with a long exposure duration to obtain the target image.

5. The method according to claim 1, wherein The driving the acquisition device to acquire multiple frames of images in the played first video to obtain a target image includes: Drive the acquisition device to acquire multiple frames of candidate images for each frame of the first video played with a short exposure duration; Synthesize corresponding pixels in the multiple frames of candidate images acquired to obtain a synthesized image; Use the synthesized image as the target image.

6. The method according to any one of claims 1-5, characterized in that, The detecting the frame dropping and / or multiple-frame situation of the player playing the first video according to the pixel information of multiple first areas in the target image includes: In response to the pixel information of at least one first area in the target image being third pixel information, determine that there is a frame dropping situation when the player plays the first video; wherein, the third pixel information is obtained by superimposing multiple pieces of the second pixel information.

7. The method according to any one of claims 1-5, characterized in that, The detecting the frame dropping and / or multiple-frame situation of the player playing the first video according to the pixel information of multiple first areas in the target image includes: In response to the pixel information of at least one first region in the target image being fourth pixel information, it is determined that there are multiple frames when the player plays the first video; wherein the fourth pixel information is obtained by superimposing a plurality of the first pixel information.

8. A video frame detection device, characterized in that, It includes: An acquisition module, configured to acquire a first video to be played on a player to be detected; wherein, among the multiple frames of images included in the first video, there are a first region including first pixel information and a second region including second pixel information; there is no overlapping region among the first regions included in the multiple frames of images; A control module, configured to control the player to play the first video, and drive an acquisition device to acquire multiple frames of images in the played first video to obtain a target image; A detection module, configured to detect the frame loss and / or multiple-frame situation of the player playing the first video according to the pixel information of multiple first regions in the target image.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in any one of claims 1-7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method described in any one of claims 1-7 is implemented.

Images