Frame rate detection method, device, computer equipment, storage medium and product

Abstract

The embodiment of the present application discloses a frame rate detection method, device, computer equipment, storage medium and product. By performing status detection on each display area of ​​the terminal, the status information of each display area is obtained. A display area in the terminal is used to display a detection image block in a video frame, and the status information represents the frame loss status of the corresponding video frame; according to the status information of each display area of ​​the terminal, the number of video frames without frame loss within a preset time period is counted; the display frame rate of the terminal is calculated based on the number of video frames. In this scheme, each display area of ​​the terminal corresponds to the detection image block contained in a video frame. By performing status detection on each display area of ​​the terminal, the refresh of the video frame can be accurately detected. According to the status information of each display area, the number of video frames without frame loss can be determined, and then the display frame rate of the terminal can be determined. There is no need to compare the content between video frames through complex calculations, thereby improving the efficiency of frame rate detection.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and specifically to a frame rate detection method, apparatus, computer equipment, storage medium and product, wherein the storage medium is a computer-readable storage medium and the product is a computer program product. Background Art

[0002] When testing the display frame rate of the device under test, the video played by the device under test is usually recorded to obtain a recorded video frame sequence; adjacent frames are compared on the recorded video frame sequence. If the content of the adjacent frames is considered to be the same, then the two adjacent video frames are the same video frame displayed by the device under test, and subsequent video frames are obtained for comparison until video frames with different contents are compared. The time difference between the two video frames with different contents is the frame interval. The display frame rate of the device under test can be calculated based on the frame interval.

[0003] The time corresponding to each video frame in the recorded video frame sequence is the time when the video recording device captures the video screen played by the measured device, rather than the precise time when the measured device refreshes the video frame, resulting in inaccurate frame intervals obtained from the test, which in turn affects the low accuracy of the calculation of the display frame rate. In addition, a large amount of calculation is required to compare whether the content between each video frame is the same, and the efficiency of this frame rate detection method is low. Summary of the invention

[0004] The embodiments of the present application provide a frame rate detection method, apparatus, computer equipment, storage medium and product, which can improve the efficiency of frame rate detection.

[0005] A frame rate detection method provided in an embodiment of the present application includes:

[0006] When the terminal displays the received video frame within a preset time period, a state detection is performed on each display area of ​​the terminal to obtain state information of each display area, wherein one display area in the terminal is used to display a detected image block in one video frame, and the state information represents a frame loss state of the corresponding video frame;

[0007] According to the status information of each display area of ​​the terminal, counting the number of video frames that have not been lost within the preset time period;

[0008] A display frame rate of the terminal is calculated based on the number of video frames.

[0009] Accordingly, an embodiment of the present application further provides a frame rate detection device, including:

[0010] A state detection unit, configured to perform state detection on each display area of ​​the terminal when the terminal displays the received video frame within a preset time period, and obtain state information of each display area, wherein one display area in the terminal is used to display a detection image block in one video frame, and the state information represents a frame loss state of the corresponding video frame;

[0011] A quantity counting unit, used for counting the number of video frames without frame loss within the preset time period according to the status information of each display area of ​​the terminal;

[0012] A frame rate calculation unit is used to calculate the display frame rate of the terminal based on the number of video frames.

[0013] In one embodiment, the state detection unit includes:

[0014] An image acquisition subunit, configured to, when the terminal displays a received video frame within a preset time period, acquire a long exposure image corresponding to the video frame displayed by the terminal within the preset time period;

[0015] The detection subunit is used to perform status detection on each display area of ​​the terminal based on the long exposure image to obtain status information of each display area.

[0016] In one embodiment, the image acquisition subunit includes:

[0017] An optical signal detection module, configured to perform optical signal detection on the video picture displayed by the terminal based on the video frame through an image acquisition device within the preset time period;

[0018] The superposition processing module is used to perform superposition processing on the light signals detected within the preset time period to obtain the long exposure image.

[0019] In one embodiment, the long exposure image includes a plurality of display image blocks, one display image block corresponds to a display area of ​​the terminal, and the detection subunit includes:

[0020] A pixel value detection module, used to perform pixel value detection on the display image blocks contained in the long exposure image to obtain the pixel color value of each display image block;

[0021] A type determination module, used to determine the area type of the display image block according to the pixel color value of each display image block;

[0022] The state determination module is used to determine the state information of the corresponding display area according to the area type of the display image block.

[0023] In one embodiment, the state detection unit includes:

[0024] a change detection subunit, configured to detect light signal changes in each of the display areas through a sensor when the terminal displays a received video frame within a preset time period, and obtain a signal detection result for each of the display areas;

[0025] The state information determination subunit is used to determine the state information of each of the display areas according to the signal detection result.

[0026] In one embodiment, the state detection unit includes:

[0027] An information acquisition subunit, used to acquire initial detection information, wherein the initial detection information includes sub-detection information;

[0028] A detection object identification subunit, configured to perform detection object identification on each display area of ​​the terminal when the terminal displays the received video frame within a preset time period, and determine a target display area where the detection image block is located;

[0029] A selection processing subunit is used to select and process the sub-detection information at the position corresponding to the initial detection information according to the target display area to obtain detection information;

[0030] The information obtaining subunit is used to obtain the status information of each of the display areas according to the detection information.

[0031] In one embodiment, the state information includes target state information, the target state information indicates that the corresponding video frame is not lost, and the quantity counting unit includes:

[0032] An area quantity counting subunit, used for counting the number of areas where the state information is the display area of ​​the target state information within the preset time period;

[0033] The quantity determination subunit is used to determine the number of regions as the number of video frames without frame loss within the preset time period.

[0034] Correspondingly, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any frame rate detection method provided in the embodiment of the present application.

[0035] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium, which is used to store a computer program, and the computer program is loaded by a processor to execute any frame rate detection method provided in the embodiment of the present application.

[0036] Accordingly, an embodiment of the present application further provides a computer program product, including a computer program, which implements any frame rate detection method provided in the embodiment of the present application when executed by a processor.

[0037] In the embodiment of the present application, when the terminal displays the received video frames within a preset time period, the status information of each display area of ​​the terminal is obtained by performing status detection on each display area of ​​the terminal. A display area in the terminal is used to display the detection image block in a video frame, and the status information represents the frame loss status of the corresponding video frame; according to the status information of each display area of ​​the terminal, the number of video frames without frame loss within the preset time period is counted; and the display frame rate of the terminal is calculated based on the number of video frames. In this solution, each display area of ​​the terminal corresponds to the detection image block contained in a video frame. By performing status detection on each display area of ​​the terminal, the refresh of the video frame can be accurately detected. The number of video frames without frame loss can be determined according to the status information of each display area, and then the display frame rate of the terminal can be determined. There is no need to compare the content between video frames through complex calculations, thereby improving the efficiency of frame rate detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a scene graph of the frame rate detection method provided in an embodiment of the present application;

[0040] Figure 2 is a flow chart of a frame rate detection method provided in an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a video frame sequence provided by an embodiment of the present application;

[0042] Figure 4 is another schematic diagram of a video frame sequence provided by an embodiment of the present application;

[0043] Figure 5 is a schematic diagram of a long exposure image provided in an embodiment of the present application;

[0044] Figure 6 Another flow chart of the frame rate detection method provided in the embodiment of the present application is

[0045] Figure 7 is another scene diagram of the frame rate detection method provided in an embodiment of the present application;

[0046] Figure 8is a schematic diagram of a frame rate detection device provided in an embodiment of the present application;

[0047] Fig. 9 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0049] The embodiments of the present application provide a frame rate detection method, device, computer equipment and computer readable storage medium. The frame rate detection device can be integrated in a computer equipment, which can be a server or a terminal.

[0050] The terminal may include a mobile phone, a wearable smart device, a tablet computer, a laptop computer, a personal computer (PC), and a vehicle-mounted computer.

[0051] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0052] For example, see Figure 1 The scene diagram for measuring the display frame rate of a cloud game client is shown. The cloud game server sends a video frame sequence to the cloud game client through the network, wherein each video frame in the video frame sequence includes a detection image block, and the detection image blocks of different video frames are located at different positions. The cloud game client is installed on the terminal. The cloud game client receives the video frame sent by the cloud game server, decodes the video, and displays it through the display screen of the terminal, wherein the terminal display screen includes multiple display areas, and one display area corresponds to the detection image block contained in a video frame.

[0053] The computer device performs status detection on each display area of ​​the terminal to obtain status information of each display area. Based on the status information of each display area, the display frame rate of the terminal is calculated based on the number of video frames without frame loss within a preset time period, and the number of video frames with frame loss can be determined.

[0054] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0055] This embodiment will be described from the perspective of a frame rate detection device. The frame rate detection device may be integrated into a computer device, which may be a server or a terminal.

[0056] The present application provides a frame rate detection method, such as Figure 2 As shown, the specific process of the frame rate detection method can be as follows:

[0057] 101. When the terminal displays the received video frame within a preset time period, a status detection is performed on each display area of ​​the terminal to obtain status information of each display area. A display area in the terminal is used to display a detected image block in a video frame, and the status information represents the frame loss status of the corresponding video frame.

[0058] The terminal may include a detected terminal capable of displaying video frames, for example, a mobile phone, a wearable smart device, a tablet computer, a notebook computer, a personal computer (PC), and a vehicle-mounted computer.

[0059] The display area may be an area on a display device of the terminal for displaying video frame content, and the display device may be a device on the terminal or an external device.

[0060] The video frame may include a detection image block, and the detection image blocks of different video frames are located at different positions, and the detection image blocks of different video frames correspond to different display areas on the terminal display, that is, a display area of ​​the terminal display corresponds to a detection image block for displaying a video frame. The detection image block may be an image area included in the video frame, and other image areas of the detection image block that are different from the video frame can be distinguished from other image areas on the video frame. If the detection image block of the video frame is detected, it means that the video frame has no frame loss.

[0061] The state information may include information indicating that the display area has displayed the detection image block of the corresponding video frame, and information indicating that the detection image block of the corresponding video frame is not displayed.

[0062] Among them, the frame loss status may include frame loss and non-frame loss. A display area corresponds to a detection image block that displays a video frame, and whether the video frame is displayed can be determined based on the status information of the display area. Therefore, the status information can represent the frame loss state of the video frame. For example, if the status information of the display area indicates that the display area has displayed the detection image block corresponding to the video frame, then the video frame has not been lost; if the status information of the display area indicates that the display area has not displayed the detection image block corresponding to the video frame, then the video frame has been lost.

[0063] For example, the server or other terminal may send a video frame sequence to the terminal, the video frame sequence may include multiple video frames, and each video frame includes at least one detection image block. For example, the server sends a video frame sequence to the terminal at a frame rate of 30 frames / s, and the video frame sequence includes video frames such as Figure 3 As shown, the positions of the detection image blocks of different video frames are different in the video frames. The image blocks in the video frames are sorted from left to right and from top to bottom, and the positions of the image blocks with corresponding serial numbers are determined as the positions of the detection image blocks according to the order of the video frames. For example, the position of the detection image block corresponding to the 1st frame of the video frame is the position of the first image block, and the position of the detection image block corresponding to the 30th frame of the video frame is the position of the 30th image block.

[0064] Optionally, each video frame in the video frame sequence may contain at least one image block, and the image block is a detection image block, for example, Figure 4 As shown, the video frame sequence includes 5 video frames, and the position of the detection image block contained in each video frame is different. The shape of the detection image block can be flexibly set according to the actual application scenario. In the video frame sequence, the shape of the detection image block can be the same or different, and the color displayed by the detection image block can also be flexibly set, which is not limited here.

[0065] The terminal receives a video frame sequence. Due to network fluctuations and other reasons, frame loss may occur. The number of video frames contained in the video frame sequence received by the terminal is less than the number of video frames contained in the video frame sequence sent by the server or other terminals. The terminal performs video decoding processing based on the received video frames and displays them on the terminal's display device.

[0066] The computer device performs status detection on each display area of ​​the display device, detects whether each display area corresponds to a detection image block that displays a video frame, and obtains status information of each display area.

[0067] The state detection for each display area can be performed by means of long exposure images, sensors, and target object recognition.

[0068] Among them, the long exposure image records the moving track of the detection image block. Since the detection image blocks contained in different video frames are located at different positions in the video frame sequence, the terminal displays the received video frame sequence, and it can be observed by the naked eye that the detection image block is moving. Therefore, the moving track of the detection image block can be recorded by the long exposure image, that is, in one embodiment, the step "when the terminal displays the received video frame within a preset time period, the state of each display area of ​​the terminal is detected to obtain the state information of each display area" can specifically include:

[0069] When the terminal displays the received video frame within the preset time period, obtaining a long exposure image corresponding to the video frame displayed by the terminal within the preset time period;

[0070] Based on the long exposure image, the status of each display area of ​​the terminal is detected to obtain the status information of each display area.

[0071] The preset time period may be a pre-set time period, such as 1 second, 5 seconds or 1 minute, and the preset time period may be the same as the period of the server sending the video frame sequence. The server may cyclically send the video frame sequence to the terminal, for example, the server sends a video frame to the terminal every 1 / 30 second, and then a complete video frame sequence is sent to the terminal after 1 second (for example, Figure 3 Therefore, the preset time period should be the same as the time when the server sends a complete video frame sequence, so as to avoid the situation where the server sends two video frames with the same content (the detection image block is located at the same position and the corresponding display area is the same), resulting in inaccurate detection of the corresponding display area. When the display area has not displayed the detection image block, it can be determined that both video frames are lost; and when the display area has displayed the detection image block, it cannot be determined whether one of the frames is lost.

[0072] For example, when the server sends a message to the terminal like Figure 3 When the video frame sequence is shown in FIG. 1 and there is no frame loss in the video frame, the long exposure image corresponding to the video frame displayed on the terminal within 1 second is obtained, and the following can be obtained: Figure 5 The long exposure image A shown in FIG. 1 shows that each display area of ​​the terminal displays the detected image block. When the server sends the terminal Figure 3 The video frame sequence shown in FIG. 1 and the 12th and 19th video frames are lost can be obtained as follows: Figure 5 The long exposure image B shown in the figure indicates that the display area for displaying the detection image blocks included in the 12th and 19th video frames has not displayed the detection image blocks. Different long exposure images corresponding to different dropped frames are different, please refer to the above examples, and will not be repeated here.

[0073] It can be detected in the long exposure image whether each display area of ​​the terminal has displayed a detection image block, and the state information of the display area is determined according to whether the detection image block has been displayed.

[0074] The long-exposure image may be obtained from a cloud server, or from another terminal, or may be obtained through an image acquisition device. That is, in one embodiment, the step of “obtaining a long-exposure image corresponding to a video frame displayed by the terminal within a preset time period” may specifically include:

[0075] During a preset time period, an image acquisition device is used to detect optical signals of a video image displayed by the terminal based on a video frame;

[0076] The light signals detected within a preset time period are superimposed and processed to obtain a long exposure image.

[0077] Among them, the image acquisition device may include an image acquisition device camera, a camcorder, a camera, a scanner, and other devices with a camera function (such as a mobile phone and a tablet computer, etc.). The image acquisition device may be an external device or a device carried by a computer device.

[0078] For example, the terminal may specifically display a video picture corresponding to the received video frame, and within a preset time period, perform optical signal detection on the video picture displayed by the terminal through an image acquisition device, and perform superposition processing on the detected optical signals to obtain a long exposure image corresponding to the video picture displayed by the terminal within the preset time period.

[0079] It can be understood that when a video frame is lost, the video frame is not displayed by the terminal, no light signal is generated in the display area corresponding to the video frame, and the light signal emitted by the detection image block contained in the video frame is not detected by the image acquisition device, so it is not superimposed on the long exposure image. Therefore, the display area corresponding to the detection image block of the lost video frame is not the detection image block in the long exposure image.

[0080] The long exposure image includes multiple display image blocks, and one display image block corresponds to a display area of ​​the terminal. By performing pixel detection on each display image block in the long exposure image, the state information of the display area corresponding to the display image block is determined according to the pixel color value of each display image block. That is, in one embodiment, the step of "based on the long exposure image, performing state detection on each display area of ​​the terminal to obtain the state information of each display area" may specifically include:

[0081] Performing pixel value detection on the display image blocks contained in the long exposure image to obtain the pixel color value of each display image block;

[0082] Determine the area type of the display image block according to the pixel color value of each display image block;

[0083] The state information of the corresponding display area is determined according to the area type of the display image block.

[0084] The pixel color value may represent the color of the display image block, and the region type may represent whether the display image block displays the detection image block.

[0085] The region type may include a first region type and a second region type, the first region type indicating that the displayed image block displays a detection image block, and the second region type indicating that the displayed image block does not display a detection image block.

[0086] For example, specifically, pixel value detection may be performed on each display image block contained in the long exposure image. For example, the pixel color value of the central pixel of each display image block may be detected, and the pixel color value of the central pixel may be used as the pixel color value of the display image block; or a pixel may be randomly obtained from the display image block, the pixel color value of the pixel may be detected, and the pixel color value may be used as the pixel color value of the display image block; or several pixels may be obtained from the display image block, the average value of the pixel color values ​​may be calculated, and the average value may be used as the pixel color value of the display image block.

[0087] Since the detection image block can be distinguished from other image areas in the video frame, it can be determined based on the pixel color value whether the display image block on the long exposure image displays the detection image block. Therefore, the area type of the display image block can be determined based on the pixel color value of the display image block. If the pixel color value meets the preset conditions, for example, it is the same as the pixel color value of the detection image block, then the area type of the display image block is determined to be the first area type; otherwise, the area type of the display image block is determined to be the second area type.

[0088] If the displayed image block is of the first area type, the status information of the corresponding display area indicates that the display area has displayed the detection image block of the corresponding video frame. If the displayed image block is of the second area type, the status information of the corresponding display area indicates that the display area does not display the detection image block of the corresponding video frame.

[0089] The above describes the state detection of the display area through the long exposure image. The following describes the state detection of the display area through the sensor. That is, in one embodiment, the step of "when the terminal displays the received video frame within the preset time period, the state detection is performed on each display area of ​​the terminal to obtain the state information of each display area" specifically includes:

[0090] When the terminal displays the received video frame within a preset time period, the sensor detects the light signal change of each display area to obtain the signal detection result of each display area;

[0091] The status information of each display area is determined according to the signal detection result.

[0092] The sensor may be a photosensor, which is a sensor that converts light signals into electrical signals using a photosensitive element.

[0093] For example, a corresponding photosensitive sensor may be set for each display area on the display device of the terminal, and the sensor may be used to detect changes in light signals in each display area. When the display area displays the detection image block contained in the corresponding video frame, the light signal emitted by the display area may change, and the sensor may detect the changes in the light signals in the display area. Therefore, the sensor may be used to detect changes in light signals in each display area to determine the status information of the display area.

[0094] If the sensor detects a change in the light signal in the display area, the sensor generates an electrical signal, and determines the signal detection result as a change in the light signal based on the electrical signal. If the sensor does not detect a change in the light signal in the display area, the signal detection result is that the light signal has not changed.

[0095] Since the shape of the detection image block can be known, the position of the detection image block can be determined by identifying the detection image block in the video screen played by the terminal, and then the video frame with frame loss can be determined. That is, in one embodiment, the step of "when the terminal displays the received video frame within a preset time period, performing status detection on each display area of ​​the terminal to obtain status information of each display area" can specifically include:

[0096] Acquire initial detection information, the initial detection information including sub-detection information;

[0097] When the terminal displays the received video frame within a preset time period, the detection object is identified for each display area of ​​the terminal to determine the target display area where the detection image block is located;

[0098] Selecting and processing the sub-detection information corresponding to the target display area in the initial detection information to obtain detection information;

[0099] The status information of each display area is obtained according to the detection information.

[0100] The initial detection information may include multiple sub-detection information, each sub-detection information in the initial detection information represents the initial state information of the corresponding display area, and the initial state information represents that the display area has no undisplayed detection image blocks. The initial detection information may be a multi-dimensional vector, one dimension in the initial detection information corresponds to one display area, one display area corresponds to one video frame, and therefore one dimension in the initial detection information is equivalent to one video frame.

[0101] For example, the initial detection information may be obtained. If 0 indicates that no detection image block is detected, and 1 indicates that a detection image block is detected, and there are 5 display areas in total, the initial detection information may be (0, 0, 0, 0, 0).

[0102] When the terminal displays the received video frames within a preset time period, multiple images or videos of the video frames displayed by the terminal are obtained, and detection object recognition is performed on each display area of ​​the terminal through the images or videos. For example, the display area corresponding to the position where the detection image block appears, that is, the target display area, is identified through a template matching algorithm. The detection image block can also be identified from the above-mentioned images or videos through a trained neural network model.

[0103] The template matching algorithm refers to finding the part of the above image or video that is similar to the template image by comparing the template image (preset detection image block) with the above image or video. By calculating the similarity between the template image and the target in the test image, the detection image block can be quickly located in the above image or video.

[0104] The neural network model can be trained through training samples so that the neural network model can locate and detect image blocks from images or videos. The trained neural network model can identify the detection objects in the above images or videos and determine the locations of the detection image blocks.

[0105] The sub-detection information corresponding to the target display area in the initial detection information is selected and processed, that is, the sub-detection information represents that the target display area displays the detection video block of the corresponding video frame, for example, 0 in the initial detection information is set to 1.

[0106] By performing detection object recognition on each display area of ​​the terminal, the display area that has displayed the detection image block can be determined, and then the sub-detection information corresponding to the display area that has displayed the detection image block is selected and processed to obtain the detection information of the display area that has displayed the detection image block. Since the status information of the display area can include information indicating that the display area has displayed the detection image block of the corresponding video frame, and information indicating that the detection image block of the corresponding video frame is not displayed, the status information of each display area can be obtained according to the detection information.

[0107] 102. According to the status information of each display area of ​​the terminal, count the number of video frames that are not lost within a preset time period.

[0108] For example, the state information of each display area may be specifically used. The state information may be statistically representative of the number of display areas of the detection image blocks of the corresponding video frame that have been displayed, so as to obtain the number of video frames without frame loss within the preset time period. That is, step 102 may specifically include:

[0109] Counting the number of display areas whose status information is the target status information within a preset time period;

[0110] Determining the number of regions as the number of video frames without frame loss within a preset time period;

[0111] The target state information may be information indicating that the display area has displayed a detection image block of the corresponding video frame.

[0112] 103. Calculate the display frame rate of the terminal based on the number of video frames.

[0113] The display frame rate can be expressed as the frequency at which a bitmap image in units of frames appears continuously on the display, and the display frame rate indicates the number of times the graphics processor can refresh the video frame per second. A high display frame rate can produce smoother and more realistic animations.

[0114] For example, if the preset time period is a unit time (such as 1 s), the number of video frames is the display frame rate of the terminal.

[0115] If the preset time period is not a unit time, the ratio between the number of video frames and the preset time period may be calculated to obtain the display frame rate of the terminal.

[0116] As can be seen from the above, in the embodiment of the present application, when the terminal displays the received video frames within a preset time period, the status information of each display area of ​​the terminal is obtained by performing status detection on each display area of ​​the terminal. A display area in the terminal is used to display the detection image block in a video frame, and the status information represents the frame loss status of the corresponding video frame; according to the status information of each display area of ​​the terminal, the number of video frames without frame loss within the preset time period is counted; and the display frame rate of the terminal is calculated based on the number of video frames. In this solution, each display area of ​​the terminal corresponds to the detection image block contained in a video frame for displaying. By performing status detection on each display area of ​​the terminal, the refresh of the video frame can be accurately detected. The number of video frames without frame loss can be determined based on the status information of each display area, and then the display frame rate of the terminal can be determined. There is no need to compare the content between video frames through complex calculations, thereby improving the efficiency of frame rate detection.

[0117] Based on the above embodiments, further detailed description will be given below with examples.

[0118] This embodiment will be described from the perspective of a frame rate detection system and its application in cloud gaming scenarios; the frame rate detection system may include a cloud gaming server, a detected terminal installed with a cloud gaming client, and a detection terminal, which may include a camera; cloud gaming refers to a gaming mode in which a server is used to render the game screen and a user client is only used for display.

[0119] The present application provides a frame rate detection method, which can be executed by a processor of a terminal, such as Figure 6 As shown, the specific process of the frame rate detection method can be as follows:

[0120] 201. The cloud game server sends a first video frame sequence to the cloud game client at a preset frame rate.

[0121] For example, it can be as follows Figure 7 As shown, the cloud game server continuously sends a first video frame sequence to the cloud game client at a preset frame rate of 30 frames per second. The first video frame sequence can be as follows: Figure 3 As shown, the cloud game server can cyclically send the first video frame sequence to the cloud game server until the frame rate detection process of the detected terminal is completed.

[0122] The first video frame sequence may be pre-stored in the cloud game server, or may be generated by the cloud game server according to a regularity of detecting positions of image blocks in the first video frame sequence.

[0123] 202. The cloud game client receives and displays the second video frame sequence.

[0124] For example, it can be as follows Figure 7 As shown, the cloud gaming client receives the second video frame sequence and displays it continuously.

[0125] In the absence of frame loss, the second video frame sequence is the same as the first video frame sequence. When frame loss occurs, the second video frame sequence only includes some video frames in the first video frame sequence.

[0126] 203. The detecting terminal obtains, through a camera, a long exposure image corresponding to the second video frame sequence displayed by the detected terminal within a preset time period.

[0127] For example, specifically, the detected terminal may display a video picture corresponding to a received video frame. Within a preset time period, the detection terminal performs optical signal detection on the video picture displayed by the detected terminal through a camera, and superimposes the detected optical signals to obtain a long exposure image corresponding to the video picture displayed by the detected terminal within the preset time period.

[0128] like Figure 7As shown in FIG. 1 , when the 12th frame and the 19th frame are lost, the long exposure image corresponding to the video screen displayed by the detected terminal within the preset time period is obtained as shown in FIG. Figure 5 Long exposure image B is shown.

[0129] Among them, the long exposure image can record the movement trajectory of the detection image block. Since the detection image blocks contained in different video frames are located at different positions in the video frame sequence, the detected terminal displays based on the received video frame sequence. It can be observed by the naked eye that the detection image block is moving, jumping from one display area to the next display area. Therefore, the movement trajectory of the detection image block can be recorded by the long exposure image. If the video frame is not lost, each displayed image block should have displayed the detection image block. If there is a display image block that has not displayed the detection image block, it means that the video frame in the video frame sequence is lost.

[0130] 204. The detection terminal performs status detection on each display area of ​​the detected terminal based on the long exposure image to obtain status information of each display area.

[0131] For example, specifically, pixel value detection may be performed on each display image block contained in the long exposure image. For example, the pixel color value of the central pixel of each display image block may be detected, and the pixel color value of the central pixel may be used as the pixel color value of the display image block; or a pixel may be randomly obtained from the display image block, the pixel color value of the pixel may be detected, and the pixel color value may be used as the pixel color value of the display image block; or several pixels may be obtained from the display image block, the average value of the pixel color values ​​may be calculated, and the average value may be used as the pixel color value of the display image block.

[0132] Since the detection image block can be distinguished from other image areas in the video frame, it can be determined based on the pixel color value whether the display image block on the long exposure image displays the detection image block. Therefore, the area type of the display image block can be determined based on the pixel color value of the display image block. If the pixel color value meets the preset conditions, for example, it is the same as the pixel color value of the detection image block, then the area type of the display image block is determined to be the first area type; otherwise, the area type of the display image block is determined to be the second area type.

[0133] If the displayed image block is of the first area type, the status information of the corresponding display area indicates that the display area has displayed the detection image block of the corresponding video frame. If the displayed image block is of the second area type, the status information of the corresponding display area indicates that the display area does not display the detection image block of the corresponding video frame.

[0134] 205. The detecting terminal counts the number of video frames without frame loss within a preset time period according to the status information of each display area of ​​the detected terminal.

[0135] For example, the specific information may be the status information of each display area. The statistical status information may represent the number of display areas of the detection image blocks that have displayed the corresponding video frame, and the number of video frames without frame loss within a preset time period may be obtained.

[0136] 206. The detecting terminal calculates a display frame rate of the detected terminal based on the number of video frames.

[0137] For example, if the preset time period is a unit time (such as 1 s), the number of video frames is the display frame rate of the terminal.

[0138] If the preset time period is not a unit time, the ratio between the number of video frames and the preset time period may be calculated to obtain the display frame rate of the terminal.

[0139] The embodiment of the present application can not only detect the display frame rate of the detected device, but also obtain the number of lost frames. When the network environment is stable, the higher the display frame rate, the better the video decoding capability and display refresh capability of the detected device. The video frames displayed by the detected device are more coherent, the presented video pictures are smoother, and the user experience is better.

[0140] When the display frame rate of the detected device is determined, the number of frame losses caused by network jitter can be determined, and a network with a good network status can be switched in time to provide better services.

[0141] As can be seen from the above, in the embodiment of the present application, the cloud game server sends a first video frame sequence to the cloud game client at a preset frame rate; the cloud game client receives a second video frame sequence and displays it; the detection terminal obtains a long exposure image corresponding to the second video frame sequence displayed by the detected terminal through a camera within a preset time period; the detection terminal performs a status detection on each display area of ​​the detected terminal based on the long exposure image to obtain the status information of each display area; the detection terminal counts the number of video frames without frame loss within a preset time period based on the status information of each display area of ​​the detected terminal; the detection terminal calculates the display frame rate of the detected terminal based on the number of video frames. In this scheme, each display area of ​​the terminal corresponds to a detection image block contained in a video frame for displaying. By performing a status detection on each display area of ​​the terminal, the refresh of the video frame can be accurately detected. The number of video frames without frame loss can be determined based on the status information of each display area, and then the display frame rate of the terminal is determined. There is no need to compare the content between video frames through complex calculations, thereby improving the efficiency of frame rate detection.

[0142] In order to better implement the frame rate detection method provided in the embodiment of the present application, a frame rate detection device is also provided in one embodiment. The meanings of the terms are the same as those in the above frame rate detection method, and the specific implementation details can refer to the description in the method embodiment.

[0143] The frame rate detection device can be integrated into a computer device, such as Figure 8 As shown, the frame rate detection device may include: a state detection unit 301, a quantity statistics unit 302 and a frame rate calculation unit 303, which are specifically as follows:

[0144] (1) Status detection unit 301: used to perform status detection on each display area of ​​the terminal when the terminal displays the received video frame within a preset time period to obtain status information of each display area. A display area in the terminal is used to display a detected image block in a video frame, and the status information represents the frame loss status of the corresponding video frame.

[0145] In one embodiment, the state detection unit 301 may include an image acquisition subunit and a detection subunit, specifically:

[0146] An image acquisition subunit: used for acquiring a long exposure image corresponding to a video frame displayed by the terminal within a preset time period when the terminal displays a received video frame within a preset time period;

[0147] Detection subunit: used to perform status detection on each display area of ​​the terminal based on the long exposure image to obtain status information of each display area.

[0148] In one embodiment, the image acquisition subunit may include an optical signal detection module and a superposition processing module, specifically:

[0149] Optical signal detection module: used to detect optical signals of video images displayed by the terminal based on video frames through an image acquisition device within a preset time period;

[0150] Overlay processing module: used to perform overlay processing on the light signals detected within a preset time period to obtain a long exposure image.

[0151] In one embodiment, the long exposure image includes a plurality of display image blocks, one display image block corresponds to a display area of ​​the terminal, and the detection subunit may include a pixel value detection module, a type determination module, and a state determination module, specifically:

[0152] Pixel value detection module: used to detect pixel values ​​of display image blocks contained in the long exposure image to obtain the pixel color value of each display image block;

[0153] Type determination module: used to determine the area type of the display image block according to the pixel color value of each display image block;

[0154] State determination module: used to determine the state information of the corresponding display area according to the area type of the display image block.

[0155] In one embodiment, the state detection unit may include a change detection subunit state and information determination subunit, specifically:

[0156] The change detection subunit is used to detect the change of the optical signal of each display area through the sensor when the terminal displays the received video frame within a preset time period, and obtain the signal detection result of each display area;

[0157] Status information determination subunit: used to determine the status information of each display area according to the signal detection result.

[0158] In one embodiment, the state detection unit may include an information acquisition subunit, a detection object identification subunit, a selection processing subunit, and an information acquisition subunit, specifically:

[0159] Information acquisition subunit: used to acquire initial detection information, which includes sub-detection information;

[0160] The detection object identification subunit is used to perform detection object identification on each display area of ​​the terminal when the terminal displays the received video frame within a preset time period, and determine the target display area where the detection image block is located;

[0161] A selection processing subunit: used for selecting and processing the sub-detection information at the position corresponding to the initial detection information according to the target display area to obtain the detection information;

[0162] Information obtaining subunit: used to obtain status information of each display area according to the detection information.

[0163] (2) Quantity counting unit 302: used to count the number of video frames that have not been lost within a preset time period according to the status information of each display area of ​​the terminal.

[0164] In one embodiment, the state information includes target state information, the target state information indicates that the corresponding video frame is not lost, and the quantity counting unit may include a region quantity counting subunit and a quantity determining subunit, specifically:

[0165] Area quantity counting subunit: used to count the number of areas whose status information is the display area of ​​the target status information within a preset time period;

[0166] The number determination subunit is used to determine the number of regions as the number of video frames without frame loss within a preset time period.

[0167] (3) Frame rate calculation unit 303: used to calculate the display frame rate of the terminal based on the number of video frames.

[0168] As can be seen from the above, in the embodiment of the present application, when the terminal displays the received video frame within a preset time period, the frame rate detection device performs state detection on each display area of ​​the terminal through the state detection unit 301 to obtain the state information of each display area. A display area in the terminal is used to display the detection image block in a video frame, and the state information represents the frame loss state of the corresponding video frame; the number of video frames without frame loss within the preset time period is counted according to the state information of each display area of ​​the terminal through the number counting unit 302; finally, the display frame rate of the terminal is calculated based on the number of video frames by the frame rate calculation unit 303. In this scheme, each display area of ​​the terminal corresponds to the detection image block contained in a video frame for displaying. By performing state detection on each display area of ​​the terminal, the refresh of the video frame can be accurately detected. The number of video frames without frame loss can be determined according to the state information of each display area, and then the display frame rate of the terminal is determined. There is no need to compare the content between video frames through complex calculations, thereby improving the efficiency of frame rate detection.

[0169] The present application also provides a computer device, which may be a terminal or a server. Fig. 9 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0170] The computer device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will appreciate that Fig. 9 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0171] The processor 1001 is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 1002, and calling data stored in the memory 1002, the processor 1001 performs various functions of the computer device and processes data. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and computer programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.

[0172] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0173] The computer device also includes a power supply 1003 for supplying power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1003 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0174] The computer device may further include an input unit 1004, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0175] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically, in this embodiment, the processor 1001 in the computer device will load the executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 will run the computer programs stored in the memory 1002, thereby realizing various functions, as follows:

[0176] When the terminal displays the received video frame within a preset time period, the status information of each display area is obtained by performing status detection on each display area of ​​the terminal. One display area in the terminal is used to display the detected image block in one video frame, and the status information represents the frame loss status of the corresponding video frame;

[0177] According to the status information of each display area of ​​the terminal, the number of video frames that have not been lost within a preset time period is counted;

[0178] Calculate the terminal's display frame rate based on the number of video frames.

[0179] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0180] As can be seen from the above, in the embodiment of the present application, when the terminal displays the received video frame within a preset time period, the computer device can obtain the status information of each display area by performing status detection on each display area of ​​the terminal, and a display area in the terminal is used to display the detection image block in a video frame, and the status information represents the frame loss status of the corresponding video frame; according to the status information of each display area of ​​the terminal, the number of video frames without frame loss within the preset time period is counted; and the display frame rate of the terminal is calculated based on the number of video frames. In this scheme, each display area of ​​the terminal corresponds to the detection image block contained in a video frame for displaying. By performing status detection on each display area of ​​the terminal, the refresh of the video frame can be accurately detected. The number of video frames without frame loss can be determined according to the status information of each display area, and then the display frame rate of the terminal can be determined. There is no need to compare the content between video frames through complex calculations, thereby improving the efficiency of frame rate detection.

[0181] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in various optional implementations in the above-mentioned embodiments.

[0182] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0183] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program can be loaded by a processor to execute any frame rate detection method provided in the embodiment of the present application.

[0184] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0185] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0186] Since the computer program stored in the computer-readable storage medium can execute any frame rate detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any frame rate detection method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0187] The above is a detailed introduction to a frame rate detection method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A frame rate detection method, characterized in that: include: When the terminal displays the received video frame within a preset time period, a status detection is performed on each display area of ​​the terminal to obtain status information of each display area, wherein a display area in the terminal is used to display a detection image block in a video frame, and the status information represents a frame loss status of the corresponding video frame; wherein the detection image blocks of different video frames are located at different positions, and the detection image blocks of different video frames correspond to different display areas on the terminal display; According to the status information of each display area of ​​the terminal, counting the number of video frames that have not been lost within the preset time period; A display frame rate of the terminal is calculated based on the number of video frames.

2. The method according to claim 1, characterized in that When the terminal displays the received video frame within a preset time period, a state detection is performed on each display area of ​​the terminal to obtain state information of each display area, including: When the terminal displays the received video frame within a preset time period, obtaining a long exposure image corresponding to the video frame displayed by the terminal within the preset time period; Based on the long exposure image, status detection is performed on each display area of ​​the terminal to obtain status information of each display area.

3. The method according to claim 2, characterized in that The acquiring, within the preset time period, the terminal displays the long exposure image corresponding to the video frame, comprising: Within the preset time period, performing optical signal detection on the video picture displayed by the terminal based on the video frame by an image acquisition device; The light signals detected within the preset time period are superimposed to obtain the long exposure image.

4. The method according to claim 2, characterized in that: The long exposure image includes a plurality of display image blocks, one display image block corresponds to a display area of ​​the terminal, and the state detection is performed on each display area of ​​the terminal based on the long exposure image to obtain the state information of each display area, including: Performing pixel value detection on display image blocks included in the long exposure image to obtain a pixel color value of each display image block; Determining the area type of the display image block according to the pixel color value of each display image block; The state information of the corresponding display area is determined according to the area type of the display image block.

5. The method according to claim 1, characterized in that When the terminal displays the received video frame within a preset time period, a state detection is performed on each display area of ​​the terminal to obtain state information of each display area, including: When the terminal displays the received video frame within a preset time period, the sensor detects the light signal change of each display area to obtain the signal detection result of each display area; The status information of each of the display areas is determined according to the signal detection result.

6. The method according to claim 1, characterized in that When the terminal displays the received video frame within a preset time period, a state detection is performed on each display area of ​​the terminal to obtain state information of each display area, including: Acquire initial detection information, where the initial detection information includes sub-detection information; When the terminal displays the received video frame within a preset time period, performing detection object recognition on each display area of ​​the terminal to determine the target display area where the detection image block is located; Selecting and processing the sub-detection information at the position corresponding to the initial detection information according to the target display area to obtain detection information; The status information of each display area is obtained according to the detection information.

7. The method according to any one of claims 1 to 6, characterized in that: The state information includes target state information, the target state information indicates that the corresponding video frame has not been lost, and counting the number of video frames that have not been lost within the preset time period according to the state information of each display area of ​​the terminal includes: Counting the number of display areas where the status information is target status information within the preset time period; The number of regions is determined as the number of video frames without frame loss within the preset time period.

8. A frame rate detection device, characterized in that: include: A state detection unit, used for performing state detection on each display area of ​​the terminal when the terminal displays the received video frame within a preset time period to obtain state information of each display area, wherein one display area in the terminal is used to display a detection image block in a video frame, and the state information represents the frame loss state of the corresponding video frame; wherein the detection image blocks of different video frames are located at different positions, and the detection image blocks of different video frames correspond to different display areas on the terminal display; A quantity counting unit, used for counting the number of video frames without frame loss within the preset time period according to the state information of each display area of ​​the terminal; A frame rate calculation unit is used to calculate the display frame rate of the terminal based on the number of video frames.

9. A computer device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the frame rate detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the frame rate detection method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the frame rate detection method according to any one of claims 1 to 7 is implemented.

Images