Display control method and electronic device

Abstract

The present application provides a method and electronic device for display control, the method is applied to a first electronic device, the method comprising: the first electronic device receives a video frame sent by a second electronic device; the first electronic device decodes the video frame to obtain a decoded target video frame; the first electronic device puts the target video frame into a first cache queue; the first electronic device determines whether to render and display the target video frame according to a first preset condition, wherein the time interval for the first electronic device to render and display the target video frame is the same as the screen refresh time interval of the first electronic device. The method can be applied to screen projection scenarios, and the technical solution can control the display of video frames, improving the real-time, clarity and smoothness during the screen projection process.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and more specifically, to a display control method and electronic device. Background Art

[0002] In the application scenario of screen projection, for example, projecting the screen of a mobile phone to another electronic device (such as a computer, smart screen, etc.), that is, the picture in the mobile phone needs to be displayed in real time on the screen of another electronic device, then it is necessary to ensure the stability, real-time and high fluency of the projected content.

[0003] During the screen projection process, the other electronic device needs to render and display each video frame in the received video stream. For example, the screen refresh frame rate of the other electronic device is 60Hz, that is, the screen is refreshed every 16.7ms, and one video frame is displayed each time. Ideally, the video receiving frame rate of the mobile phone's video stream received by the other electronic device should also be consistent with its screen refresh frame rate, that is, one video frame is received every 16.7ms. However, due to network fluctuations and other problems during the video stream transmission process, the screen refresh frame rate of the other electronic device and the video receiving frame rate of the video stream may not completely match. When the video receiving frame rate received by the other electronic device is greater than its screen refresh frame rate, some video frames may be lost during the rendering and display process, causing frame loss problems; or, when the video receiving frame rate received by the other electronic device is less than its screen refresh frame rate, the delay of some video frames may increase. Summary of the invention

[0004] The present application provides a display control method and electronic device, which can control the display of video frames and improve the real-time performance, clarity and smoothness during the screen projection process.

[0005] In a first aspect, a method for display control is provided, which is applied to a first electronic device, and the method includes: the first electronic device receives a video frame sent by a second electronic device; the first electronic device decodes the video frame to obtain a decoded target video frame; the first electronic device puts the target video frame into a first cache queue; the first electronic device determines whether to render and display the target video frame according to a first preset condition, wherein the time interval for the first electronic device to render and display the target video frame is the same as the screen refresh time interval of the first electronic device.

[0006] Based on the embodiments of the present application, the first electronic device can actively control the decoded target video frame, and determine whether to render and display the target video frame according to preset conditions. When it is determined to render and display the target video frame, the time interval for rendering and displaying the target video frame by the first electronic device is the same or substantially the same as the time interval for refreshing its screen, thereby accurately controlling the target video frame and improving the display frame rate, real-time performance, clarity and smoothness of the projection screen.

[0007] In combination with the first aspect, in an implementation method of the first aspect, the first electronic device determines whether to render and display the target video frame according to a first preset condition, including: the first electronic device determines a first duration that the target video frame stays in the first cache queue; if the first duration is greater than a first threshold, the target video frame is discarded.

[0008] Based on the embodiment of the present application, when the first duration of the target video frame staying in the first cache queue is greater than the first threshold, it means that the target video frame stays in the first cache queue for a long time and the delay is large. At this time, the target video frame can be discarded, which is beneficial to ensure the real-time performance of the screen projection.

[0009] In combination with the first aspect, in an implementation of the first aspect, the method also includes: if the first duration is less than or equal to the first threshold, the first electronic device determines a first number of video frames cached in the first cache queue; if the first number is greater than a second threshold, the target video frame is discarded; if the first number is less than or equal to the second threshold, the first electronic device determines to render and display the target video frame.

[0010] Based on the embodiment of the present application, when the cache amount of the first cache queue is greater than the second threshold, it means that the number of video frames cached in the first cache queue is large, which may cause the subsequently decoded video frames to be unable to enter the first cache queue. Therefore, in this case, the target video frame is discarded, so that the number of cached video frames in the first cache queue is within an appropriate range, avoiding the subsequent decoded video frames from being unable to enter the first cache queue, thereby improving the real-time and smoothness of the screen projection.

[0011] In combination with the first aspect, in an implementation manner of the first aspect, the first electronic device determines to render and display the target video frame, including: the first electronic device renders and displays the target video frame after a second duration.

[0012] It should be understood that the second duration may be 0 or other values, which may be related to the number of video frames in the first cache queue and the refresh time interval of the screen.

[0013] Based on the embodiment of the present application, when the first electronic device determines to render and display the target video frame, it may be rendered and displayed immediately, or it may be rendered and displayed after a period of time. The technical solution can actively control the rendering and display of the target video frame to improve the display frame rate of the projection screen.

[0014] In combination with the first aspect, in an implementation of the first aspect, before the first electronic device places the target video frame into a first cache queue, the method also includes: the first electronic device determines a first duration that the target video frame needs to stay in the first cache queue; if the first duration is greater than a first threshold, the target video frame is discarded.

[0015] Based on the embodiment of the present application, before placing the target video frame into the cache queue, the first electronic device can pre-determine the first duration for which the target video frame needs to stay in the cache queue if the target video frame is placed into the cache queue, and when the first duration is greater than the first threshold, the target video frame can be discarded. This technical solution can prevent the target video frame from staying in the first cache queue for a long time and having a large delay, thereby facilitating the real-time performance of screen projection.

[0016] In combination with the first aspect, in an implementation of the first aspect, the method also includes: if the first duration is less than or equal to a first threshold, the first electronic device determines a first number of video frames cached in the first cache queue; if the first number is greater than a second threshold, the target video frame is discarded; if the first number is less than or equal to the second threshold, the target video frame is placed in the first cache queue.

[0017] Based on the embodiment of the present application, when the cache amount of the first cache queue is greater than the second threshold, it means that the number of video frames cached in the first cache queue is large, which may cause the subsequently decoded video frames to be unable to enter the first cache queue. Therefore, in this case, the target video frame is discarded, so that the number of cached video frames in the first cache queue is within an appropriate range, avoiding the subsequent decoded video frames from being unable to enter the first cache queue, thereby improving the real-time and smoothness of the screen projection.

[0018] In combination with the first aspect, in an implementation method of the first aspect, the first electronic device determines whether to render and display the target video frame according to a first preset condition, including: after the target video frame stays in the first cache queue for the first period of time, the first electronic device renders and displays the target video frame.

[0019] It should be understood that the first duration may also be 0.

[0020] Based on the embodiment of the present application, after the target video frame is placed in the first cache queue, it can be sent to the display module of the first electronic device for rendering and display after the first duration. The technical solution can actively control the rendering and display of the target video frame to improve the display frame rate of the projection screen.

[0021] In combination with the first aspect, in an implementation of the first aspect, the method also includes: the first electronic device determines a first average time for a third preset number of target video frames to be decoded, rendered and displayed on a screen of the first electronic device; if the first average time is greater than a third threshold, reducing the screen refresh frame rate of the first electronic device.

[0022] It should be understood that the third preset number may be 30, 20, etc., the third threshold may be 20ms, 18ms, etc., and the third preset number of target video frames may be continuous or intermittent.

[0023] Based on the embodiments of the present application, when the average time taken for a certain number of target video frames to be decoded and displayed in the first electronic device is greater than the third threshold, it means that the delay of the multiple video frames is large. At this time, the first electronic device can downgrade the screen refresh frame rate, that is, reduce the screen refresh frame rate, thereby avoiding excessive delay in screen projection and improving the real-time and smoothness of screen projection.

[0024] In combination with the first aspect, in an implementation manner of the first aspect, before reducing the screen refresh frame rate of the first electronic device, the method also includes: the first electronic device determines that the current screen refresh frame rate is at the original screen refresh frame rate.

[0025] Based on the embodiments of the present application, before the first electronic device reduces the screen refresh frame rate, it can be determined that the current screen refresh frame rate is at the original size. This technical solution can avoid the first electronic device reducing the frame rate multiple times, resulting in a low frame rate.

[0026] In combination with the first aspect, in an implementation of the first aspect, the method also includes: after an interval of a fourth preset number of target video frames, the first electronic device continues to determine the second average time taken for the third preset number of target video frames to be decoded, rendered and displayed on the screen of the first electronic device; if the second average time is less than or equal to a fourth threshold, increasing the screen refresh frame rate of the first electronic device.

[0027] Based on the embodiment of the present application, after an interval of a preset number of target video frames, the first electronic device continues to determine the average time taken for a third preset number of target video frames to be decoded, rendered and displayed on the screen of the first electronic device. When the average time is less than a fourth threshold, it means that the delay of the multiple frames of target video frames is small. At this time, the first electronic device can increase the screen refresh frame rate to display more video frames, thereby improving the clarity of the projected picture.

[0028] In combination with the first aspect, in an implementation of the first aspect, before increasing the screen refresh frame rate of the first electronic device, the method further includes: the first electronic device determines that a current screen refresh frame rate is less than the original screen refresh frame rate.

[0029] Based on the embodiment of the present application, before the first electronic device increases the screen frame rate, it is necessary to determine that the current screen refresh frame rate is less than the original screen refresh frame rate.

[0030] In a second aspect, an electronic device is provided, comprising one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs comprising instructions, which, when executed by the one or more processors, enable the display control method as described in the first aspect and any possible implementation thereof to be executed.

[0031] In a third aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the display control method as described in the first aspect and any possible implementation thereof is executed.

[0032] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the display control method as described in the first aspect and any possible implementation thereof.

[0033] In a fifth aspect, a computer program product is provided, the computer program product comprising a computer program code, and when the computer program code is run on a computer, the computer executes the display control method as described in the first aspect and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of a scenario in which the embodiments of the present application can be applied.

[0035] Figure 2 It is a schematic diagram of a screen projection display provided in an embodiment of the present application.

[0036] Figure 3 It is a schematic diagram of another screen projection display provided in an embodiment of the present application.

[0037] Figure 4 This is a schematic flowchart of display control of a video frame by an electronic device provided in an embodiment of the present application.

[0038] Figure 5 This is a schematic flowchart of another electronic device displaying and controlling a video frame provided in an embodiment of the present application.

[0039] Figure 6 It is a schematic diagram of a display control method provided in an embodiment of the present application.

[0040] Figure 7 It is a schematic diagram of a display control method provided in an embodiment of the present application.

[0041] Figure 8 It is a schematic flowchart of a display control method provided in an embodiment of the present application.

[0042] Fig. 9 It is a schematic flowchart of a display control method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0044] The display control method in the embodiments of the present application can be applied to electronic devices such as smart phones, tablet computers, laptop computers, personal computers (PCs), ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), vehicle-mounted devices, wearable devices, etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0045] Before introducing the technical solutions of the embodiments of the present application, some professional terms involved in the present application are briefly introduced.

[0046] Video reception frame rate: is a measure for measuring the number of frames received for display. The unit is frames per second (FPS) or Hertz (Hz). In some possible embodiments, the video is encoded on the first electronic device and then sent to the second electronic device. The video reception frame rate may refer to the number of frames received per second by the second electronic device.

[0047] Refresh frame rate: refers to the speed at which the image is updated on the screen, that is, the number of times the image on the screen appears per second, and the unit is Hertz. The higher the refresh rate, the less flickering the image on the screen will be, and the higher the stability will be. In some possible embodiments, the refresh frame rate refers to the refresh frame rate of the second electronic device, for example, the second electronic device displays 60 frames of images per second.

[0048] WiFi-P2P: defined as WiFi peer to peer (P2P), also known as WiFi direct (wifidirect), which allows devices in a wireless network to connect to each other and transmit data without going through a wireless router.

[0049] Figure 1 Schematic diagram of a scenario to which the embodiment of the present application can be applied. Figure 1 As shown, the first electronic device 100a may be a mobile phone, and the second electronic device 100b may be a PC. In the embodiment of the present application, projection of the screen from a mobile phone to a PC is used as an example for explanation.

[0050] See also Figure 1 , the mobile phone screen is displaying the picture 110 of Huawei Video. When the mobile phone is projected to the PC, the picture 110 is displayed on the screen of the PC, and the display area 121 on the screen of the PC can display the content of the picture 110. Optionally, a title bar part 122 may be included above the display area 121, and the title bar part may include the name of the application "Huawei Video" and controls for minimizing, maximizing, and closing the interface display.

[0051] Therefore, in order to ensure user experience, it is necessary to ensure that the screen 110 of the mobile phone can be displayed on the screen of the PC stably, in real time and smoothly.

[0052] Figure 2 It is a schematic diagram of a screen projection display provided in an embodiment of the present application.

[0053] like Figure 2 As shown, electronic device 200a can be a mobile phone, and electronic device 200b can be a PC. In the embodiment of the present application, projection of a screen from a mobile phone to a PC is used as an example for explanation. In some embodiments, the mobile phone and the PC can establish a connection through a WiFi-P2P connection method. In other embodiments, the mobile phone and the PC can also establish a connection through wireless local area networks (WLAN), Bluetooth, ultra wide band (UWB), wired connection, and other connection methods.

[0054] See also Figure 2During the screen projection process, the image drawing module 210 on the mobile phone obtains each frame of the video stream of the mobile phone, and then calls the encoder 220 to encode the video stream, for example, using H.264 encoding to encode the video stream, and then sends the encoded video stream to the PC through the Wi-Fi module 230 (such as a Wi-Fi chip).

[0055] After the Wi-Fi module 240 in the PC side receives the encoded video stream, it sends the encoded video stream to the image receiving module 250 of the PC's screen projection service. The image receiving module 250 calls the decoder 260 to decode the video stream (for example, calls the hardware processor at the chip layer for decoding). The decoder 260 sends the decoded video stream to the image display module 270. The image display module 270 renders the video frames in the video stream and displays them on the PC screen.

[0056] In the above screen projection process, the PC needs to render and display each frame in the received video stream, and the PC screen has a certain screen refresh frame rate. For example, the screen refresh frame rate is 60Hz, that is, the PC screen refreshes once every 16.7ms, and displays a video frame each time. Ideally, the video receiving frame rate of the mobile phone video stream received by the PC should also be consistent with the PC screen refresh frame rate, that is, a video frame is received every 16.7ms. However, due to network fluctuations and other issues during the video stream transmission process, the PC screen refresh frame rate and the video receiving frame rate of the mobile phone video stream may be different. Therefore, when the video receiving frame rate received by the PC is greater than the PC screen refresh frame rate, some video frames of the mobile phone may be lost during the display rendering process of the PC, resulting in a decrease in the clarity of the PC display, or when the video receiving frame rate received by the PC is less than the PC screen refresh frame rate, some video frames may be delayed more during the display rendering process of the PC.

[0057] In view of this, an embodiment of the present application provides a display control method, which can improve the real-time, clarity and smoothness of the screen projection process by actively controlling the display of video frames.

[0058] Figure 3 It is a schematic diagram of another screen projection display provided in an embodiment of the present application.

[0059] like Figure 3 As shown, the electronic device 300a may be a mobile phone, and the electronic device 300b may be a PC. In the embodiment of the present application, the mobile phone is used to project the screen onto the PC as an example for explanation, and the mobile phone and the PC may establish a connection through a WiFi-P2P connection mode.

[0060] See also Figure 3During the screen projection process, the image drawing module 310 on the mobile phone obtains each frame of the video stream of the mobile phone, and then calls the encoder 320 to encode the video stream, for example, using H.264 encoding to encode the video stream, and then sends the encoded video stream to the PC through the Wi-Fi module 330 (such as a Wi-Fi chip).

[0061] After receiving the encoded video stream, the Wi-Fi module 340 in the PC side sends the encoded video stream to the image receiving module 350 of the PC's screen projection service. The image receiving module 350 calls the decoder 360 to decode the video stream (for example, calls the hardware processor at the chip layer for decoding). The decoder 360 sends the decoded video stream to the display control module 370. The display control module 370 can control the display mode of the video frames in the decoded video stream. The display control module 370 sends the controlled video frames to the image display module 380. The image display module 370 renders the video stream and displays it on the PC screen.

[0062] Specifically, the display control module 370 may discard the video frames in the decoded video stream, or send them to the image display module 380 with a delay, or send them to the image display module 380 immediately, and so on.

[0063] The display control module 370 may include a decision module to execute the above decision.

[0064] Figure 4 4 is a schematic flow chart of an electronic device displaying and controlling a video frame provided in an embodiment of the present application. The method may include steps 401 to 407. In the embodiment of the present application, the screen refresh frame rate of a PC is 60 Hz as an example for explanation, and the time interval between two refreshes of the PC screen is 16.7 ms.

[0065] In this embodiment, the mobile phone projects the screen to the PC, that is, the screen content of the mobile phone is displayed on the screen of the PC. After the PC receives the content of the video stream of the mobile phone, the video frames in the video stream can be decoded, and the following steps can be performed for the decoded video frames.

[0066] 401, the PC receives a target video frame to be rendered.

[0067] Exemplarily, in this step, the display control module in the PC receives the decoded target video frame sent by the decoder, and actively controls the target video frame, such as placing the target video frame into the first cache queue, or discarding the target video frame.

[0068] 402 , determine whether a first duration that the target video frame will stay in the first cache queue is greater than a first threshold.

[0069] The first threshold may be calculated based on a screen refresh frame rate of the PC. For example, if the screen refresh frame rate of the PC is 60 Hz, the first threshold may be 18 ms, 20 ms, or the like.

[0070] If the first duration that the target video frame will stay in the first cache queue is greater than the first threshold, go to step 404 .

[0071] In some examples, if the PC receives the first target video frame, the second target video frame, and the third target video frame at 0ms, 7ms, and 15ms, respectively. When the first target video frame enters the first cache queue, since there are no other video frames in the first cache queue, the first target video frame can be directly sent to the display module for rendering and display. During the display of the first target video frame, the PC receives the second target video frame at 7ms, and at 7ms, the second target video frame is placed in the first cache queue. Since the first cache queue sends the cached video frames in the queue to the display module for rendering and display every 16.7ms, the second target video frame needs to stay in the first cache queue for 16.7ms-7ms=9.7ms. The PC receives the third target video frame at 15ms. At this time, the video frame displayed on the PC screen is the first target video frame, and there is a second target video frame waiting to be displayed in the first cache queue, so the third target video frame needs to stay in the first cache queue for 1.7ms+16.7ms=18.4ms. If the first threshold is 18ms, the PC discards the third target video frame.

[0072] It should be understood that the above 0th ms, 7th ms and 15th ms are only relative concepts and should not limit the present application. For other video frames after the third target video frame, please refer to the above description, which will not be repeated for brevity.

[0073] 403 : If the first duration that the target video frame will stay in the first cache queue is less than or equal to the first threshold, determine whether the cache amount of the first cache queue is greater than a second threshold.

[0074] Exemplarily, the second threshold may be 5 frames, 2 frames, 1 frame, etc.

[0075] When the first duration that the target video frame will stay in the first cache queue is less than or equal to the first threshold, and if the cache amount of the first cache queue is greater than the second threshold, go to step 404 .

[0076] In an embodiment of the present application, when the first cache queue has cached a number of video frames greater than the second threshold, it means that the target video frame needs to be rendered and displayed after the second threshold number of video frames. For example, the second threshold is 2 frames, the screen refresh frame rate of the PC is 60Hz, and the time for each refresh of the PC screen is 16.7ms. The number of video frames cached in the first cache queue is 3 frames, and the target video frame needs to be rendered and displayed after 50.1ms. Even if the target video frame enters the first cache queue, the video frame at the front of the first cache queue is sent for rendering and display, the target video frame still needs to be rendered and displayed after 33.4ms, and the delay is still high. The intuitive feeling brought to the user is that the picture is stuck. Therefore, in order to avoid or reduce the stuck phenomenon, the target video frame can be discarded.

[0077] The embodiment of the present application does not limit the order of step 402 and step 403. For example, step 403 may be performed before step 402, or step 402 and step 403 may be performed simultaneously.

[0078] It should be understood that step 402 and step 403 may be optional steps.

[0079] 404: discard the target video frame.

[0080] In a possible implementation, the target video frame is placed in a first cache queue, and if a first duration for which the target video frame will stay in the first cache queue is greater than a first threshold, the target video frame is discarded.

[0081] In the embodiment of the present application, when the first duration of the target video frame to be retained in the first cache queue is greater than the first threshold, it means that the target video frame needs to be rendered and displayed after a certain delay. If the delay is large, it will cause the screen to freeze. Discarding the target video frame can avoid or reduce the screen freeze, thereby ensuring the real-time and smoothness of the screen.

[0082] In another possible implementation, if the buffer amount of the first buffer queue is greater than the second threshold, the target video frame is discarded.

[0083] Specifically, if there are already several video frames waiting to be rendered and displayed in the first cache queue, when the PC receives the target video frame, if the number of video frames cached in the first cache queue is greater than a second threshold, for example, the second threshold is 2 frames, the target video frame is discarded.

[0084] 405 : If the number of video frames cached in the first cache queue is less than or equal to the second threshold, put the target video frame into the first cache queue.

[0085] In this step, if the cache amount of the first cache queue is less than or equal to the first threshold, the target video frame is placed in the first cache queue, and the target video frame can be sent to the display module for rendering and display more quickly, thereby reducing the delay of the target video frame.

[0086] 406 , the target video frame stays in the first cache queue for a first time period.

[0087] In this step, the first duration may be related to the number of video frames cached in the first cache queue.

[0088] Exemplarily, if the time between the PC receiving the target video frame and the previous frame is 8ms, the target video frame cannot be sent to the display module for display immediately because it does not match the refresh time of the PC screen. The PC can then put the target video frame into the first cache queue, and when the screen is refreshed next time, send the target video frame to the display module for display. In other words, the target video frame needs to stay in the first cache queue for 8.7ms.

[0089] Optionally, the above steps 401 to 406 may be executed by a display control module in a PC.

[0090] Optionally, in some embodiments, the PC may directly place the target video frame into the first cache queue, and then determine whether to discard the target video frame by determining the duration of the target video frame's stay in the first cache queue and the number of video frames cached in the first cache queue. If the duration of the target video frame's stay in the first cache queue is greater than a first threshold, the target video frame may be discarded; if the duration of its stay in the first cache queue is less than or equal to the first threshold, it may be determined whether the number of video frames cached in the first cache queue is greater than a second threshold; if it is greater than the second threshold, the target video frame may be discarded; if it is less than or equal to the second threshold, the target video frame may be placed in the first cache queue.

[0091] It should be understood that the present application does not limit the order of determining the relationship between the duration of the target video frame staying in the first cache queue and the first threshold, and the relationship between the number of video frames cached in the first cache queue and the second threshold.

[0092] 407, rendering the target video frame.

[0093] Exemplarily, in this step, the target video frame may be sent to a display module of a PC at an appropriate time, and the display module renders and displays the target video frame.

[0094] It should be understood that the embodiment of the present application does not limit the order of step 405 to step 407.

[0095] Based on the embodiment of the present application, the second electronic device can actively control the decoded target video frame, and by judging the residence time of the target video frame in the cache queue and the size of the cache in the queue, it can decide to discard the target video frame or put it into the cache queue. This technical solution can avoid or reduce the screen freeze and ensure the real-time and smoothness of the screen.

[0096] Figure 5 1 is a schematic flow chart of another electronic device displaying and controlling a video frame provided in an embodiment of the present application. The method may include steps 501 to 506. In the embodiment of the present application, the screen refresh frame rate of a PC is 60 Hz as an example, and the time interval between two refreshes of the PC screen is 16.7 ms.

[0097] 501, the PC takes out the target video frame to be rendered from the first cache queue.

[0098] It should be understood that in this embodiment, the decoded target video frame will first be sent to the first cache queue, and then the PC will take the target video frame out of the first cache queue and send it to the display module for rendering and display, or the target video frame will be discarded.

[0099] 502 , determining whether the residence time of the target video frame in the first cache queue is greater than a first threshold.

[0100] Exemplarily, the time t1 when the target video frame enters the first cache queue and the time t2 when the target video frame is taken out can be recorded, then the residence time of the target video frame in the first cache queue is t=t2-t1, when the time t is greater than the first threshold, the display control module discards the target video frame, when the time t is less than or equal to the first threshold, it will continue to judge the relationship between the number of video frames cached in the first cache queue and the second threshold.

[0101] The first threshold may be 18 ms, 20 ms, etc.

[0102] In this step, when the target video frame stays in the first cache queue for longer than the first threshold, it means that the target video frame has been waiting in the first cache queue for a long time. At this time, the target video frame can be discarded, which is beneficial to ensure the real-time performance of the screen projection.

[0103] 503 : If the residence time of the target video frame in the first cache queue is less than or equal to the first threshold, determine whether the cache amount of the first cache queue is greater than a second threshold.

[0104] In an example, if the residence time of the target video frame in the first cache queue is less than or equal to a first threshold, and the number of video frames cached in the first cache queue is greater than a second threshold, the target video frame is discarded.

[0105] In another example, if the target video frame stays in the first cache queue for a period of time that is less than or equal to a first threshold, and the number of video frames cached in the first cache queue is less than or equal to a second threshold, the target video frame is sent to the display module for rendering and display, or the target video frame is allowed to wait for a period of time before being sent to the display module for rendering and display.

[0106] Exemplarily, the second threshold may be 5 frames, 2 frames, 1 frame, etc.

[0107] It should be understood that the embodiment of the present application does not limit the order of steps 502 and 503. For example, step 503 may be performed before step 502, or steps 502 and 503 may be performed simultaneously.

[0108] It should be understood that step 502 and step 503 may be optional steps.

[0109] In this step, when the cache amount of the first cache queue is greater than the second threshold, it means that the number of cached video frames in the first cache queue is large, which may cause the subsequently decoded video frames to be unable to enter the first cache queue. Therefore, in this case, the target video frame is discarded, so that the number of cached video frames in the first cache queue is within an appropriate range, avoiding the subsequent decoded video frames from being unable to enter the first cache queue, thereby improving the real-time and smoothness of the screen projection.

[0110] 504, discard the target video frame.

[0111] In a possible implementation, if the residence time of the target video frame in the first cache queue is greater than a first threshold, the target video frame is discarded.

[0112] Exemplarily, if the display control module receives the first target video frame, the second target video frame and the third target video frame at 0ms, 7ms and 15ms respectively, the time when the first target video frame, the second target video frame and the third video frame enter and exit the first cache queue are recorded respectively. For the first target video frame, since there are no other video frames in the first cache queue when entering the first cache queue, the first target video frame is taken out when entering the first cache queue, that is, the length of time the first target video frame stays in the first cache queue is 0, and the number of video frames cached in the first cache queue is also 0 at this time, and the first target video frame is sent to the display module for rendering and display. During the display of the first target video frame, when the display control module receives the second target video frame at the 7th ms, the second target video frame is placed in the first cache queue. Since the video displayed on the PC screen at this time is the first target video frame, the second target video frame needs to stay in the first cache queue for 16.7ms-7ms=9.7ms. If the 9.7ms is greater than the first threshold, the second target video frame is discarded. If the 9.7ms is less than or equal to the first threshold, at this time, there are no other video frames in the first cache queue except the second target video frame, and the number of video frames cached in the first cache queue is 0. The second target video frame is sent to the display module for rendering and display after waiting for a period of time. The display control module receives the third target video frame at 15ms. At this time, the video frame displayed on the PC screen is the first target video frame. There is a second target video frame waiting to be displayed in the first cache queue. The third target video frame needs to stay in the first cache queue for 1.7ms+16.7ms=18.4ms. If the 18.4ms is greater than the first threshold, the third target video frame is discarded. If the 18.4ms is less than or equal to the first threshold, the cached video frame in the first cache queue is the second target video frame, that is, the number of cached video frames is 1 frame. At this time, the relationship between 1 and the second threshold can be judged. If 1 is greater than the second threshold, the third target video frame is discarded.

[0113] It should be understood that the above-mentioned 0th ms, 7th ms and 15th ms are merely relative concepts and should not impose any limitation on the present application.

[0114] For other video frames after the third target video frame, please refer to the above description, which will not be repeated for the sake of brevity.

[0115] 505 , if the number of video frames cached in the first cache queue is less than or equal to the second threshold, sending the target video frame to the display module after the second time period.

[0116] In this step, the target video frame is sent to the display module for rendering and display after the second duration. The second duration may also be 0.

[0117] For example, if the rendering time of the previous frame from the target video frame to the target video frame is 8ms when the PC receives the target video frame, the target video frame cannot be sent to the display module for display immediately because it does not match the refresh time of the PC screen. The PC may allow the target video frame to be sent to the display module for rendering and display after 8.7ms; or, the target video frame is first sent to the display module, but will not be rendered and displayed immediately, but will wait 8.7ms for rendering and display.

[0118] It should be understood that in this technical solution, the PC can call the system interface to send the target video frame to the display module, and the time interval for calling the system interface to send the target video frame to the display module is basically consistent with the refresh time interval of the PC screen. For example, the time interval can be expressed as: screen refresh time (16.7ms)-(timestamp of calling the sleep function-timestamp of calling the previous frame rendering).

[0119] It should be understood that the above steps 501 to 505 can be executed by the display control module of the PC.

[0120] 506, rendering the target video frame.

[0121] It should be understood that in this step, the target video frame can be sent to the display module of the PC at an appropriate time, and the display module renders and displays the target video frame.

[0122] Alternatively, the target video frame is directly sent to a display module, and the display module renders and displays the target video frame.

[0123] Optionally, step 506 may also be performed before step 505 .

[0124] It should be understood that in this technical solution, the PC can send the target video frame to the display module through the system interface, and the time interval for calling the system interface to send the target video frame to the display module is basically consistent with the refresh time interval of the PC screen. For example, the time interval can be expressed as: screen refresh time interval (16.7ms)-time required to call the system interface-x, where x is an adjustment value, generally a positive value, which is a value increased due to the inaccuracy of the sleep function.

[0125] Based on the embodiment of the present application, the electronic device can actively control the decoded target video frame, and by judging the residence time of the target video frame in the cache queue and the size of the cache in the queue, it can decide to discard the target video frame or send it to the display module for display. This technical solution can avoid or reduce the screen freeze and ensure the real-time and smoothness of the screen.

[0126] Figure 6 It is a schematic diagram of a display control method provided in an embodiment of the present application.

[0127] like Figure 6 As shown, taking the screen refresh frame rate of a PC as 60Hz as an example, the time interval between two refreshes of the PC screen is 16.7ms, that is, the time for each video frame to be displayed is 16.7ms. For receiving video frames, in order to match the refresh rate of the PC screen, ideally, the time interval between two video frames received by the PC is also 16.7ms, so that no frame loss can be achieved, and real-time performance and smoothness can also be guaranteed. However, due to various reasons, the time interval between two video frames received by the PC cannot be 16.7ms between every two video frames. For example, the duration of a certain video frame may be less than or equal to 16.7ms, or greater than 16.7ms.

[0128] For example, a video frame is received at the 7th ms, and the PC screen only displays the video frame for the next 16.7 ms. During this process, when the PC receives another video frame at the 15th ms, the newly received video frame will be discarded, resulting in frame loss.

[0129] See also Figure 6 During the screen projection process, for the video frames received by the PC, the 0th video frame can be understood as the initial video frame, that is, the initial video frame before the PC displays the screen projection content. The length of the initial video frame can be 16.7ms or may be greater than 16.7ms.

[0130] In one possible implementation, see Figure 6 In (a), before the target video frame enters the first cache queue, the PC can predetermine the length of time the target video frame will stay in the first cache queue and the number of video frames cached in the first cache queue to determine whether to put the target video frame into the first cache queue or discard it directly.

[0131] Specifically, the time interval between the first video frame and the second video frame received by the display control module is 16.7ms, and the display control module puts the first video frame into the first cache queue. Since there are no other video frames in the first cache queue except the first video frame, that is, the residence time of the first video frame in the first cache queue is 0, and the number of video frames cached in the first cache queue is 0, the first video frame in the first cache queue can be directly sent to the display module for rendering and display. The time interval between the second video frame and the third video frame received by the display control module is 7ms. At this time, the residence time of the second video frame in the first cache queue is 0, and the number of video frames cached in the first cache queue is 0, so the second video frame is put into the first cache queue, and then the second video frame is sent to the display module for rendering and display. During this process, the display control module receives the third video frame again. The time interval between the third video frame and the fourth video frame is 8ms. Since the video frame being displayed on the PC screen at this time is the second video frame, the third video frame will be cached in the first cache queue for 9.7ms at this time. The number of cached video frames in the first cache queue is 0. If the 9.7ms is less than the first threshold, the third video frame will be placed in the first cache queue, and when waiting for the next refresh of the PC screen, the third video frame will be sent to the display module for rendering and display. If during the caching process of the third video frame, the display control module receives the fourth video frame, and the video frame displayed on the PC screen is still the second video frame, and the third video frame is cached in the first cache queue, the display control module can decide whether to discard the fourth video frame or put it into the first cache queue based on the length of time the fourth video frame will stay in the first cache queue and the number of cached video frames in the first cache queue.

[0132] For example, if the display control module determines that the length of time that the fourth video frame will stay in the first cache queue is greater than the first threshold, the fourth video frame is discarded. For another example, if the display control module determines that the length of time that the fourth video frame will stay in the first cache queue is less than or equal to the first threshold, the number of cached video frames in the first cache queue is continued to be determined, and if the number of cached video frames in the first cache queue is greater than the second threshold, the fourth video frame is discarded.

[0133] In another possible implementation, see Figure 6 In (b), the PC directly puts the decoded video frame into the first cache queue, and can determine whether to send the target video frame to the display module for rendering and display, or directly discard it according to the length of time the target video frame stays in the first cache queue and the number of video frames cached in the first cache queue.

[0134] For example, the display control module of the PC can record the moment when the target video frame enters and exits the first cache queue, and then calculate the time that the target video frame stays in the first cache queue. If the time that the target video frame stays in the first cache queue is greater than the first threshold, the target video frame is discarded, that is, it is not sent to the display module for rendering and display. If the time that the target video frame stays in the first cache queue is less than or equal to the first threshold, it is necessary to continue to judge the relationship between the number of video frames cached in the first cache queue and the second threshold. If the number of video frames cached in the first cache queue is greater than the second threshold, the target video frame is discarded. If the number of video frames cached in the first cache queue is less than or equal to the second threshold, the target video frame is sent to the display module for rendering and display, or the target video frame is put to sleep and then sent to the display module for rendering and display.

[0135] Specifically, the display control module of the PC puts the received video frame into the first cache queue. Exemplarily, the time interval between the first video frame and the second video frame is 16.7ms. The display control module puts the first video frame into the first cache queue. Since there are no other video frames in the first cache queue except the first video frame, that is, the residence time of the first video frame in the first cache queue is 0, and the number of video frames cached in the first cache queue is 0, the first video frame in the first cache queue can be directly sent to the display module for rendering and display. The time interval between the second video frame and the third video frame received by the display control module is 7ms. At this time, the residence time of the second video frame in the first cache queue is 0, and the number of video frames cached in the first cache queue is 0, so the second video frame in the first cache queue can be directly sent to the display module for rendering and display. When the PC displays the second video frame, the display control module receives the third video frame, the fourth video frame and the fifth video frame, and the interval between the third video frame and the fourth video frame is 5ms, and the interval between the fourth video frame and the fifth video frame is 3ms. Since the PC screen refreshes a frame every 16.7ms, the third video frame will be cached in the first cache queue for 9.7ms, and the number of cached video frames in the first cache queue is 0. During the caching process of the 3rd video frame, the 4th video frame and the 5th video frame will be placed in the first cache queue. When the display control module sends the 3rd video frame to the display module, the relationship between the 9.7ms duration of the 3rd video frame staying in the first cache queue and the first threshold can be determined. If the 9.7ms is greater than the first threshold, the 3rd video frame is discarded. If the 9.7ms is less than or equal to the first threshold, the relationship between the number of video frames cached in the first cache queue and the second threshold is determined. At this time, the 4th video frame and the 5th video frame are cached in the first cache queue, and the number of cached video frames is 2. If 2 is greater than the second threshold, the 3rd video frame is discarded. If 2 is less than or equal to the second threshold, the 3rd video frame is sent to the display module for rendering and display.

[0136] It should be understood that the processing of other video frames after the third video frame can refer to the above description, which will not be repeated for the sake of brevity.

[0137] It should also be understood that in the above embodiment, the PC may first determine the relationship between the number in the first cache queue and the second threshold, and then determine the relationship between the residence time of the target video frame in the first cache queue and the first threshold, or determine both at the same time, and the embodiment of the present application is not limited to this.

[0138] In the embodiment of the present application, the PC can send the target video frame to the display module through the system interface.

[0139] Based on the embodiment of the present application, the PC can actively control the decoded video frame, and by judging the length of time the target video frame will stay in the cache queue and the size of the cache in the queue, it can decide to discard or display the video frame. This technical solution can avoid or reduce the screen freeze, and ensure the real-time and smoothness of the screen.

[0140] Based on the above embodiment, when the mobile phone and the PC are connected via WiFi-P2P, when the mobile phone projects the screen to the PC, taking the screen refresh frame rate of the PC as 60Hz as an example, the technical solution of the display control method of the present application can be applied to reduce the frame loss rate from 17.12% to 8.6%, that is, the PC can display 54 frames instead of 49 frames, thereby improving the real-time, clarity and smoothness of the screen projection.

[0141] It should be understood that in the embodiment of the present application, the screen refresh frame rate of the PC may also be 90 Hz or 120 Hz or other frame rates, and the embodiment of the present application is not limited to this.

[0142] Figure 7 It is a schematic diagram of another display control method provided in an embodiment of the present application.

[0143] like Figure 7 As shown, taking the screen refresh frame rate of the PC as 60Hz as an example, the time interval between two refreshes of the PC screen is 16.7ms, that is, the time for each video frame to be displayed is 16.7ms. For receiving video frames, in order to match the refresh rate of the PC screen, ideally, the time interval between two video frames received by the PC is also 16.7ms, so that no frame loss can be achieved, and real-time performance and fluency can also be guaranteed. However, due to various reasons, the time interval between two video frames received by the display control module of the PC cannot be 16.7ms between every two video frames. For example, the duration of a certain video frame may be less than or equal to 16.7ms, or greater than 16.7ms.

[0144] See also Figure 7 During the screen projection process, for the video frames received by the PC, the 0th video frame can be understood as the initial video frame, that is, the initial video frame before the PC displays the screen projection content. The length of the initial video frame can be 16.7ms or may be greater than 16.7ms.

[0145] In one possible implementation, before the target video frame enters the first cache queue, the PC can predetermine the length of time the target video frame will stay in the first cache queue and the number of video frames cached in the first cache queue to determine whether to place the target video frame in the first cache queue or discard it directly.

[0146] Specifically, the time interval between the first video frame and the second video frame received by the display control module of the PC is 16.7ms, and the display control module puts the first video frame into the first cache queue. Since there are no other video frames in the first cache queue except the first video frame, that is, the stay time of the first video frame in the first cache queue is 0, and the number of video frames cached in the first cache queue is 0, the first video frame in the first cache queue can be directly sent to the display module for rendering and display. After 16.7ms, the display control module receives the second video frame. The time interval between the second video frame received by the display control module and the third video frame is 30ms. When the display control module receives the second video frame, the residence time of the second video frame in the first cache queue is 0, and the number of video frames cached in the first cache queue is 0. It is placed in the first cache queue, and then the second video frame is sent to the display module for rendering and display. Since the PC screen is refreshed every 16.7ms, and the display control module receives the third video frame 30ms after receiving the second video frame, the second video frame will be displayed on the PC. Two 16.7ms durations are displayed on the screen. When the second video frame is in the process of displaying the second 16.7ms, when the display control module receives the third video frame, it determines that the current second video frame has been displayed for more than 16.7ms, then the third video frame will not stay in the cache queue, but will be directly sent to the next display module, and the display module will autonomously control the next screen refresh rendering. The display control module can determine the relationship between the duration of the third video frame's stay in the first cache queue and the first threshold and the number of video frames cached in the first cache queue, and determine whether to put the third video frame into the first cache queue or discard it directly. For other subsequent video frames, it is also necessary to determine whether to put the target video frame into the first cache queue or discard it directly based on the duration it will stay in the first cache queue and the number of video frames cached in the first cache queue. For the sake of brevity, it will not be repeated.

[0147] In another possible implementation, the PC directly places the decoded video frame into the first cache queue, and can determine whether to send the target video frame to the display module for rendering and display, or directly discard it based on the length of time the target video frame stays in the first cache queue and the number of video frames cached in the first cache queue.

[0148] Based on the embodiment of the present application, the PC can actively control the decoded video frames, and by judging the duration of the target video frame's stay in the cache queue and the size of the cache in the queue, it can decide whether to discard the video frame or send it to the display module for rendering and display. This technical solution can avoid or reduce screen freezes, and ensure the real-time and smoothness of the screen.

[0149] Figure 8 601 to 604.

[0150] 601, receiving a decoded target video frame.

[0151] Exemplarily, the WIFI module in the PC receives the video stream sent by the mobile phone and sends it to the image receiving module of the screen projection service of the PC. The image receiving module calls the decoder to decode the video frames in the video stream, and the display control module receives the video frames.

[0152] 602, sending the decoded target video frame to a display module for rendering and display.

[0153] 603 , determining an average time taken for decoding a third preset number of target video frames and sending them to a display module for rendering and display.

[0154] The third preset number of target video frames may be continuous video frames or spaced video frames.

[0155] Exemplarily, the third preset number may be 30 frames, 25 frames, etc.

[0156] 604. If the average time is greater than a third threshold, reduce the screen refresh frame rate of the PC.

[0157] For example, you can reduce the PC's screen refresh frame rate to half of its original value, or to two-thirds of its original value, etc.

[0158] Optionally, if the screen refresh frame rate of the PC has been reduced once, the screen refresh frame rate of the PC is kept unchanged even if the average time is greater than the third threshold.

[0159] Optionally, the method may further include steps 605 to 606:

[0160] 605 , executing step 603 once every a fourth preset number of target video frames.

[0161] In some embodiments, the fourth preset number can be 20 frames, the third preset number can be 30 frames, and step 603 records the average time from decoding to sending to the display module for rendering and display from the 1st target video frame to the 30th target video frame, and then step 604 is executed. Then, after an interval of 20 video frames, that is, starting from the 50th target video frame, the average time from decoding to sending to the display module for rendering and display from the 50th target video frame to the 80th target video frame is recorded, and then step 604 is executed, and the cycle is repeated.

[0162] It should be understood that when step 604 is executed again, if the PC is already in a frame reduction state, that is, the current screen refresh frame rate is already lower than the original screen refresh frame rate, there is no need to further reduce the screen refresh frame rate of the PC. Alternatively, multiple frame reductions may be performed.

[0163] The fourth preset number may also be 15 frames, etc.

[0164] Optionally, step 603 is performed once every certain period of time, for example, step 603 is performed once every 600ms or 500ms.

[0165] 606. If the average time is less than or equal to the fourth threshold, increase the screen refresh frame rate of the PC.

[0166] For example, the screen refresh frame rate of the PC may be increased to the original value, or a portion thereof, which is not limited in the embodiments of the present application.

[0167] Optionally, if the average time is less than or equal to the fourth threshold, it is possible to first determine whether the screen refresh frame rate of the PC is lower than the original screen refresh frame rate. If so, the screen refresh frame rate of the PC is increased; if not, the screen refresh frame rate of the PC is kept unchanged.

[0168] Based on the embodiment of the present application, when the average time taken for decoding and displaying multiple consecutive video frames is greater than a preset value, it means that the delay of the multiple video frames is large. At this time, the PC can reduce the screen refresh frame rate, that is, reduce the screen refresh frame rate, thereby avoiding excessive delay in screen projection and improving the real-time and smoothness of screen projection.

[0169] Fig. 9 is a schematic flow chart of a display control method provided in an embodiment of the present application. The method can be applied to a first electronic device, such as Fig. 9 As shown, the method may include steps 910 to 940.

[0170] 910. The first electronic device receives a video frame sent by the second electronic device.

[0171] The embodiment of the present application is explained by taking the example of projecting the screen of a second electronic device to a first electronic device. By way of example, the first electronic device may be the PC mentioned above, and the second electronic device may be the mobile phone mentioned above. When the mobile phone projects the screen to the PC, the PC receives the encoded video frames sent by the mobile phone.

[0172] 920: The first electronic device decodes the video frame to obtain a decoded target video frame.

[0173] It should be understood that the video frame received by the first electronic device is an encoded video frame, and the video frame can be decoded by a decoder to obtain a decoded target video frame.

[0174] For example, Figure 2 As shown, after the Wi-Fi module 240 in the PC receives the encoded video stream, it sends the encoded video stream to the image receiving module 250 of the PC's screen projection service. The image receiving module 250 calls the decoder 260 to decode the video stream (for example, calls the hardware processor at the chip layer for decoding).

[0175] 930: The first electronic device puts the target video frame into a first cache queue.

[0176] In some embodiments, the first electronic device directly puts the target video frame into the first cache queue. In other embodiments, the first electronic device may make some judgments before putting the target video frame into the first cache queue to determine whether to put the target video frame into the first cache queue.

[0177] 940. The first electronic device determines whether to render and display the target video frame according to a first preset condition, wherein a time interval for the first electronic device to render and display the target video frame is the same as a screen refresh time interval of the first electronic device.

[0178] In some embodiments, when the first electronic device directly places the target video frame into the first cache queue, the first preset condition may be the length of time the target video frame stays in the first cache queue, and / or, the first preset condition may be the number of target video frames cached in the first cache queue.

[0179] In other embodiments, when the first electronic device determines to place the target video frame into the first cache queue based on some conditions, the first preset condition may be the length of time that the target video frame stays in the first cache queue, that is, the first electronic device determines whether to render and display the target video frame immediately or to render and display the target video frame after caching for a certain period of time.

[0180] In this step, when the first electronic device determines to render and display the target video frame, the time interval for rendering and displaying the target video frame is the same as the screen refresh time interval of the first electronic device.

[0181] Exemplarily, the time interval at which the display control module in the first electronic device sends the target video frame to the display module for rendering and display is the same or substantially the same as the screen refresh time interval of the first electronic device. The display control module can send the target video frame to the display module for display by calling the system interface, and the time interval at which the display control module calls the system interface is the same or substantially the same as the screen refresh time interval of the first electronic device.

[0182] For example, see Figure 6 The video frames in the first cache queue are sent to the display module for display every 16.7 ms, and the screen refresh time interval of the PC is 16.7 ms.

[0183] Based on the embodiments of the present application, the first electronic device can actively control the decoded target video frame, and determine whether to render and display the target video frame according to preset conditions. When it is determined to render and display the target video frame, the time interval for rendering and displaying the target video frame by the first electronic device is the same or substantially the same as the time interval for refreshing its screen, thereby accurately controlling the target video frame and improving the display frame rate, real-time performance, clarity and smoothness of the projection screen.

[0184] Optionally, the first electronic device determines whether to render and display the target video frame according to a first preset condition, including: the first electronic device determines a first duration that the target video frame stays in the first cache queue; if the first duration is greater than a first threshold, the target video frame is discarded.

[0185] Exemplarily, the moment when the target video frame enters the first cache queue and the moment when it exits the first cache queue can be recorded, so as to determine the first duration of the target video frame's stay in the first cache queue, and determine whether the first duration is greater than a first threshold. When the first duration is greater than the first threshold, the target video frame is discarded. For example, the first threshold can be 18ms, 20ms, etc.

[0186] Based on the embodiment of the present application, when the first duration of the target video frame staying in the first cache queue is greater than the first threshold, it means that the target video frame stays in the first cache queue for a long time and the delay is large. At this time, the target video frame can be discarded, which is beneficial to ensure the real-time performance of the screen projection.

[0187] Optionally, the method also includes: if the first duration is less than or equal to the first threshold, the first electronic device determines a first number of video frames cached in the first cache queue; if the first number is greater than a second threshold, the target video frame is discarded; if the first number is less than or equal to the second threshold, the first electronic device determines to render and display the target video frame.

[0188] Exemplarily, the second threshold may be 5 frames, 2 frames, 1 frame, etc.

[0189] When the first number of video frames cached in the first cache queue is greater than the second threshold, it means that the number of video frames cached in the first cache queue is large and the delay is large, which may cause the subsequently decoded video frames to fail to enter the first cache queue. Therefore, in this case, the target video frame is discarded, so that the number of cached video frames in the first cache queue is within an appropriate range, avoiding the subsequent decoded video frames from failing to enter the first cache queue, thereby improving the real-time and smoothness of screen projection.

[0190] Optionally, the first electronic device determines to render and display the target video frame, including: the first electronic device renders and displays the target video frame after a second duration.

[0191] It should be understood that the second duration may also be 0. For example, if there are no other video frames in the cache queue except the target video frame, then the target video frame may be directly sent to the display module for rendering and display.

[0192] In some embodiments, the second duration may not be 0. For example, if there are other video frames in the cache queue before the target video frame, the target video frame will be sent to the display module for rendering and display after a period of time.

[0193] Based on the embodiment of the present application, when the first electronic device determines to render and display the target video frame, it may be rendered and displayed immediately, or it may be rendered and displayed after a period of time. The technical solution can actively control the rendering and display of the target video frame to improve the display frame rate of the projection screen.

[0194] Optionally, before the first electronic device places the target video frame into a first cache queue, the method also includes: the first electronic device determines a first duration that the target video frame needs to stay in the first cache queue; if the first duration is greater than a first threshold, discarding the target video frame.

[0195] Exemplarily, the first threshold may be 18 ms, 20 ms, etc.

[0196] In the embodiment of the present application, before the target video frame is placed in the first cache queue, the first duration for which it needs to stay in the first cache queue can be pre-determined, and when the first duration is greater than the first threshold, the target video frame can be discarded. This technical solution can prevent the target video frame from staying in the first cache queue for a long time and having a large delay, thereby facilitating the real-time performance of screen projection.

[0197] Optionally, the method also includes: if the first duration is less than or equal to a first threshold, the first electronic device determines a first number of video frames cached in the first cache queue; if the first number is greater than a second threshold, the target video frame is discarded; if the first number is less than or equal to the second threshold, the target video frame is placed in the first cache queue.

[0198] Exemplarily, the second threshold may be 5 frames, 3 frames, 2 frames, etc.

[0199] Based on the embodiment of the present application, when the cache amount of the first cache queue is greater than the second threshold, it means that the number of video frames cached in the first cache queue is large, which may cause the subsequently decoded video frames to be unable to enter the first cache queue. Therefore, in this case, the target video frame is discarded, so that the number of cached video frames in the first cache queue is within an appropriate range, avoiding the subsequent decoded video frames from being unable to enter the first cache queue, thereby improving the real-time and smoothness of the screen projection.

[0200] Optionally, the first electronic device determines whether to render and display the target video frame according to a first preset condition, including: after the target video frame stays in the first cache queue for the first period of time, the first electronic device renders and displays the target video frame.

[0201] It should be understood that the first duration may also be 0, for example, there is only the target video frame in the cache queue.

[0202] Based on the embodiment of the present application, after the target video frame is placed in the first cache queue, it can be sent to the display module of the first electronic device for rendering and display after the first duration. The technical solution can actively control the rendering and display of the target video frame to improve the display frame rate of the projection screen.

[0203] Optionally, the method also includes: the first electronic device determines a first average time taken for a third preset number of target video frames to be decoded, rendered and displayed on the screen of the first electronic device; if the first average time is greater than a third threshold, reducing the screen refresh frame rate of the first electronic device.

[0204] It should be understood that the third preset number may be 30, 20, etc., the third threshold may be 20ms, 18ms, etc., and the third preset number of target video frames may be continuous or intermittent.

[0205] Exemplarily, the first average time may be the average time of steps 603 and 604, and when the average time is greater than the third threshold, the first electronic device may reduce its screen refresh frame rate, for example, reducing the screen refresh frame rate to half or two-thirds of the original.

[0206] Based on the embodiments of the present application, when the average time taken for a certain number of target video frames to be decoded and displayed in the first electronic device is greater than the third threshold, it means that the delay of the multiple video frames is large. At this time, the first electronic device can downgrade the screen refresh frame rate, that is, reduce the screen refresh frame rate, thereby avoiding excessive delay in screen projection and improving the real-time and smoothness of screen projection.

[0207] Optionally, before reducing the screen refresh frame rate of the first electronic device, the method further includes: the first electronic device determining that a current screen refresh frame rate is at an original screen refresh frame rate.

[0208] Based on the embodiments of the present application, before the first electronic device reduces the screen refresh frame rate, it can be determined that the current screen refresh frame rate is at the original size. This technical solution can avoid the first electronic device reducing the frame rate multiple times, resulting in a low frame rate.

[0209] Optionally, if the screen refresh frame rate of the first electronic device has been reduced once, the screen refresh frame rate of the first electronic device is kept unchanged even if the first average time is greater than the third threshold.

[0210] Optionally, the method also includes: after an interval of a fourth preset number of target video frames, the first electronic device continues to determine a second average time for the third preset number of target video frames to be decoded, rendered and displayed on the screen of the first electronic device; if the second average time is less than or equal to a fourth threshold, increasing the screen refresh frame rate of the first electronic device.

[0211] The first preset number may be 20 frames, 15 frames, etc., and the fourth threshold may be 10ms, 15ms, etc.

[0212] In other embodiments, the first electronic device may continue to determine the second average time taken for decoding, rendering and displaying a third preset number of target video frames on the screen of the first electronic device after a preset interval. For example, the preset time may be 600ms, 500ms, etc.

[0213] In the embodiment of the present application, the screen refresh frame rate of the first electronic device can be increased to the original value, or a portion of it can be increased, and this is not limited in the embodiment of the present application.

[0214] Based on the embodiment of the present application, after an interval of a preset number of target video frames, the first electronic device continues to determine the average time taken for a third preset number of target video frames to be decoded, rendered and displayed on the screen of the first electronic device. When the average time is less than a fourth threshold, it means that the delay of the multiple frames of target video frames is small. At this time, the first electronic device can increase the screen refresh frame rate to display more video frames, thereby improving the clarity of the projected picture.

[0215] Optionally, before increasing the screen refresh frame rate of the first electronic device, the method further includes: the first electronic device determining that a current screen refresh frame rate is less than the original screen refresh frame rate.

[0216] Based on the embodiment of the present application, before the first electronic device increases the screen frame rate, it is necessary to determine that the current screen refresh frame rate is less than the original screen refresh frame rate.

[0217] An embodiment of the present application also provides an electronic device, comprising one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs comprising instructions, which, when executed by the one or more processors, enable the display control method described in any possible implementation described above to be executed.

[0218] An embodiment of the present application also provides a chip, which includes a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the display control method described in any possible implementation method in the foregoing text is executed.

[0219] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the display control method in the above-mentioned embodiment.

[0220] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the display control method in the above-mentioned embodiment.

[0221] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor may execute the computer execution instructions stored in the memory so that the chip executes the display control method in the above-mentioned method embodiments.

[0222] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0223] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0224] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0225] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0226] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0227] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0228] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0229] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A display control method, characterized in that: The method is applied to a first electronic device, and the method includes: The first electronic device receives a video frame sent by the second electronic device; The first electronic device decodes the video frame to obtain a decoded target video frame; The first electronic device places the target video frame into a first cache queue; The first electronic device determines whether to render and display the target video frame according to a first preset condition, wherein a time interval for the first electronic device to render and display the target video frame is the same as a screen refresh time interval of the first electronic device; The first electronic device determines whether to render and display the target video frame according to a first preset condition, including: The first electronic device determines a first duration that the target video frame stays in the first cache queue; If the first duration is greater than a first threshold, discarding the target video frame; The method further comprises: If the first duration is less than or equal to the first threshold, the first electronic device determines a first number of video frames cached in the first cache queue; If the first number is greater than a second threshold, discarding the target video frame; If the first number is less than or equal to a second threshold, the first electronic device determines to render and display the target video frame.

2. The method according to claim 1, characterized in that The first electronic device determines to render and display the target video frame, including: The first electronic device renders and displays the target video frame after a second duration.

3. The method according to claim 1, characterized in that Before the first electronic device places the target video frame into a first cache queue, the method further includes: The first electronic device determines a first duration for which the target video frame needs to stay in the first cache queue; If the first duration is greater than a first threshold, the target video frame is discarded.

4. The method according to claim 3, characterized in that The method further comprises: If the first duration is less than or equal to a first threshold, the first electronic device determines a first number of video frames cached in the first cache queue; If the first number is greater than a second threshold, discarding the target video frame; If the first number is less than or equal to a second threshold, the target video frame is placed in the first cache queue.

5. The method according to claim 3 or 4, characterized in that: The first electronic device determines whether to render and display the target video frame according to a first preset condition, including: After the target video frame stays in the first cache queue for the first time period, the first electronic device renders and displays the target video frame.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The first electronic device determines a first average time taken for a third preset number of target video frames to be decoded, rendered, and displayed on a screen of the first electronic device; If the first average time is greater than a third threshold, the screen refresh frame rate of the first electronic device is reduced.

7. The method according to claim 6, characterized in that Before reducing the screen refresh frame rate of the first electronic device, the method further includes: The first electronic device determines that the current screen refresh frame rate is at the original screen refresh frame rate.

8. The method according to claim 7, characterized in that The method further comprises: After a fourth preset number of target video frames, the first electronic device continues to determine a second average time taken for decoding, rendering and displaying the third preset number of target video frames on the screen of the first electronic device; If the second average time is less than or equal to a fourth threshold, the screen refresh frame rate of the first electronic device is increased.

9. The method according to claim 8, characterized in that Before increasing the screen refresh frame rate of the first electronic device, the method further includes: The first electronic device determines that a current screen refresh frame rate is less than the original screen refresh frame rate.

10. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more computer programs, the one or more computer programs include instructions, when the instructions are executed by one or more processors, the method of display control as described in any one of claims 1-9 is executed.

11. A chip, characterized in that: The chip includes a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the display control method according to any one of claims 1 to 9 is executed.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the display control method according to any one of claims 1 to 9 is executed.

13. A computer program product, characterized in that The invention comprises computer instructions, and when the computer instructions are executed on an electronic device, the display control method according to any one of claims 1 to 9 is executed.

Images