Screen projection delay measurement method, mobile terminal and computer-readable storage medium
By obtaining the numerical difference between the screen and calculating the device screen during the screen projection process, the accurate measurement and stability evaluation of the screen projection delay are achieved, the problem of inability to measure the screen projection delay in the existing technology is solved, and the screen projection is improved The synchronization of the process and the accuracy of network bandwidth adjustment.
Patent Information
- Application Number
- CN202110047129.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-14
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-01-14
AI Technical Summary
The prior art cannot accurately measure the screen projection delay, resulting in the problem that the device screen display is out of synchronization during the screen projection process.
By projecting the screen displayed on the local end to the receiving end, and increasing the value at every preset time, obtaining the first and second pictures of multiple moments, calculating the numerical difference of each moment, and obtaining the delay measurement value , and the video file can be taken by the camera device to extract pictures frame by frame, calculate the average value of the difference or remove the maximum and minimum value before calculating the delay measurement value.
Accurate measurement of screen projection delay is achieved, and network bandwidth can be increased according to the measured value to reduce display out-of-synchronization, and variance is calculated to determine screen projection stability, and a measurement report is generated.
Smart Images

Figure CN112866685B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communication technology, and in particular to a screen projection delay measurement method, a mobile terminal and a computer-readable storage medium. Background Art
[0002] Wireless screen projection is also called wireless screen sharing, flying screen, and screen sharing. Specifically, it is to display the screen of device A (such as mobile phone, tablet, notebook, computer) on the screen of another device B (tablet, notebook, computer, TV, all-in-one machine, projector) through some technical method, and the output content includes various media information and real-time operation screen.
[0003] Due to factors such as the network environment, the images displayed on the screens of device A and B may not be synchronized, that is, there is a delay in the screen projection process. Currently, people can only feel the image and sound delay through their five senses, but cannot measure the exact screen projection delay. Summary of the invention
[0004] The main purpose of the present invention is to provide a screen projection delay measurement method, a mobile terminal and a computer-readable storage medium, aiming to solve the technical problem that the accurate screen projection delay cannot be measured in the prior art.
[0005] To achieve the above object, the present invention provides a method for measuring screen projection delay, the method comprising:
[0006] Projecting the screen displayed on the local end to the receiving end, the screen including a numerical value, the numerical value increases by 1 every preset time period;
[0007] Obtaining a first picture displayed on the local end and a second picture displayed on the receiving end at multiple moments;
[0008] Extracting a first value from a first picture corresponding to each moment and extracting a second value from a second picture;
[0009] The difference between the first value and the second value corresponding to each moment is calculated to obtain a plurality of difference values, and the delay measurement value is calculated according to the plurality of difference values.
[0010] Optionally, the step of acquiring the first picture displayed on the local end and the second picture displayed on the receiving end at multiple moments includes:
[0011] Acquire a video file captured by a camera device, wherein the camera device is used to capture a first picture displayed on the local end and a second picture displayed on the receiving end in the same frame;
[0012] Extract each frame of the video file frame by frame;
[0013] The first picture displayed on the local end and the second picture displayed on the receiving end contained in each frame of picture are used as the first picture displayed on the local end and the second picture displayed on the receiving end at each moment.
[0014] Optionally, the step of calculating the delay measurement value according to the multiple difference values includes:
[0015] An average value of the multiple difference values is calculated, and the average value is used as the delay measurement value.
[0016] Optionally, the step of calculating the delay measurement value according to the multiple difference values includes:
[0017] The maximum value and the minimum value among the multiple differences are eliminated, and the average value of the remaining differences is calculated, and the average value is used as the delay measurement value.
[0018] Optionally, after the step of calculating the delay measurement value according to the multiple difference values, the method further includes:
[0019] Detecting whether the delay measurement value is greater than a preset threshold;
[0020] If it is greater than the preset threshold, the network bandwidth for screen projection will be increased.
[0021] Optionally, after the step of calculating the delay measurement value according to the multiple difference values, the method further includes:
[0022] The variance of the multiple difference values is calculated, and the projection stability is determined according to the variance.
[0023] Optionally, after the step of determining the projection stability according to the variance, the method further includes:
[0024] Get the screen projection bitrate;
[0025] A measurement report is obtained based on the projection bit rate, delay measurement value, and projection stability.
[0026] In addition, to achieve the above-mentioned purpose, the present invention also provides a mobile terminal, which includes: a memory, a processor, and a screen projection delay measurement program stored in the memory and runnable on the processor, and the screen projection delay measurement program, when executed by the processor, implements the steps of the screen projection delay measurement method as described above.
[0027] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a screen projection delay measurement program is stored. When the screen projection delay measurement program is executed by a processor, the steps of the screen projection delay measurement method as described above are implemented.
[0028] In the present invention, a screen displayed on the local end is projected to the receiving end, the screen includes a numerical value, and the numerical value increases by 1 every preset time; the first screen displayed on the local end and the second screen displayed on the receiving end at multiple moments are obtained; the first numerical value is extracted from the first screen corresponding to each moment and the second numerical value is extracted from the second screen; the difference between the first numerical value and the second numerical value corresponding to each moment is calculated to obtain multiple differences, and the delay measurement value is calculated based on the multiple differences. Through the present invention, accurate measurement of screen projection delay is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present invention;
[0030] Figure 2 A communication network system architecture diagram provided for an embodiment of the present invention;
[0031] Figure 3 It is a flow chart of an embodiment of a method for measuring screen projection delay of the present invention;
[0032] Figure 4 This is a schematic diagram of a scene in which a camera device shoots the local end and the receiving end in one embodiment of the screen projection delay measurement method of the present invention.
[0033] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0034] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0035] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.
[0036] In the embodiment of the present invention, the screen projection delay measurement method is applied to a mobile terminal, which can be implemented in various forms. For example, the mobile terminal involved in the present invention may include a mobile terminal such as a mobile phone, a tablet computer, a laptop computer, a PDA, a personal digital assistant (PDA), etc.
[0037] The following description will be made by taking a tablet computer as an example. It will be understood by those skilled in the art that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present invention can also be applied to other types of mobile terminals.
[0038] See also Figure 1 , Figure 1 To implement the hardware structure diagram of the mobile terminal of each embodiment of the present invention, the terminal 100 may include: RF (Radio Frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111. Those skilled in the art can understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0039] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:
[0040] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution) and TDD-LTE (Time Division Duplexing-Long Term Evolution), etc.
[0041] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.
[0042] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.
[0043] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.
[0044] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the terminal posture (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the terminal, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.
[0045] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.
[0046] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the terminal. Specifically, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.
[0047] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of the touch event. Then, the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.
[0048] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.
[0049] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0050] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. By running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, it executes various functions of the mobile terminal and processes data, thereby monitoring the main screen page display terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.
[0051] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.
[0052] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.
[0053] like Figure 1 As shown, the memory 109 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a screen projection delay measurement program. The processor 110 may be used to call the screen projection delay measurement program stored in the memory 109 and perform the following steps:
[0054] Projecting the screen displayed on the local end to the receiving end, the screen including a numerical value, the numerical value increases by 1 every preset time period;
[0055] Obtaining a first picture displayed on the local end and a second picture displayed on the receiving end at multiple moments;
[0056] Extracting a first value from a first picture corresponding to each moment and extracting a second value from a second picture;
[0057] The difference between the first value and the second value corresponding to each moment is calculated to obtain a plurality of difference values, and the delay measurement value is calculated according to the plurality of difference values.
[0058] Further, the processor 110 may be used to call the screen projection delay measurement program stored in the memory 109, and further perform the following steps:
[0059] Acquire a video file captured by a camera device, wherein the camera device is used to capture a first picture displayed on the local end and a second picture displayed on the receiving end in the same frame;
[0060] Extract each frame of the video file frame by frame;
[0061] The first picture displayed on the local end and the second picture displayed on the receiving end contained in each frame of picture are used as the first picture displayed on the local end and the second picture displayed on the receiving end at each moment.
[0062] Further, the processor 110 may be used to call the screen projection delay measurement program stored in the memory 109, and further perform the following steps:
[0063] An average value of the multiple difference values is calculated, and the average value is used as the delay measurement value.
[0064] Further, the processor 110 may be used to call the screen projection delay measurement program stored in the memory 109, and further perform the following steps:
[0065] The maximum value and the minimum value among the multiple differences are eliminated, and the average value of the remaining differences is calculated, and the average value is used as the delay measurement value.
[0066] Further, the processor 110 may be used to call the screen projection delay measurement program stored in the memory 109, and further perform the following steps:
[0067] Detecting whether the delay measurement value is greater than a preset threshold;
[0068] If it is greater than the preset threshold, the network bandwidth for screen projection will be increased.
[0069] Further, the processor 110 may be used to call the screen projection delay measurement program stored in the memory 109, and further perform the following steps:
[0070] The variance of the multiple difference values is calculated, and the projection stability is determined according to the variance.
[0071] Further, the processor 110 may be used to call the screen projection delay measurement program stored in the memory 109, and further perform the following steps:
[0072] Get the screen projection bitrate;
[0073] A measurement report is obtained based on the projection bit rate, delay measurement value, and projection stability.
[0074] To facilitate understanding of the embodiments of the present invention, the communication network system on which the mobile terminal of the present invention is based is described below.
[0075] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present invention. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are sequentially connected for communication.
[0076] Specifically, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.
[0077] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Among them, eNodeB 2021 can be connected to other eNodeBs 2022 through a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 can provide UE 201 with access to EPC 203 .
[0078] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Among them, MME2031 is a control node that processes signaling between UE201 and EPC203, and provides bearer and connection management. HSS2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and save some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).
[0079] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.
[0080] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present invention is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems, etc., which are not limited here.
[0081] Based on the above mobile terminal hardware structure and communication network system, various embodiments of the method of the present invention are proposed.
[0082] Reference Figure 3 , Figure 3 FIG. 1 is a flow chart of an embodiment of a method for measuring screen projection delay according to the present invention. Figure 3As shown, in one embodiment, the screen projection delay measurement method includes:
[0083] Step S10, projecting the screen displayed on the local end to the receiving end, wherein the screen includes a numerical value, and the numerical value increases by 1 every preset time period;
[0084] In this embodiment, this end refers to the executor of the screen projection delay measurement method, such as a mobile terminal. A screen projection connection is established between the local end and the receiving end, and the picture displayed on the local end is projected to the receiving end, wherein the picture displayed on the local end includes a numerical value, and the numerical value increases by 1 every preset time. Specifically, a numerical value that increases by 1 every preset time can be drawn on the original display screen of the local end through the floatwindow technology. The preset time is set according to actual needs, for example, it is set to 1 millisecond. That is, when the local end is displaying the picture, there is a numerical value in the picture that increases with time.
[0085] Step S20, obtaining a first picture displayed on the local end and a second picture displayed on the receiving end at multiple moments;
[0086] In this embodiment, during the screen projection process, the first picture displayed on the local end and the second picture displayed on the receiving end at multiple moments are obtained. For example, at time t1, the display screen of the local end and the display screen of the receiving end are framed together, so that the first picture displayed on the local end at time t1 and the second picture displayed on the receiving end can be obtained; similarly, at time t2, the display screen of the local end and the display screen of the receiving end are framed together, so that the first picture displayed on the local end at time t2 and the second picture displayed on the receiving end can be obtained, and so on, so that the first picture displayed on the local end at multiple moments and the second picture displayed on the receiving end can be obtained.
[0087] Furthermore, in one embodiment, step S20 includes:
[0088] Acquire a video file captured by a camera device, wherein the camera device is used to capture a first picture displayed on the local end and a second picture displayed on the receiving end in the same frame; extract each frame of the video file frame by frame; and use the first picture displayed on the local end and the second picture displayed on the receiving end contained in each frame as the first picture displayed on the local end and the second picture displayed on the receiving end at each moment.
[0089] In this embodiment, refer to Figure 4 , Figure 4 This is a schematic diagram of a scene in which a camera device shoots a local end and a receiving end in one embodiment of a method for measuring screen projection delay of the present invention. Figure 4As shown, the local display screen and the receiving end display screen are placed side by side, and the two are photographed in the same frame by a camera device, so that the first picture displayed on the local end and the second picture displayed on the receiving end can be photographed in the same frame. The video file photographed by the camera device is obtained, and then each frame of the video file is extracted frame by frame. Since each frame of the picture contains the first picture displayed on the local end and the second picture displayed on the receiving end, the first picture displayed on the local end and the second picture displayed on the receiving end contained in each frame of the picture can be used as the first picture displayed on the local end and the second picture displayed on the receiving end at each moment. Among them, each moment is the recording moment of each frame of the picture.
[0090] Step S30, extracting a first value from the first picture corresponding to each moment and extracting a second value from the second picture;
[0091] In this embodiment, the first picture is grayed and binarized, and then the processed picture is subjected to text OCR recognition to obtain the first value in the first picture. Similarly, the second value is extracted from the second picture in the same manner. Of course, considering that if there is interference from other values in the picture, it will lead to the extraction of the wrong first value / second value. In order to avoid this situation, a picture without a value can be selected as the original picture, and then a value that increases over time can be added to the original picture to form the final picture that needs to be projected.
[0092] Step S40, calculating the difference between the first value and the second value corresponding to each moment to obtain a plurality of differences, and calculating the delay measurement value according to the plurality of differences.
[0093] In this embodiment, the first value and the second value corresponding to each moment can be obtained according to step S30, for example, the first value A1 and the second value B1 corresponding to t1, the first value A2 and the second value B2 corresponding to t2, the first value A3 and the second value B3 corresponding to t3, ..., the first value An and the second value Bn corresponding to tn. The difference corresponding to t1 is A1 minus B1, the difference corresponding to t2 is A2 minus B2, and so on, multiple differences can be obtained, so as to obtain the delay measurement value based on the multiple differences.
[0094] In this embodiment, the screen displayed on the local end is projected to the receiving end, and the screen includes a numerical value, and the numerical value increases by 1 every preset time; the first screen displayed on the local end and the second screen displayed on the receiving end at multiple moments are obtained; the first numerical value is extracted from the first screen corresponding to each moment and the second numerical value is extracted from the second screen; the difference between the first numerical value and the second numerical value corresponding to each moment is calculated to obtain multiple differences, and the delay measurement value is calculated based on the multiple differences. Through this embodiment, accurate measurement of screen projection delay is achieved.
[0095] Furthermore, in one embodiment, the step of calculating the delay measurement value according to the multiple difference values includes:
[0096] An average value of the multiple difference values is calculated, and the average value is used as the delay measurement value.
[0097] In this embodiment, the average value of multiple differences is calculated, and the calculated average value is used as the delay measurement value. For example, if the value increases by 1 every 1 millisecond, if the calculated average value is 15, then the delay measurement value is 15, indicating that the screen projection delay is 15 milliseconds. Similarly, if the value increases by 1 every 2 milliseconds, if the calculated average value is 15, then the delay measurement value is 15, indicating that the screen projection delay is 30 milliseconds.
[0098] Furthermore, in one embodiment, the step of calculating the delay measurement value according to the multiple difference values includes:
[0099] The maximum value and the minimum value among the multiple differences are eliminated, and the average value of the remaining differences is calculated, and the average value is used as the delay measurement value.
[0100] In this embodiment, in order to make the delay measurement value finally calculated more representative, the maximum value and the minimum value among the multiple differences are firstly eliminated, and then the average value of the remaining differences is calculated, and the calculated average value is used as the delay measurement value.
[0101] Furthermore, in one embodiment, after step S40, the method further includes:
[0102] Detect whether the delay measurement value is greater than a preset threshold; if greater than the preset threshold, increase the network bandwidth of the screen projection.
[0103] In this embodiment, considering that when the screen projection delay is small, there is no need to adjust the network bandwidth of the screen projection. Only when the screen projection delay is relatively large, it means that the display asynchronism between the local end and the receiving end is more prominent, and the network bandwidth of the screen projection needs to be adjusted. Therefore, a preset threshold can be set according to actual needs. If the delay measurement value is greater than the preset threshold, it means that the display asynchronism between the local end and the receiving end is more prominent, and the network bandwidth of the screen projection needs to be increased. Specifically, the network bandwidth of the screen projection can be increased according to a fixed value.
[0104] Furthermore, in one embodiment, after step S40, the method further includes:
[0105] The variance of the multiple difference values is calculated, and the projection stability is determined according to the variance.
[0106] In this embodiment, the variance of multiple difference values can also be calculated, and the projection stability can be determined according to the numerical range of the variance. For example, two numerical ranges are preset, corresponding to the unstable state and the stable state respectively. If the variance is in the numerical range corresponding to the unstable state, it is determined that the current projection is in an unstable state; if the variance is in the numerical range corresponding to the stable state, it is determined that the current projection is in a stable state.
[0107] Furthermore, in one embodiment, after the step of determining the projection stability according to the variance, the method further includes:
[0108] Get the screen projection bit rate; get a measurement report based on the screen projection bit rate, delay measurement value, and screen projection stability.
[0109] In this embodiment, the screen projection bit rate can be set before screen projection. After obtaining the delay measurement value and determining the screen projection stability, the screen projection bit rate, the delay measurement value and the screen projection stability can be combined to obtain a measurement report, which is used to indicate the delay and stability of the screen projection at the screen projection bit rate. It is easy to understand that by setting different screen projection bit rates before screen projection and executing this embodiment, the delay and stability of the screen projection at various screen projection bit rates can be obtained.
[0110] In addition, an embodiment of the present invention also proposes a computer-readable storage medium, on which a screen projection delay measurement program is stored. When the screen projection delay measurement program is executed by a processor, the above screen projection delay measurement method steps are implemented.
[0111] Among them, the specific steps implemented when the screen projection delay measurement program is executed by the processor are basically the same as the various embodiments of the above-mentioned screen projection delay measurement method, and will not be repeated here.
[0112] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0113] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0115] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for measuring screen projection delay, characterized in that: The screen projection delay measurement method includes: Casting a screen displayed on the local end to the receiving end, the screen including a numerical value, the numerical value increases by 1 every preset time, wherein the preset time is 1 millisecond; Obtaining a first picture displayed on the local end and a second picture displayed on the receiving end at multiple moments; Extracting a first value from the first picture corresponding to each moment and extracting a second value from the second picture, wherein the first picture and the second picture are grayed and binarized, and text recognition is performed on the processed pictures to obtain a first value in the first picture and a second value in the second picture; wherein, if the picture is interfered by other values, a picture without a value is selected as the original picture, and a value that increases with time is added to the original picture to form the picture that needs to be projected; Calculate the difference between the first value and the second value corresponding to each moment to obtain multiple differences, and calculate the delay measurement value according to the multiple differences; The step of obtaining the first picture displayed on the local end and the second picture displayed on the receiving end at multiple moments includes: Acquire a video file captured by a camera device, wherein the camera device is used to capture a first picture displayed on the local end and a second picture displayed on the receiving end in the same frame; Extract each frame of the video file frame by frame; The first picture displayed on the local end and the second picture displayed on the receiving end contained in each frame of the picture are used as the first picture displayed on the local end and the second picture displayed on the receiving end at each moment; The step of calculating the delay measurement value according to the multiple difference values comprises: Calculate the average value of multiple differences and use the average value as the delay measurement value, Alternatively, the maximum value and the minimum value among the multiple differences are eliminated, and the average value of the remaining differences is calculated, and the average value is used as the delay measurement value; After the step of calculating the delay measurement value according to the multiple difference values, the method further includes: Detecting whether the delay measurement value is greater than a preset threshold; If it is greater than the preset threshold, the network bandwidth for screen projection will be increased; Calculating the variance of the multiple difference values, and determining the projection stability according to the variance; Get the screen projection bitrate; Get a measurement report based on the projection bitrate, delay measurement value, and projection stability; in, Determine the projection stability according to the numerical range of the variance; including: Two numerical intervals are preset, corresponding to an unstable state and a stable state respectively. If the variance is in the numerical interval corresponding to the unstable state, it is determined that the current screen projection is in an unstable state; if the variance is in the numerical interval corresponding to the stable state, it is determined that the current screen projection is in a stable state.
2. A mobile terminal, characterized in that: The mobile terminal includes: a memory, a processor, and a screen projection delay measurement program stored in the memory and executable on the processor, wherein the screen projection delay measurement program implements the following steps when executed by the processor: Projecting the screen displayed on the local end to the receiving end, the screen including a numerical value, the numerical value increases by 1 every preset time period; Obtaining a first picture displayed on the local end and a second picture displayed on the receiving end at multiple moments; Extracting a first value from the first picture corresponding to each moment and extracting a second value from the second picture, wherein the first picture and the second picture are grayed and binarized, and text recognition is performed on the processed pictures to obtain a first value in the first picture and a second value in the second picture; wherein, if the picture is interfered by other values, a picture without a value is selected as the original picture, and a value that increases with time is added to the original picture to form the picture that needs to be projected; Calculate the difference between the first value and the second value corresponding to each moment to obtain multiple differences, and calculate the delay measurement value according to the multiple differences; The step of obtaining the first picture displayed on the local end and the second picture displayed on the receiving end at multiple moments includes: Acquire a video file captured by a camera device, wherein the camera device is used to capture a first picture displayed on the local end and a second picture displayed on the receiving end in the same frame; Extract each frame of the video file frame by frame; The first picture displayed on the local end and the second picture displayed on the receiving end contained in each frame of the picture are used as the first picture displayed on the local end and the second picture displayed on the receiving end at each moment; The step of calculating the delay measurement value according to the multiple difference values comprises: Calculate the average value of multiple differences and use the average value as the delay measurement value, Alternatively, the maximum value and the minimum value among the multiple differences are eliminated, and the average value of the remaining differences is calculated, and the average value is used as the delay measurement value; After the step of calculating the delay measurement value according to the multiple difference values, the method further includes: Detecting whether the delay measurement value is greater than a preset threshold; If it is greater than the preset threshold, the network bandwidth for screen projection will be increased; Calculating the variance of the multiple difference values, and determining the projection stability according to the variance; Get the screen projection bitrate; Get a measurement report based on the projection bitrate, delay measurement value, and projection stability; in, Determine the projection stability according to the numerical range of the variance; including: Two numerical intervals are preset, corresponding to an unstable state and a stable state respectively. If the variance is in the numerical interval corresponding to the unstable state, it is determined that the current screen projection is in an unstable state; if the variance is in the numerical interval corresponding to the stable state, it is determined that the current screen projection is in a stable state.
3. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a screen projection delay measurement program, and when the screen projection delay measurement program is executed by the processor, the steps of the screen projection delay measurement method according to claim 1 are implemented.
