Screen projection data failure analysis method, screen projection method, device, equipment and medium

Abstract

The application discloses a screen projection data fault analysis method, a screen projection method, a screen projection device, a screen projection equipment and a medium. When screen projection is stuck, current log information of a screen projection application is acquired, a first current log corresponding to a video stream and a second current log corresponding to an audio stream are determined from the current log information, at least one abnormal identifier is acquired, a first current data value corresponding to each abnormal identifier is determined from the first current log, a second current data value corresponding to each abnormal identifier is determined from the second current log, the first current data value and the second current data value corresponding to any abnormal identifier are compared, a fault result is obtained, the influence factors of the sticking are automatically discovered through comparative analysis of the video stream and the audio stream in the log, the real-time performance is improved, the data stream of the screen projection is adjusted in combination with the fault result, the influence of the influence factors on the sticking and the occurrence of the screen projection sticking condition are reduced, and better screen projection experience can be brought to the user.

Description

Technical Field

[0001] This invention relates to the field of communications, and in particular to a fault analysis method, projection method, device, equipment, and medium for projected data. Background Technology

[0002] With the continuous development of network technology, screen sharing between multiple devices has become increasingly convenient for users. This sharing typically involves projecting the screen from one device to another, such as screen mirroring or DLNA (Digital Living Network Alliance) projection. However, in practice, screen sharing often experiences lag. Because screen sharing involves a large amount of data, many factors can contribute to lag, and these factors can vary at different times. While manual detection can identify some prominent and persistent lag issues, it's impossible for users to intervene manually during use. The lack of real-time detection of lag factors leads to a poor user experience. Therefore, automating the detection of lag factors and adjusting them to reduce screen sharing lag is a pressing issue. Summary of the Invention

[0003] Based on this, a fault analysis method, projection method, device, equipment and medium for screen projection data is provided to solve the problem of how to automatically discover the influencing factors of lag, thereby helping to adjust the corresponding influencing factors and reduce the occurrence of screen projection lag.

[0004] In a first aspect, embodiments of the present invention provide a method for fault analysis of screen projection data, the fault analysis method comprising the following steps:

[0005] When a stutter occurs when the screen is cast from the client to the device, the current log information generated by the corresponding screen casting application in the client is obtained, and the first current log corresponding to the video stream and the second current log corresponding to the audio stream are determined from the current log information.

[0006] Obtain at least one preset exception identifier, determine the first current data value corresponding to each exception identifier from the first current log, and determine the second current data value corresponding to each exception identifier from the second current log;

[0007] For any anomaly identifier, compare the first current data value and the second current data value corresponding to the anomaly identifier to obtain a first comparison result. If the comparison result is different, then the anomaly identifier is determined to be a fault result.

[0008] Secondly, embodiments of the present invention provide a screen mirroring method, the screen mirroring method comprising the following steps:

[0009] Obtain all fault results and the real-time data stream of the current screen projection;

[0010] For any fault result, a first real-time data value corresponding to the fault result is determined from the video stream of the real-time data stream, and a second real-time data value corresponding to the fault result is determined from the audio stream of the real-time data stream.

[0011] The first real-time data value and the second real-time data value are unified to obtain a unified data value. The unified data value is used to update the real-time video stream to obtain an updated data stream, which is then sent to the device.

[0012] Thirdly, embodiments of the present invention provide a fault analysis device for screen projection data, comprising:

[0013] The current log determination module is used to obtain the current log information generated by the corresponding screen casting application in the client when the screen casting from the client to the device is stuck, and determine the first current log corresponding to the video stream and the second current log corresponding to the audio stream from the current log information;

[0014] The current data value determination module is used to obtain at least one preset anomaly identifier, determine the first current data value corresponding to each anomaly identifier from the first current log, and determine the second current data value corresponding to each anomaly identifier from the second current log;

[0015] The first comparison result determination module is used to compare the first current data value and the second current data value corresponding to any anomaly identifier to obtain a first comparison result. If the comparison result is different, the anomaly identifier is determined to be a fault result.

[0016] Fourthly, embodiments of the present invention provide a screen projection device, comprising:

[0017] The acquisition module is used to acquire all fault results and the real-time data stream of the current screen projection;

[0018] The real-time data value determination module is used to determine, for any fault result, a first real-time data value corresponding to the fault result from the video stream of the real-time data stream, and a second real-time data value corresponding to the fault result from the audio stream of the real-time data stream.

[0019] The data stream update module is used to unify the first real-time data value and the second real-time data value to obtain a unified data value, use the unified data value to update the real-time video stream to obtain an updated data stream, and send the updated data stream to the device.

[0020] Fifthly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described screen projection data fault analysis method.

[0021] In a sixth aspect, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the screen projection method described above.

[0022] In a seventh aspect, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described fault analysis method for screen projection data.

[0023] Eighthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described screen projection method.

[0024] The technical effects achieved by this invention, which differ from existing technologies, are as follows: When screen mirroring from the client to the device experiences lag, this invention obtains the first current log corresponding to the video stream and the second current log corresponding to the audio stream in the client's screen mirroring application, and acquires at least one preset anomaly identifier. It determines the first current data value corresponding to each anomaly identifier from the first current log, and the second current data value corresponding to each anomaly identifier from the second current log. The first and second current data values ​​corresponding to each anomaly identifier are compared; if the comparison results are different, the anomaly identifier is determined to be a fault result. Specifically, by comparing and analyzing the video and audio streams in the logs when screen mirroring lags, the fault result corresponding to the lag is automatically determined, achieving automated discovery of the factors affecting lag and improving the real-time nature of lag factor discovery. Furthermore, by adjusting the screen mirroring data stream based on the fault result, the impact of influencing factors on lag is reduced, thus reducing the occurrence of screen mirroring lag and providing users with a better screen mirroring experience. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of an application environment for a screen projection data fault analysis method provided in Embodiment 1 of the present invention;

[0027] Figure 2 This is a flowchart illustrating a method for fault analysis of screen projection data provided in Embodiment 1 of the present invention;

[0028] Figure 3 This is a flowchart illustrating a method for fault analysis of screen projection data provided in Embodiment 2 of the present invention;

[0029] Figure 4 This is a flowchart illustrating a method for fault analysis of screen projection data provided in Embodiment 3 of the present invention;

[0030] Figure 5 This is a flowchart illustrating a method for fault analysis of screen projection data provided in Embodiment 4 of the present invention;

[0031] Figure 6 This is a flowchart illustrating a screen projection method provided in Embodiment 5 of the present invention;

[0032] Figure 7 This is a schematic diagram of the structure of a screen projection data fault analysis device provided in Embodiment 7 of the present invention;

[0033] Figure 8 This is a schematic diagram of the structure of a projection device provided in Embodiment 8 of the present invention;

[0034] Figure 9 This is a schematic diagram of the structure of a computer device provided in Embodiment 9 of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] The fault analysis method for screen projection data provided in this application can be applied to, for example... Figure 1In this application environment, the client communicates with the device via a network. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The device can be a terminal with a screen that can communicate with the client, receiving and displaying screen projection data sent by the client, thus enabling screen projection from the client to the device. This device can include, but is not limited to, smart TVs, smart screens, and other computer devices capable of audio and video display.

[0037] like Figure 2 The diagram shown is a flowchart illustrating a screen projection data fault analysis method provided in Embodiment 1 of the present invention. This embodiment provides a screen projection data fault analysis method, which is applied to... Figure 1 Taking the client as an example, the steps may include the following:

[0038] Step S201: When there is a lag when the client casts the screen to the device, obtain the current log information generated by the corresponding screen casting application in the client, and determine the first current log corresponding to the video stream and the second current log corresponding to the audio stream from the current log information.

[0039] In this embodiment, the client can be a mobile terminal used by the user, and the device can be a TV. The mobile terminal has a corresponding application installed. The application can be run to obtain the corresponding source data. If the application is cast to the screen, it is determined to be a casting reference. For example, the application is a video application (APP). The video APP can obtain the corresponding playback source to play video and audio. Under certain network conditions (such as being in the same local area network), the content played in the video APP can be cast to the device for playback.

[0040] During screen mirroring, video or audio stuttering may occur. Video stuttering can manifest as video interruption or freezing, while audio stuttering can manifest as sound interruption. When stuttering is detected when the client is mirroring to the device, the cause of the stuttering needs to be analyzed in order to obtain a corresponding solution.

[0041] When screen mirroring experiences lag, the log information generated by the screen mirroring application within a certain period of time after the lag occurs is retrieved from the application's log directory. The corresponding time period is recorded as the current moment, and the corresponding log information is recorded as the current log information. For screen mirroring of the playback source, this includes video data and audio data. The generated logs can then be segmented into video logs and audio logs based on the different data objects.

[0042] During the segmentation of the current log information, the first current log corresponding to the video stream and the second current log corresponding to the audio stream can be determined from the current log information by keyword matching. In this embodiment, the keywords used in the keyword matching method can be preset video stream keywords, such as video stream identifier (Identification Document, ID), resolution or frame rate, etc., or preset audio stream keywords, such as audio stream identifier (Identification Document, ID), sampling rate or number of channels, etc.

[0043] Specifically, firstly, a programming language or corresponding text processing tool is used to obtain the current log information generated by the screen mirroring application after the lag occurs. Then, a video keyword list and an audio keyword list are constructed using preset video stream keywords and preset audio stream keywords, respectively. Finally, a keyword matching algorithm, such as regular expressions or string search, is used to perform similarity matching between each keyword in the keyword list and each log record in the current log information to determine the log records corresponding to the video stream and the audio stream, respectively. The log records corresponding to the video stream are then summarized and integrated to obtain the first current log corresponding to the video stream, and the log records corresponding to the audio stream are then summarized and integrated to obtain the second current log corresponding to the audio stream.

[0044] Step S202: Obtain at least one preset exception identifier, determine the first current data value corresponding to each exception identifier from the first current log, and determine the second current data value corresponding to each exception identifier from the second current log.

[0045] In this embodiment, the exception identifier can refer to an identifier item set according to requirements. This identifier item can be matched with the corresponding identifier object from the log information generated by the screen casting application. For example, the exception identifier can be a timestamp identifier, a thread / process ID identifier, or a runtime environment identifier, etc., where the timestamp identifier can be timestamp, the thread ID identifier can be tid, the process ID identifier can be pid, and the runtime environment identifier can be Windows or Linux.

[0046] The first current data value can refer to the data value corresponding to the identifier object matched by the exception identifier in the first current log, and the second current data value can refer to the data value corresponding to the identifier object matched by the exception identifier in the second current log. For example, the timestamp identifier determines the identifier object in the first current log: timestamp, and the data value corresponding to timestamp in the first current log is -1, that is, the first current data value is -1.

[0047] When determining the current data value corresponding to the anomaly identifier, the first current data value corresponding to each anomaly identifier can be determined from the first current log by keyword matching, and the second current data value corresponding to each anomaly identifier can be determined from the second current log. In this embodiment, the keywords used in the keyword matching method can be various anomaly identifiers, such as timestamp identifiers, thread / process ID identifiers, or runtime environment identifiers.

[0048] Specifically, the runtime status data value of each anomaly identifier in the first current log and the runtime status data value of each anomaly identifier in the second current log are determined by keyword matching. First, an anomaly identifier keyword list is constructed using each anomaly identifier as a keyword. Then, a keyword matching algorithm is used to perform similarity matching between each anomaly identifier in the keyword list and each log record in the first and second current logs, respectively, to determine the log record corresponding to each anomaly identifier in the first and second current logs. Finally, the first current data value corresponding to each anomaly identifier is determined based on the log record in the first current log, and the second current data value corresponding to each anomaly identifier is determined based on the log record in the second current log.

[0049] Step S203: For any abnormal identifier, compare the first current data value and the second current data value corresponding to the abnormal identifier to obtain a first comparison result. If the comparison result is different, then the abnormal identifier is determined to be a fault result.

[0050] In this embodiment, the first current data value and the second current data value are comparable. The first comparison result can be the same or different. Of course, the first comparison result can also be represented by the difference between the first current data value and the second current data value. If the difference is zero, it means that the first comparison result is the same; if the difference is not zero, it means that the first comparison result is different. The fault result is the corresponding anomaly identifier, which can characterize the influencing factor of the lag as the corresponding anomaly identifier. For example, the anomaly identifier is a timestamp identifier, which indicates that the timestamp is the influencing factor causing the screen mirroring lag.

[0051] During the comparison of the current data value corresponding to the anomaly identifier, the first current data value and the second current data value corresponding to the anomaly identifier can be compared by a string matching algorithm to obtain the first comparison result.

[0052] Specifically, for example, if the exception identifier is a timestamp, when screen mirroring experiences lag, the first current data value corresponding to this exception identifier in the first current log is "2023-08-02T17:30:45.000z", and the second current data value corresponding to it in the second current log is "2023-08-02T19:60:20.000z". A brute-force matching algorithm in string matching is used to match the first and second current data values ​​corresponding to the timestamp, resulting in the first current data value "2023-08-02T17:30:45.000z" being matched. Set ".000z" as the pattern string, and set the second current data value "2023-08-02T19:60:20.000z" as the target string. Starting from the first character of the target string, match each character in the pattern string with the corresponding character in the pattern string. When the 13th character is matched, if the character "7" in the pattern string does not match the character "9" in the target string, then the first current data value and the second current data value are considered to be different, and the timestamp is determined to be a fault result. The settings of the pattern string and the target string can be changed according to the specific situation, and no specific restrictions are imposed here.

[0053] In this embodiment, when screen mirroring from the client to the device experiences lag, the system obtains the first current log corresponding to the video stream and the second current log corresponding to the audio stream in the client's screen mirroring application, and acquires at least one preset anomaly identifier. It then determines the first current data value corresponding to each anomaly identifier from the first current log and the second current data value corresponding to each anomaly identifier from the second current log. The system compares the first and second current data values ​​for each anomaly identifier; if the comparison results are different, the anomaly identifier is determined to be a fault result. Through these steps, the system automatically determines the fault result corresponding to the lag by comparing and analyzing the video and audio streams in the logs when screen mirroring lags. This achieves automated discovery of factors affecting lag, improving the real-time nature of lag factor discovery. Furthermore, by adjusting the screen mirroring data stream based on the fault result, the system reduces the impact of influencing factors on lag, thereby reducing the occurrence of screen mirroring lag and providing users with a better screen mirroring experience.

[0054] like Figure 3 The diagram shown is a flowchart illustrating a method for analyzing screen projection data faults according to Embodiment 2 of the present invention. In Embodiment 1, step S203, determining the anomaly identifier as a fault result, may include the following steps:

[0055] Step S301: Obtain the data standard format corresponding to the anomaly identifier.

[0056] In this embodiment, the data standard format can refer to the arrangement format in which the first current data value corresponding to any anomaly identifier is stored in the first current log, and the arrangement format in which the second current data value corresponding to any anomaly identifier is stored in the second current log. This data standard format can be in the form of numerical values, characters, or binary numbers. For example, when the anomaly identifier is a timestamp, the data standard format of its corresponding first and second current data values ​​can be "YYYY-MM-DD HH:MM:SS". According to this data standard format, the first or second current data value corresponding to the anomaly identifier can be in the form of "2023-09-09 12:05:14". It should be noted that the data standard format corresponding to each anomaly identifier can be set according to specific circumstances, and no specific restrictions are imposed here.

[0057] Step S302: Detect the stuttering situation when the client casts the screen to the device. If the stuttering situation is detected as video stuttering, then check whether the second current data value conforms to the above data standard format.

[0058] In this embodiment, if a screen interruption or freeze is detected during screen projection, the stuttering can be identified as video stuttering. Regular expressions can be used to detect whether the second current data value conforms to the above-mentioned data standard format.

[0059] Specifically, first, the data standard format of the second current data value to be detected is determined. Then, a corresponding regular expression is created based on the data standard format using regular expression syntax. Finally, the second current data value is matched with the created regular expression to detect whether the second current data value conforms to the determined data standard format.

[0060] Step S303: If the second current data value is detected to conform to the above data standard format, the abnormality is identified as a fault result; if the second current data value is detected to not conform to the above data standard format, the abnormality is identified as a fault to be confirmed.

[0061] Among them, the unconfirmed fault can represent the corresponding abnormal identifier as a possible factor causing the lag, but further confirmation is required. For example, if the abnormal identifier is a timestamp identifier, and the preset data standard format is "YYYY-MM-DD HH:MM:SS", the second current data value corresponding to this timestamp identifier is "12:30:12". If this second current data value does not conform to the preset data standard format, then the timestamp identifier is confirmed as an unconfirmed fault. Further analysis of the historical data value corresponding to this timestamp identifier is still needed to determine whether the timestamp identifier is a factor causing the lag.

[0062] Specifically, regular expressions can be used to check whether the second current data value conforms to the aforementioned data standard format. For example, when the anomaly identifier is a timestamp, the preset data standard format is "YYYY-MM-DD HH:MM:SS". The regular expression generated based on this data standard format is " / (\d{4})-(\d{2})-(\d{2})(\d{2}):(\d{2}):(\d{2}) / ". If the second current data value to be checked is "2023-09-09 12:30:12", the format check using the regular expression can determine that the second current data value conforms to the preset data standard format, thus the timestamp is determined to be a fault result. If the second current data value to be checked is "-1", the format check using the regular expression can determine that the second current data value does not conform to the preset data standard format, thus the timestamp is determined to be a fault to be confirmed.

[0063] In this embodiment, by obtaining the data standard format corresponding to the anomaly identifier, when the stuttering condition is video stuttering, if the second current data value conforms to the data standard format, the anomaly identifier is determined to be a fault result; if the second current data value does not conform to the data standard format, the anomaly identifier is determined to be a fault to be confirmed. Through the above steps, the preliminary judgment of the factors affecting video stuttering is automatically realized, improving the real-time performance of the discovery of factors affecting video stuttering.

[0064] like Figure 4 As shown in the flowchart of a screen projection data fault analysis method provided in Embodiment 3 of the present invention, after detecting the lag when the client projects the screen to the device in step S302 of Embodiment 2 above, the fault analysis method further includes the following steps:

[0065] Step S401: If the detected stutter is an audio stutter, then check whether the first current data value conforms to the above data standard format.

[0066] Specifically, if a sound interruption or discontinuity is detected during screen projection, the stuttering can be identified as audio stuttering. A preset regular expression can be used to detect whether the first current data value conforms to the data standard format. For specific implementation details, please refer to step S302 in Embodiment 2, which will not be described in detail here.

[0067] Step S402: If the first current data value is detected to conform to the above data standard format, the abnormality is identified as a fault result; if the first current data value is detected to not conform to the above data standard format, the abnormality is identified as a fault to be confirmed.

[0068] Specifically, regular expressions can be used to detect whether the first current data value conforms to the above data standard format. For specific implementation methods, please refer to the detailed content of step S303 in Embodiment 2, which will not be described in detail here.

[0069] In other implementations, if the detected stuttering is video stuttering or audio stuttering, it is necessary to check whether the first current data value and the second current data value corresponding to the anomaly identifier conform to the data standard format. If only one of the first current data value and the second current data value corresponding to the anomaly identifier conforms to the data standard format, the anomaly identifier is determined to be a fault to be confirmed. If both the first current data value and the second current data value corresponding to the anomaly identifier conform to the data standard format, the anomaly identifier is confirmed to be not a fault result. If neither the first current data value nor the second current data value corresponding to the anomaly identifier conforms to the data standard format, the anomaly identifier is determined to be a fault result.

[0070] In this embodiment, when the stuttering is audio stuttering, if the first current data value conforms to the data standard format, the anomaly is identified as a fault result; if the first current data value does not conform to the data standard format, the anomaly is identified as a fault to be confirmed. Through the above steps, the preliminary judgment of the factors affecting audio stuttering is automatically realized, improving the real-time detection of the factors affecting audio stuttering.

[0071] like Figure 5 The diagram shown is a flowchart illustrating a fault analysis method for screen projection data provided in Embodiment 4 of the present invention. After obtaining the fault result and the fault to be confirmed in the above embodiment, the fault analysis method further includes the following steps:

[0072] S501: Obtain all fault results and all faults to be confirmed.

[0073] Specifically, obtain the anomaly identifiers corresponding to all fault results that cause video stuttering and the anomaly identifiers corresponding to all unconfirmed faults, as well as the anomaly identifiers corresponding to all fault results that cause audio stuttering and the anomaly identifiers corresponding to all unconfirmed faults.

[0074] S502: If the number of all fault results is zero, then obtain the historical log information of the moment before the stutter occurred, and determine the first historical log corresponding to the video stream and the second historical log corresponding to the audio stream from the historical log information.

[0075] In this embodiment, when screen casting lags, the log information generated by the screen casting application within a certain period of time before the lag occurs is obtained from the log directory of the screen casting application. The corresponding time period is recorded as the previous moment, and the corresponding log information is recorded as historical log information. For the screen casting of the playback source, it includes video data and audio data. The generated log can be divided into video log and audio log according to the different data objects.

[0076] During the segmentation of historical log information, the first historical log corresponding to the video stream and the second historical log corresponding to the audio stream can be determined from the historical log information by keyword matching. For specific implementation methods, please refer to the detailed content of step S201 in Embodiment 1, which will not be described in detail here.

[0077] S503: Based on the anomaly identifiers corresponding to all the unconfirmed faults, determine the first historical data value corresponding to each anomaly identifier from the first historical log, and determine the second historical data value corresponding to each anomaly identifier from the second historical log.

[0078] In this embodiment, the first historical data value may refer to the data value corresponding to the identifier object matched by the anomaly identifier in the first historical log, and the second historical data value may refer to the data value corresponding to the identifier object matched by the anomaly identifier in the second historical log.

[0079] When determining the historical data value corresponding to each anomaly identifier, the running status data value of the anomaly identifier corresponding to the fault to be confirmed in the first historical log and the running status data value of the anomaly identifier corresponding to the fault to be confirmed in the second historical log can be determined by keyword matching. For specific implementation methods, please refer to the detailed content of step S202 in Embodiment 1, which will not be described in detail here.

[0080] S504: For any abnormal identifier among all the abnormal identifiers corresponding to the fault to be confirmed, compare the first historical data value and the second historical data value corresponding to the abnormal identifier to obtain a second comparison result. If the second comparison result is different, then the abnormal identifier of the fault to be confirmed is determined to be the fault result.

[0081] Specifically, in the process of comparing the historical data values ​​corresponding to the anomaly identifier, the first historical data value and the second historical data value corresponding to each anomaly identifier can be compared by a string matching algorithm to obtain a second comparison result. For specific implementation details, please refer to the detailed content of step S203 in Embodiment 1, which will not be described in detail here.

[0082] In this embodiment, by acquiring all fault results and pending faults detected in the current stuttering detection, when all stuttering detection results are pending faults, the first historical log corresponding to the video stream and the second historical log corresponding to the audio stream at the moment before the stuttering occurred are acquired. Based on the anomaly identifiers corresponding to all pending faults, the first historical data value corresponding to each anomaly identifier is determined from the first historical log, and the second historical data value corresponding to each anomaly identifier is determined from the second historical log. The first historical data value and the second historical data value corresponding to each anomaly identifier are compared. If the comparison result is different, the anomaly identifier of the pending fault is determined to be a fault result. Through the above steps, the determination of whether the pending fault is a stuttering influencing factor is automatically realized, which not only improves the real-time performance of stuttering influencing factor detection, but also improves the accuracy of stuttering influencing factor detection through further judgment of the pending fault.

[0083] like Figure 6 The diagram shown is a flowchart of a screen mirroring method provided in Embodiment 5 of the present invention. This embodiment provides a screen mirroring method, which is applied to... Figure 1 Taking the client as an example, the steps may include the following:

[0084] Step S601: Obtain all fault results and the real-time data stream of the current screen projection.

[0085] In this embodiment, the real-time data stream can refer to the real-time data stream transmitted from the client to the device during the screen casting process. In terms of the screen casting of the playback source, it includes video data and audio data. The resulting real-time data stream can be divided into real-time video stream and real-time audio stream according to the different data objects.

[0086] Specifically, it retrieves all fault results that cause video stuttering and all fault results that cause audio stuttering, and obtains the real-time video and audio streams of the screen mirroring application by calling the Application Programming Interface (API).

[0087] Step S602: For any fault result, determine the first real-time data value corresponding to the fault result from the video stream of the above real-time data stream, and determine the second real-time data value corresponding to the fault result from the audio stream of the above real-time data stream.

[0088] In this embodiment, the first real-time data value may refer to the real-time data value corresponding to the anomaly identifier of the fault result in the video stream, and the second real-time data value may refer to the real-time data value corresponding to the anomaly identifier of the fault result in the audio stream.

[0089] When determining the real-time data value corresponding to the fault result, the first real-time data value corresponding to the fault result can be determined from the video stream and the second real-time data value corresponding to the fault result can be determined from the audio stream by keyword matching. In this embodiment, the keywords used in the keyword matching method can be the ID of the video stream, the ID of the audio stream, etc.

[0090] Specifically, firstly, the real-time logs corresponding to the video stream are retrieved from the log information of the current screen mirroring application by using the video stream ID as the keyword, and the real-time logs corresponding to the audio stream are retrieved from the log information of the current screen mirroring application by using the audio stream ID as the keyword. Then, the real-time log record for each fault result is determined in the real-time logs corresponding to the video stream by using the exception identifier corresponding to each fault result as the keyword, and the real-time log record for each fault result is determined in the real-time logs corresponding to the audio stream. Finally, the first real-time data value corresponding to each fault result is determined based on the real-time log record corresponding to each fault result in the video stream, and the second real-time data value corresponding to each fault result is determined based on the real-time log record corresponding to each fault result in the audio stream.

[0091] Step S603: Unify the first real-time data value and the second real-time data value to obtain a unified data value, update the real-time video stream using the unified data value to obtain an updated data stream, and send the updated data stream to the device.

[0092] Specifically, the first real-time data value and the second real-time data value corresponding to each fault result are modified and adjusted according to the correct operating status data value to obtain a unified data value after modification and adjustment. The data stream is then updated with the modified unified data value and sent to the device for playback.

[0093] For example, when the fault result is that the timestamp running status in the video stream is abnormal, firstly, the second real-time data value corresponding to the timestamp in the audio stream is uniformly assigned to the first real-time data value corresponding to the timestamp, so as to obtain the updated first real-time data value and realize the update of the real-time data stream. Then, the updated real-time video stream and real-time audio stream are encapsulated into data packets and sent to the device in the form of data packets through transmission devices such as routers and converters.

[0094] In this embodiment, by acquiring all fault results and the current real-time data stream of the projected screen, the first real-time data value corresponding to each fault result in the video stream and the second real-time data value corresponding to each fault result in the audio stream are determined. The obtained first and second real-time data values ​​are then processed uniformly to obtain an updated data stream, which is then sent to the device for playback. Through these steps, the updating of the real-time data stream of the projected screen based on the fault results is automated, reducing the frequency of screen casting stuttering and providing users with a better screen casting experience.

[0095] In one embodiment, the step S603 in the above embodiment five, which unifies the first real-time data value and the second real-time data value to obtain a unified data value, may include:

[0096] When video stuttering occurs when the client projects the screen to the device, the second real-time data value is used as the unified data value; when audio stuttering occurs when the client projects the screen to the device, the first real-time data value is used as the unified data value.

[0097] Specifically, when video stuttering occurs, the second real-time data value is used as the unified data value and assigned to the first real-time data value to update the first real-time data value. When audio stuttering occurs, the first real-time data value is used as the unified value and assigned to the second real-time data value to update the second real-time data value.

[0098] In other implementations, if the anomaly identifier corresponding to the fault result is a timestamp, and when video stuttering occurs when the client projects the screen to the device, although the audio does not stutter, the second real-time data value of the timestamp corresponding to the fault result in the audio stream is an incorrect running status data value, then the system clock value of the projecting client can be used as a unified data value. Here, the time of the system clock increases linearly. The first real-time data value of the timestamp in the video stream and the second real-time data value in the audio stream are updated to the same data value as the system clock. Similarly, when audio stuttering occurs when the client projects the screen to the device, but the first real-time data value of the timestamp corresponding to the fault result in the video stream is an incorrect running status data value, the client's system clock value can also be used as a unified data value.

[0099] In this embodiment, when the video stutters, the first real-time data value is updated using the second real-time data value as a unified data value; when the audio stutters, the second real-time data value is updated using the first real-time data value as a unified data value. This realizes the update processing of the real-time data stream for screen casting, providing a better way to solve the screen casting stuttering problem and effectively reducing the frequency of screen casting stuttering.

[0100] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0101] In one embodiment, a fault analysis device for screen projection data is provided, which corresponds one-to-one with the fault analysis method for screen projection data in the above embodiments. For example... Figure 7 As shown, the fault analysis device for the projected data includes a current log determination module 71, a current data value determination module 72, and a first comparison result determination module 73. Detailed descriptions of each functional module are as follows:

[0102] The current log determination module 71 is used to obtain the current log information generated by the corresponding screen casting application in the client when the screen casting from the client to the device is stuck, and determine the first current log corresponding to the video stream and the second current log corresponding to the audio stream from the current log information;

[0103] The current data value determination module 72 is used to obtain at least one preset anomaly identifier, determine the first current data value corresponding to each anomaly identifier from the first current log, and determine the second current data value corresponding to each anomaly identifier from the second current log;

[0104] The first comparison result determination module 73 is used to compare the first current data value and the second current data value corresponding to any anomaly identifier to obtain a first comparison result. If the comparison result is different, the anomaly identifier is determined to be a fault result.

[0105] Optionally, the first comparison result determination module 73 described above includes:

[0106] A data standard format acquisition unit is used to acquire the data standard format corresponding to the anomaly identifier;

[0107] The stuttering detection unit is used to detect stuttering when the client projects its screen onto the device.

[0108] The video stuttering detection unit is used to detect stuttering when the client projects the screen to the device. If the stuttering is detected as video stuttering, the unit checks whether the second current data value conforms to the data standard format.

[0109] The second current data value detection unit is used to determine the anomaly identifier as the fault result if the second current data value is detected to conform to the data standard format, and to determine the anomaly identifier as a fault to be confirmed if the second current data value is detected to not conform to the data standard format.

[0110] Optionally, the aforementioned stuttering detection unit further includes:

[0111] An audio stuttering detection subunit is used to detect whether the first current data value conforms to the data standard format if the stuttering situation is detected as audio stuttering.

[0112] The first current data value detection subunit is used to determine the anomaly identifier as the fault result if the first current data value is detected to conform to the data standard format, and to determine the anomaly identifier as the fault to be confirmed if the first current data value is detected to not conform to the data standard format.

[0113] Optionally, the fault analysis device for the projected data also includes:

[0114] The fault result acquisition module acquires all fault results and all faults awaiting confirmation.

[0115] The historical log determination module is used to obtain the historical log information of the moment before the stutter occurred if the number of all fault results is zero, and determine the first historical log corresponding to the video stream and the second historical log corresponding to the audio stream from the historical log information.

[0116] The historical data value determination module is used to determine the first historical data value corresponding to each anomaly identifier from the first historical log and the second historical data value corresponding to each anomaly identifier from the second historical log, based on the anomaly identifiers corresponding to all the anomalies to be confirmed.

[0117] The second comparison result determination module is used to compare the first historical data value and the second historical data value corresponding to any one of the abnormal identifiers corresponding to all the faults to be confirmed, and obtain a second comparison result. If the second comparison result is different, the abnormal identifier of the fault to be confirmed is determined as the fault result.

[0118] In one embodiment, a screen projection device is provided, which corresponds one-to-one with the screen projection methods in the above embodiments. For example... Figure 8 As shown, the projection device includes an acquisition module 81, a real-time data value determination module 82, and a data stream update module 83. Detailed descriptions of each functional module are as follows:

[0119] The acquisition module 81 is used to acquire all fault results and the real-time data stream of the current screen projection;

[0120] The real-time data value determination module 82 is used to determine, for any fault result, a first real-time data value corresponding to the fault result from the video stream of the real-time data stream, and a second real-time data value corresponding to the fault result from the audio stream of the real-time data stream;

[0121] The data stream update module 83 is used to unify the first real-time data value and the second real-time data value to obtain a unified data value, use the unified data value to update the real-time video stream to obtain an updated data stream, and send the updated data stream to the device.

[0122] Optionally, the data stream update module 83 described above includes:

[0123] The real-time data unification module is used to use the second real-time data value as the unified data value when video stuttering occurs when the client projects the screen to the device, and to use the first real-time data value as the unified data value when audio stuttering occurs when the client projects the screen to the device.

[0124] Specific limitations regarding the fault analysis device and screen projection device for screen projection data can be found in the limitations regarding the fault analysis method and screen projection method for screen projection data described above, and will not be repeated here. Each module in the aforementioned fault analysis device and screen projection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0125] Figure 9 This is a schematic diagram of the structure of a terminal device provided in Embodiment 9 of the present invention. Figure 9 As shown, the terminal device of this embodiment includes: at least one processor ( Figure 9 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor. When the processor executes the computer program, it implements the steps in the embodiments of the fault analysis method and the projection method for any of the above-described projection data.

[0126] The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 9 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. A terminal device may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as network interfaces, displays, and input devices.

[0127] The processor referred to can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0128] The memory includes readable storage media, internal memory, etc., wherein the internal memory can be the main memory of the terminal device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of the terminal device, or in some embodiments, it can be an external storage device of the terminal device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital Card (SD), or a Flash Card. Furthermore, the memory can include both internal storage units and external storage devices of the terminal device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. The memory can also be used to temporarily store data that has been output or will be output.

[0129] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0130] The present invention can implement all or part of the processes in the above embodiments of the method, or it can be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device executes the steps in the above method embodiments.

[0131] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0132] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0133] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0134] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0135] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for fault analysis of screen-projected data, characterized in that, The fault analysis method includes the following steps: When a stutter occurs when the screen is cast from the client to the device, the current log information generated by the corresponding screen casting application in the client is obtained, and the first current log corresponding to the video stream and the second current log corresponding to the audio stream are determined from the current log information. Obtain at least one preset exception identifier, determine the first current data value corresponding to each exception identifier from the first current log, and determine the second current data value corresponding to each exception identifier from the second current log; For any anomaly identifier, compare the first current data value and the second current data value corresponding to the anomaly identifier to obtain a first comparison result. If the comparison result is different, then the anomaly identifier is determined to be a fault result. The step of determining the anomaly identifier as a fault result includes: Obtain the data standard format corresponding to the anomaly identifier; The system detects the stuttering situation when the client projects the screen to the device. If the stuttering situation is detected as video stuttering, the system checks whether the second current data value conforms to the data standard format. If the second current data value is detected to conform to the data standard format, the anomaly identifier is determined to be the fault result; if the second current data value is detected to not conform to the data standard format, the anomaly identifier is determined to be a fault to be confirmed. The fault analysis method further includes: Obtain all fault results and all faults awaiting confirmation; If the number of all fault results is zero, then obtain the historical log information of the moment before the stutter occurred, and determine the first historical log corresponding to the video stream and the second historical log corresponding to the audio stream from the historical log information; Based on the anomaly identifiers corresponding to all the unconfirmed faults, determine the first historical data value corresponding to each anomaly identifier from the first historical log, and determine the second historical data value corresponding to each anomaly identifier from the second historical log; For any one of the abnormal identifiers corresponding to all the faults to be confirmed, compare the first historical data value and the second historical data value corresponding to the abnormal identifier to obtain a second comparison result. If the second comparison result is different, then the abnormal identifier of the fault to be confirmed is determined to be the fault result.

2. The fault analysis method as described in claim 1, characterized in that, After detecting the lag when the client projects the screen to the device, the method further includes: If the detected stuttering is an audio stuttering, then check whether the first current data value conforms to the data standard format; If the first current data value is detected to conform to the data standard format, the anomaly identifier is determined to be the fault result; if the first current data value is detected to not conform to the data standard format, the anomaly identifier is determined to be the fault to be confirmed.

3. A screen projection method, characterized in that, Based on the fault analysis method as described in any one of claims 1 to 2, the screen projection method includes the following steps: Obtain all fault results and the real-time data stream of the current screen projection; For any fault result, a first real-time data value corresponding to the fault result is determined from the video stream of the real-time data stream, and a second real-time data value corresponding to the fault result is determined from the audio stream of the real-time data stream. The first real-time data value and the second real-time data value are unified to obtain a unified data value. The unified data value is used to update the real-time video stream to obtain an updated data stream, which is then sent to the device.

4. The screen projection method as described in claim 3, characterized in that, The step of unifying the first real-time data value and the second real-time data value to obtain a unified data value includes: When video stuttering occurs when the client projects its screen to the device, the second real-time data value is used as the unified data value; when audio stuttering occurs when the client projects its screen to the device, the first real-time data value is used as the unified data value.

5. A fault analysis device for screen-projected data, characterized in that, include: The current log determination module is used to obtain the current log information generated by the corresponding screen casting application in the client when the screen casting from the client to the device is stuck, and determine the first current log corresponding to the video stream and the second current log corresponding to the audio stream from the current log information; The current data value determination module is used to obtain at least one preset anomaly identifier, determine the first current data value corresponding to each anomaly identifier from the first current log, and determine the second current data value corresponding to each anomaly identifier from the second current log; The first comparison result determination module is used to compare the first current data value and the second current data value corresponding to any anomaly identifier to obtain a first comparison result. If the comparison result is different, the anomaly identifier is determined to be a fault result. The first comparison result determination module includes: A data standard format acquisition unit is used to acquire the data standard format corresponding to the anomaly identifier; The stuttering detection unit is used to detect stuttering when the client projects its screen onto the device. The video stuttering detection unit is used to detect stuttering when the client projects the screen to the device. If the stuttering is detected as video stuttering, the unit checks whether the second current data value conforms to the data standard format. The second current data value detection unit is used to determine the anomaly identifier as the fault result if the second current data value is detected to conform to the data standard format, and to determine the anomaly identifier as a fault to be confirmed if the second current data value is detected to not conform to the data standard format. The fault analysis device for the projected data also includes: The fault result acquisition module acquires all fault results and all faults awaiting confirmation. The historical log determination module is used to obtain the historical log information of the moment before the stutter occurred if the number of all fault results is zero, and determine the first historical log corresponding to the video stream and the second historical log corresponding to the audio stream from the historical log information. The historical data value determination module is used to determine the first historical data value corresponding to each anomaly identifier from the first historical log and the second historical data value corresponding to each anomaly identifier from the second historical log, based on the anomaly identifiers corresponding to all the anomalies to be confirmed. The second comparison result determination module is used to compare the first historical data value and the second historical data value corresponding to any one of the abnormal identifiers corresponding to all the faults to be confirmed, and obtain a second comparison result. If the second comparison result is different, the abnormal identifier of the fault to be confirmed is determined as the fault result.

6. A screen projection device, characterized in that, Based on the fault analysis method as described in any one of claims 1 to 2, including: The acquisition module is used to acquire all fault results and the real-time data stream of the current screen projection; The real-time data value determination module is used to determine, for any fault result, a first real-time data value corresponding to the fault result from the video stream of the real-time data stream, and a second real-time data value corresponding to the fault result from the audio stream of the real-time data stream. The data stream update module is used to unify the first real-time data value and the second real-time data value to obtain a unified data value, use the unified data value to update the real-time video stream to obtain an updated data stream, and send the updated data stream to the device.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the fault analysis method as described in any one of claims 1 to 2, or the screen projection method as described in any one of claims 3 to 4.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the fault analysis method as described in any one of claims 1 to 2, or the screen projection method as described in any one of claims 3 to 4.

Images