Multimedia content playback lag detection method, and electronic device

Abstract

Provided in the present application are a multimedia content playback lag detection method, and an electronic device. When an electronic device displays a multimedia content playback interface, a preset operation is received, and a lag detection module of the electronic device may detect whether a multimedia application reports a playback start event in a preset time period from the activation of the preset operation. If a playback start event reported by the multimedia application has not been received within the preset time period, the lag detection module may instruct the electronic device to adjust a network, such that the electronic device uses a network having a better quality to perform communication. The present method can accurately identify, when a user triggers an electronic device to play multimedia content, whether the multimedia content playback is lagging, thereby improving the accuracy of multimedia content playback lag detection.

Description

Multimedia content playback jam detection method and electronic device

[0001] This application claims priority to the Chinese patent application with application number 202311626999.6 filed with the State Intellectual Property Office of China on November 29, 2023, and priority to the Chinese patent application entitled “A method and electronic device for detecting jamming in multimedia content playback”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a method for detecting multimedia content playback freezes and an electronic device. Background Art

[0003] Multimedia content may include audio and / or video. For example, some multimedia content is independent audio, meaning that when played, audio can be heard but no video can be viewed. Other multimedia content is independent video, meaning that when played, video can be viewed but no audio can be heard. Still other multimedia content is a combination of audio and video, meaning that when played, audio can be heard and video can be viewed.

[0004] During the online playback of multimedia content, jamming often occurs due to problems such as poor network conditions. Currently, parameters such as signal strength and network transmission delay are generally used to detect whether multimedia content is jammed. However, electronic devices are based on caching technology, and usually cache a portion of data for playback during the playback of multimedia content. When the network is poor, the electronic device may be using the cached data for playback, and the multimedia content is not jammed at this time. Therefore, the jamming of multimedia content playback is not necessarily synchronized with the occurrence of poor network conditions. The accuracy of the above method of determining audio and video jams when a poor network is detected is relatively low. Summary of the Invention

[0005] The present application provides a method and electronic device for detecting multimedia content playback jams, which can improve the accuracy of multimedia content playback jam detection by combining user operations on the electronic device with multimedia playback events to detect whether the multimedia content is jammed.

[0006] In a first aspect, the present application provides a method for detecting jamming in multimedia content playback. The method is applied to an electronic device, which includes a first application, which can be an application for playing multimedia content. The electronic device receives a first operation on a first interface, which is an interface for controlling the playback of multimedia content in a first application, and the first operation is used to play the first multimedia content; when the electronic device does not obtain a first start playback event reported by the first application after the first operation, the electronic device adjusts the network for data communication; wherein the first start playback event is used to indicate the start of playback of the first multimedia content. The first start playback event can be reported by the first application when the playback data of the first multimedia content is obtained.

[0007] The failure to obtain the first start playing event reported by the first application may indicate that the first multimedia content has not started playing.

[0008] The first interface may refer to the user interface 310 shown in FIG3A of the present application. The first operation may refer to the upward sliding operation shown in FIG3C and FIG3D of the present application. The first multimedia content may refer to the audio and video corresponding to the video screen 322 shown in FIG3D.

[0009] It can be seen that the above method can accurately identify whether the electronic device plays the multimedia content smoothly without lag when the user triggers the electronic device to play the multimedia content. In the case where the electronic device has pre-cached the playback data of the multimedia content, even if the network of the electronic device is not good when the user triggers the electronic device to play the multimedia content, the multimedia application can also obtain the pre-cached playback data, and then immediately respond to the user operation to play the multimedia content and report the start playback event. The electronic device can detect that the multimedia content is not stuck based on the above user operation and the start playback event. This can reduce the situation where the electronic device responds to the user operation to play the multimedia content smoothly, but mistakenly detects that the multimedia content playback is stuck based on parameters indicating a poor network. The above method can improve the accuracy of detecting multimedia content playback jams.

[0010] Furthermore, when a jam is detected in the multimedia content playback, the electronic device can adjust the network to eliminate the jam as quickly as possible, thereby improving the user's experience in listening to and watching the multimedia content.

[0011] In conjunction with the first aspect, in some embodiments, the first operation matches a preset operation. The preset operation may be associated with the first application. That is, the electronic device may store a correspondence table between multimedia applications and different preset operations. The preset operation associated with a multimedia application may be an operation that acts on the multimedia application to control the multimedia content playback interface for triggering multimedia content playback. Optionally, the preset operation may be associated with the first interface. That is, the electronic device may store a correspondence table between the multimedia playback interface and different preset operations. The preset operation associated with a multimedia playback interface may be an operation that acts on the multimedia playback interface to trigger multimedia content playback.

[0012] As can be seen, the electronic device can determine whether the first operation is an operation that triggers multimedia content playback by judging whether the first operation matches the preset operation, and then decide whether to wait for the start playback event to detect whether the multimedia content playback is stuck. This can reduce the situation where the received operation does not match the preset operation and improve the accuracy of multimedia content playback stuck detection.

[0013] In combination with the first aspect, in some embodiments, the network for data communication is adjusted from the first network to the second network, and the first application in the electronic device uses the second network to obtain the playback data of the first media content; the electronic device plays the first multimedia content according to the playback data of the first multimedia content.

[0014] The quality of the second network is better than that of the first network.

[0015] The network for data communication is adjusted from the first network to the second network, including but not limited to: adjusting from a Wi-Fi network to a cellular mobile network, adjusting from the cellular mobile network corresponding to the primary SIM card to the cellular mobile network corresponding to the secondary SIM card, adjusting from a cellular mobile network of one standard to a cellular mobile network of another standard (such as adjusting from a 4G cellular mobile network to a 5G cellular mobile network), adjusting the cellular mobile network from one resident cell to another resident cell, and the like.

[0016] As can be seen, when a multimedia content playback freeze is detected, the electronic device can adjust the network to use a higher quality network for communication. In this way, the electronic device can obtain the multimedia content playback data as quickly as possible and eliminate the multimedia content playback freeze.

[0017] In combination with the first aspect, in some embodiments, after the electronic device adjusts the network for data communication, the electronic device changes the first identifier on the first interface to a second identifier, where the first identifier is used to indicate that the network for data communication is the first network, and the second identifier is used to indicate that the network for data communication is the second network.

[0018] The first identifier mentioned above may refer to the Wi-Fi identifier 311B shown in FIG3G of the present application. The second identifier may refer to the cellular mobile network identifier 311E shown in FIG3H of the present application.

[0019] In combination with the first aspect, in some embodiments, the electronic device plays the first multimedia content according to the playback data of the first multimedia content, which may specifically include: the electronic device plays the first audio and / or plays the first video according to the playback data of the first multimedia content.

[0020] In combination with the first aspect, in some embodiments, during the playback of first multimedia content, the electronic device experiences a freeze in playback of the first multimedia content; the electronic device detects a first data exhaustion event and adjusts the network for data communication based on the first data exhaustion event; wherein the first data exhaustion event is used to indicate that the playback data used to play the first multimedia content in the electronic device has been exhausted and no playback data is temporarily available.

[0021] As can be seen, the first data exhausted event can indicate that the first multimedia content experienced freezes during playback. Based on the first data exhausted event, the electronic device can accurately detect whether the first multimedia content experienced freezes during playback. If freezes occur, the electronic device can adjust the network quality to use a higher-quality network for communication to eliminate freezes. This can further enhance the user's audio listening experience.

[0022] In combination with the first aspect, in some embodiments, when the electronic device obtains the first start playback event reported by the first application after the first operation, the electronic device plays the first multimedia content; wherein, the playback data of the first multimedia content is obtained by the first application from the memory, or is obtained by the first application using a network for data communication.

[0023] In combination with the first aspect, in some embodiments, the electronic device receives a second operation on the first interface, and the second operation does not match the preset operation; or, the electronic device receives a third operation on the second interface, and the second interface is not an interface for controlling the playback of multimedia content; the electronic device waits to receive subsequent user operations.

[0024] As can be seen, when the received operation does not match the preset operation, the electronic device can detect multimedia content playback freezes without waiting for the start playback event. This can reduce the situation of false detection of multimedia content playback freezes and improve the accuracy of multimedia content playback freeze detection.

[0025] In combination with the first aspect, in some embodiments, the first multimedia content is paused, and the electronic device receives a fourth operation on the first interface, and the fourth operation is used to play the first multimedia content; the electronic device recognizes that the fourth operation acts on the first area of ​​the first interface, and the fourth operation matches the preset operation, and the electronic device determines whether the first start playback event reported by the first application is obtained within the second time period; wherein the second time period is a preset time period starting from the second moment, and the second moment is the time when the second operation is taken effect; when the electronic device does not obtain the first start playback event reported by the first application within the second time period, the electronic device adjusts the network for data communication; when the electronic device obtains the first start playback event reported by the first application within the second time period, the electronic device plays the first multimedia content.

[0026] The first area may be a preset area on the first interface. Different multimedia playback interfaces may correspond to different preset areas. The electronic device may more accurately identify the user operation used to trigger the playback of multimedia content by combining the playback status of the multimedia content, the type of user operation, and the location of the operation. This allows the electronic device to detect whether the multimedia content being played is stuck in more scenarios where multimedia content playback is triggered by user operations.

[0027] In combination with the first aspect, in some embodiments, before the electronic device adjusts the network for data communication, the electronic device determines, based on activity information of the first interface, that the first interface is an interface for controlling playback of multimedia content.

[0028] In combination with the first aspect, in some embodiments, before the electronic device adjusts the network for data communication, the electronic device detects that a data transmission indicator after the first operation indicates that the network quality is worse than a preset network quality.

[0029] It can be understood that when multimedia content freezes based on user operations and multimedia playback events, the data transmission indicator indicating that the network quality is worse than the preset network quality can better illustrate that the poor network quality causes multimedia content freezes, thereby improving the confidence of the multimedia content freeze detection results.

[0030] In conjunction with the first aspect, in some embodiments, the electronic device receives a fifth operation on a second interface, where the second interface is an interface for controlling multimedia content playback in a first application, and the fifth operation is used to play the second multimedia content; when the electronic device does not obtain a second start playback event reported by the first application after the fifth operation, and a data transmission indicator after the fifth operation indicates that the network quality is better than a preset network quality, the electronic device closes one or more processes, one or more of which are not processes of the first application, and the second start playback event is used to indicate the start of playback of the second multimedia content. The second start playback event can be reported by the first application when playback data of the second multimedia content is obtained.

[0031] The failure to obtain the second start playing event reported by the first application may indicate that the second multimedia content has not started playing.

[0032] The above data transmission indicator is the round-trip time RTT.

[0033] It is understandable that when multimedia content freezes based on user operations and multimedia playback events, the data transmission indicator indicates that the network quality is better than the preset network quality, which means that the multimedia content freeze is not caused by poor network quality. For example, the load of the electronic device processor is too high, and there are too many background processes running. It may also cause multimedia content freezes. Therefore, electronic devices can better judge whether multimedia content freezes due to poor network quality in combination with data transmission indicators, and then decide whether the network needs to be adjusted to eliminate freezes. This can better eliminate the factors that cause multimedia content freezes, restore smooth playback of multimedia content, and enhance the user experience of listening to and watching multimedia content.

[0034] In combination with the first aspect, in some embodiments, when the electronic device does not obtain the first start playback event within a first time period, the electronic device adjusts the network for data communication, the first time period is a preset time period starting from a first moment, and the first moment is the time when the first operation takes effect.

[0035] In some embodiments, the data transmission indicator after the first operation indicates that the network quality is worse than the preset network quality, which may specifically include: the RTT in the first time period is greater than the preset RTT threshold, or the average RTT in the first time period is greater than the preset RTT threshold.

[0036] In some embodiments, when the electronic device does not obtain the second start play event reported by the first application after the fifth operation, and the data transmission indicator after the fifth operation indicates that the network quality is better than the preset network quality, the electronic device closes one or more processes. Specifically, when the electronic device does not obtain the second start play event reported by the first application within a third time period, and the data transmission indicator within the third time period indicates that the network quality is better than the preset network quality, the electronic device closes one or more processes. The third time period can be a preset time period starting from the third moment. The third moment can be the time when the fifth operation takes effect.

[0037] Among them, the data transmission indicator in the third time period indicates that the network quality is better than the preset network quality, which may specifically include: the RTT in the third time period is less than or equal to the preset RTT threshold, or the average RTT in the third time period is less than or equal to the preset RTT threshold.

[0038] In a second aspect, the present application provides a method for detecting interface browsing freezes. The method can be applied to an electronic device, which includes a first application. The electronic device receives a first operation on a first interface in the first application, and the first operation is used to switch the display content on the electronic device screen within the first application; the electronic device obtains a data transmission indicator after the first operation, and the data transmission indicator is used to indicate the network quality of the electronic device for data communication; if the data transmission indicator after the first operation indicates that the network quality is worse than the preset network quality, the electronic device adjusts the network for data communication.

[0039] The first interface may be a graphic interface displaying text and / or pictures.

[0040] The first application may be a multimedia application, a browser, a news APP, a shopping APP, a reading APP, etc.

[0041] As can be seen, the above method can determine whether a user experiences lag when browsing the interface of the first application. When lag is detected, the electronic device can adjust the network to use a higher-quality network for communication. In this way, the electronic device can obtain the data required to display the interface as quickly as possible, thereby eliminating lag in browsing the interface as quickly as possible and improving the user's browsing experience.

[0042] In combination with the second aspect, in some embodiments, the first interface matches a preset interface, and the first operation matches a preset operation.

[0043] That is, the electronic device can determine whether the first operation is a user operation for switching the display content on the electronic device screen within the first application by determining whether the first operation is a preset operation acting on a preset interface.

[0044] In conjunction with the second aspect, in some embodiments, the first interface is a long page, and the first operation is used to switch the displayed content on the screen of the electronic device to the content of a different area on the first interface. For example, the first operation may be an operation of sliding up or down on the first interface. In this way, the electronic device can switch the display of the content of different areas on the first interface on the screen according to the first operation. Alternatively, the first operation is used to switch the first interface to the second interface in the first application. For example, the first operation may be an operation of sliding left or right on the first interface. In this way, the electronic device can switch the content displayed on the screen to other interfaces on the left or right of the first interface according to the first operation.

[0045] In combination with the second aspect, in some embodiments, the data transmission indicator after the first operation includes a data transmission indicator within a first time period, where the first time period is a preset time period starting from a first moment, and the first moment is the time when the first operation takes effect.

[0046] In combination with the second aspect, in some embodiments, the data transmission indicator is the round-trip time RTT, and the data transmission indicator after the first operation indicates that the network quality is worse than the preset network quality, specifically including: the RTT in the first time period is greater than the preset RTT threshold, or the average of the RTT in the first time period is greater than the preset RTT threshold.

[0047] In combination with the second aspect, in some embodiments, the network for data communication is adjusted from the first network to the second network. After the electronic device adjusts the network for data communication, the electronic device changes the first identifier on the first interface to the second identifier. The first identifier is used to indicate that the network for data communication is the first network, and the second identifier is used to indicate that the network for data communication is the second network.

[0048] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program and the processor is used to call the computer program, so that the electronic device can implement any possible method as described in the first aspect or the second aspect.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enables the electronic device to execute any possible implementation method in the first aspect or the second aspect.

[0050] In a fifth aspect, the present application provides a computer program product, which may include computer instructions. When the computer instructions are run on an electronic device, the electronic device executes any possible implementation method in the first aspect or the second aspect.

[0051] In a sixth aspect, the present application provides a chip, which is applied to an electronic device, and the chip includes one or more processors, which are used to call computer instructions to enable the electronic device to execute any possible implementation method as in the first aspect or the second aspect.

[0052] It is understandable that the electronic device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, the computer program product provided in the fifth aspect, and the chip provided in the sixth aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] FIG1 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0054] FIG2A is a schematic diagram of a software structure of an electronic device 100 provided in an embodiment of the present application;

[0055] FIG2B is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application;

[0056] 3A to 3H are schematic diagrams of some multimedia content playback jam detection scenarios provided by embodiments of the present application;

[0057] FIG4A and FIG4B are schematic diagrams of some multimedia content playback jam detection scenarios provided by embodiments of the present application;

[0058] FIG5 is a flow chart of a method for detecting jamming in multimedia content playback provided by an embodiment of the present application;

[0059] FIG6 is a flow chart of another method for detecting jamming in multimedia content playback provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0061] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0062] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0063] The following embodiments of this application relate to a human-machine interface (HMI). An HMI is a media interface for interaction and information exchange between an application (APP) or operating system (OS) and a user. It converts information between its internal form and a form acceptable to the user. The HMI may include a user interface (UI), physical buttons, a virtual gesture recognition module, etc. A user interface is source code written in a specific computer language such as Java or extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, ultimately presenting content that the user can recognize. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed graphically. It can be visual interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device. A virtual gesture recognition module can be used to recognize virtual gestures made by a user on the electronic device, such as air gestures.

[0064] The present application provides a method for detecting jamming in multimedia content playback. By combining user operations on electronic devices with multimedia playback events to detect whether multimedia content is jammed, the accuracy of multimedia content playback jam detection can be improved. Multimedia applications can refer to apps that can play audio and / or video. For example, multimedia applications can include but are not limited to: music apps, film and television apps, short video apps, social apps, etc. The embodiments of the present application do not limit multimedia applications. Multimedia applications can also be called audio and video playback applications and other names.

[0065] Specifically, when the multimedia application starts playing a multimedia content, it can report a start playback event to the multimedia processing module in the electronic device. In the case that the interface currently displayed by the electronic device is the multimedia playback interface of the multimedia application, if the electronic device receives a preset operation, the electronic device can detect whether the multimedia application reports a start playback event after the preset operation. The above-mentioned preset operation may include an operation that triggers the multimedia application to play multimedia content, such as an upward swipe operation, a downward swipe operation, and the like. If the multimedia application reports a start playback event after the preset operation, it may indicate that the multimedia content playback is not stuck. If the multimedia application does not report a start playback event after the preset operation, it may indicate that the multimedia content playback is stuck. When a jam in the multimedia content playback is detected, the electronic device can perform operations such as network adjustment to use a better quality network for communication. This can reduce the duration of the jam and improve the user's experience of listening to and watching multimedia content.

[0066] The preset operation may include a preset time period after the preset operation starts. That is, the electronic device may detect whether the multimedia application reports a start-playing event within the preset time period after the preset operation starts.

[0067] It is understood that after the electronic device receives the preset operation, the multimedia application needs to respond to the preset operation and obtain the playback data of the multimedia content (such as audio data and / or video data) in order to play the multimedia content. The playback data of the multimedia content may be pre-cached by the electronic device, or may be obtained by the electronic device in real time over a communication network. If the multimedia application is able to immediately obtain the playback data of the multimedia content, it can immediately play the multimedia content and report a start playback event. In this way, when the user triggers the multimedia application to play the multimedia content through an operation, the multimedia application can quickly load the playback data, and the multimedia content playback is not interrupted. Therefore, when the multimedia content playback is not interrupted, the electronic device will usually receive the start playback event reported by the multimedia application quickly after receiving the preset operation. However, if the network is poor and the multimedia application is unable to obtain the playback data of the multimedia content for a long time, the multimedia application will not report the start playback event for a long time. As a result, when the user triggers the multimedia application to play the multimedia content through an operation, the multimedia application cannot quickly load the playback data and cannot respond to the user operation to play the multimedia content in a timely manner, resulting in interruptions in the multimedia content. Therefore, when multimedia content playback is stuck, the electronic device may wait for a long time after receiving the preset operation to obtain the start playback event reported by the multimedia application, or may even be unable to wait until the multimedia application reports the start playback event.

[0068] The above method can accurately identify whether the electronic device plays the multimedia content smoothly without lag when the user triggers the electronic device to play the multimedia content. In the case where the electronic device has pre-cached the playback data of the multimedia content, even if the network of the electronic device is not good when the user triggers the electronic device to play the multimedia content, the multimedia application can also obtain the pre-cached playback data, and then immediately respond to the user operation to play the multimedia content and report the start playback event. Based on the above user operation and the start playback event, the electronic device can detect that the multimedia content is not stuck. This can reduce the situation where the electronic device responds to the user operation to play the multimedia content smoothly, but mistakenly detects that the multimedia content playback is stuck based on parameters indicating a poor network. The above method can improve the accuracy of detecting multimedia content playback jams.

[0069] In some embodiments, multimedia playback events can be divided into audio events and video events. Audio events can be used to indicate the playback status of audio. Video events can be used to indicate the playback status of video. For independent audio, electronic devices can detect whether the audio playback is stuck based on audio events. For independent video, electronic devices can detect whether the video playback is stuck based on video events. For multimedia content that combines audio and video, since audio and video are played synchronously, electronic devices can detect whether the audio and video playback is stuck based on audio events, or they can also detect whether the audio and video playback is stuck based on video events.

[0070] The multimedia processing module in the electronic device may include an audio module and a video module, wherein the multimedia application may report audio events to the audio module and report video events to the video module.

[0071] For ease of understanding, the "audio events" involved in this application are introduced here.

[0072] When a multimedia application plays audio, it can report corresponding audio events to the audio module responsible for audio management and control in the electronic device based on the audio playback status. Audio events may include but are not limited to: start event, pause event, stop event, data exhaustion event, data resumption event, etc.

[0073] The start event may be reported by a multimedia application when it starts playing an audio. The multimedia application starting to play an audio may include: the multimedia application playing from the starting position of an audio, and the multimedia application continuing to play an audio after the audio is paused.

[0074] The pause event can be reported by a multimedia application when pausing an audio. In response to a user operation to pause the audio, the multimedia application can report the pause event. When the audio is paused, the electronic device can save the progress of the audio when it was paused so that it can resume playback.

[0075] A stop event can be reported by a multimedia application when it stops playing an audio. When an audio ends or switches from one audio to another, the multimedia application can report a stop event. In some embodiments, the audio module can determine the stop event independently without the multimedia application reporting it. For example, the audio module can determine the moment when the audio stops playing based on the data obtained for playing the audio, thereby determining the stop event of the audio.

[0076] A timeout event can occur between a start event and a pause event, or between a start event and a stop event. A timeout event can indicate that, while a multimedia application is playing audio, no audio data can be played after a preset duration has expired. The preset duration can be, for example, 500 milliseconds (ms), 1 second (s), or similar values. The present embodiment does not limit the value of the preset duration. The multimedia application enters the audio playback state upon reporting the start event. The multimedia application can simultaneously play audio while acquiring subsequent audio data. Using caching technology, the multimedia application can pre-cache audio data for a period of time following the current playback position. This way, if the network is temporarily poor and audio data cannot be acquired, the multimedia application can still use the cached audio data for audio playback. If the cached audio data is played completely, but the network is still poor and no further audio data can be acquired, the audio will experience pauses. The multimedia application can send the audio data to the audio module. The audio module can then play the audio based on the received audio data. If, after receiving the start event, no audio data can be played after a preset duration has expired, the audio module can determine a timeout event for the audio.

[0077] The resume event may be an event after the timeout event. The resume event may indicate that the audio module has audio data to resume playing again during the process of playing the audio. After determining the timeout event of the audio, the audio module may wait for subsequent playback data of the audio. When the audio data is obtained again, the audio module may determine the resume event of the audio. In some embodiments, the audio module may generate the resume event of the audio when the audio data is continuously obtained within a period of time (such as 500ms, or 1s, or 2s, etc.). The above-mentioned continuous acquisition of audio data within a period of time may indicate that the network is currently stable and good.

[0078] In some embodiments, when the audio being played switches from audio a1 to audio a2, the multimedia application may report a stop event for audio a1 and a start event for audio a2. The stop event for audio a1 may indicate that audio a1 stops playing. The start event for audio a2 may indicate that audio a2 starts playing. This application does not limit the order in which the stop and start events are reported.

[0079] In the scenario of playing audio a1, in response to the operation of pausing the playback, the multimedia application can report a pause event. Then, in response to the operation of continuing to play audio a1, the multimedia application can report a start event.

[0080] As you can see, each audio track can be independently mapped to an audio event. A multimedia application can report the audio event corresponding to a particular audio track based on its playback status. When the playing audio track switches to another, the multimedia application can report the audio event corresponding to the other audio track.

[0081] The above audio events are merely exemplary descriptions of the present application and should not be construed as limiting the present application. Multimedia applications may also report more other types of audio events.

[0082] In some embodiments, video events may also include: start play event, pause play event, stop play event, data exhaustion event, data recovery event, etc. The types of video events can refer to the introduction of audio events above. No further details will be given here.

[0083] In the subsequent embodiments of this application, the jam detection method of this application is specifically described by taking the example of an electronic device detecting jams in multimedia content playback based on user operation events and audio events. The method for detecting jams in multimedia content playback based on user operation events and video events can refer to the method for detecting jams in multimedia content playback based on user operation events and audio events.

[0084] The structure of the electronic device involved in this application is introduced below.

[0085] FIG1 exemplarily shows a schematic diagram of the hardware structure of an electronic device 100 .

[0086] As shown in Figure 1, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio processing module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0087] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0088] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0089] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0090] Processor 110 may also include a memory for storing instructions and data. In some examples, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces processor 110 latency, and thus improves system efficiency.

[0091] The USB interface 130 is an interface that complies with the USB standard. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. The USB interface 130 can also be used to connect headphones to play audio through the headphones.

[0092] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also power the electronic device through the power management module 141.

[0093] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0094] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0095] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0096] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1.

[0097] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0098] The electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.

[0099] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0100] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0101] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the naked eye.

[0102] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0103] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0104] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0105] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0106] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a user operation detection function, an audio playback function, a video playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0107] The electronic device 100 can implement audio functions such as music playback and recording through the audio processing module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0108] The audio processing module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio processing module 170 can also be used to encode and decode audio signals. In some examples, the audio processing module 170 can be set in the processor 110, or some functional modules of the audio processing module 170 can be set in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 170D is used to connect wired headphones.

[0109] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a gravity sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0110] Buttons 190 include a power button, a volume button, etc. Motor 191 can generate vibration prompts. Indicator 192 can be an indicator light that can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0111] The SIM card interface 195 is used to connect a SIM card. A SIM card can be connected to and disconnected from the electronic device 100 by inserting or removing it from the SIM card interface 195. The electronic device 100 may support one or N SIM card interfaces, where N is a positive integer greater than 1. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some examples, the electronic device 100 uses an eSIM, or embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0112] The electronic device 100 may be a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiment of the present application does not limit the specific type of the electronic device 100.

[0113] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplarily described.

[0114] FIG. 2A exemplarily shows a schematic diagram of the software structure of the electronic device 100 .

[0115] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, The system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime (Android Runtime) and system library, and kernel layer.

[0116] The application layer can include a series of application packages.

[0117] As shown in FIG2A , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, short message, multimedia application, etc. Among them, multimedia application may refer to a type of APP related to audio and video playback in the electronic device 100.

[0118] The application framework layer provides APIs and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0119] As shown in FIG. 2A , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, an activity manager, and the like.

[0120] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0121] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0122] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0123] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0124] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0125] The Notification Manager allows applications to display notification information in the status bar (such as the pull-down notification bar). It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the Notification Manager is used to notify the completion of downloads, message reminders, etc. The Notification Manager can also be used to display notifications in the form of icons or scrolling text in the status bar at the top of the system, such as notifications from applications running in the background, or notifications that appear on the screen in the form of dialog windows. For example, text messages can be displayed in the status bar, prompts can be sounded, electronic devices can vibrate, indicator lights can flash, etc.

[0126] The Activity Manager is responsible for managing activities, starting, switching, and scheduling components in the system, as well as managing and scheduling applications. The Activity Manager can be called by upper-level applications to open corresponding activities.

[0127] An activity is an application component that provides users with an interactive window to perform certain operations, such as playing audio, taking photos, making calls, and so on. Each activity can obtain a window for drawing the user interface. A window can fill the screen of the electronic device 100, or it can be smaller than the screen of the electronic device 100, or it can float above other windows. In other words, an activity can be a visual interface or view interface for user operations. Each user interface of an application is an activity, and different user interfaces are different activities. An application can include one or more activities. For example, the audio and video playback interface in a multimedia application is an activity, and the comment viewing interface is another activity.

[0128] Android Runtime includes core libraries and a virtual machine. Android Runtime is responsible for scheduling and management of the Android system.

[0129] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0130] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0131] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0132] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0133] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0134] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0135] A 2D graphics engine is a drawing engine for 2D drawings.

[0136] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0137] The software structure shown in FIG2A is merely an exemplary illustration of the present application and should not be construed as limiting the present application. In some embodiments, the electronic device 100 may include more or fewer software modules than shown in FIG2A, or may combine certain software modules, split certain software modules, or arrange software modules in different locations.

[0138] FIG. 2B exemplarily shows another structural diagram of the electronic device 100 .

[0139] As shown in Figure 2B, the electronic device 100 includes a multimedia application, a freeze detection module, a human-computer interaction module, a network management module, and a multimedia processing module. The multimedia application can refer to the introduction of the above embodiment.

[0140] The human-computer interaction module can be used for the user to interact with the electronic device 100, and to determine the information of the user's interaction with the electronic device 100 through the human-computer interaction interface. For example, the human-computer interaction module can detect user operations on the electronic device 100. The above-mentioned user operations may include but are not limited to: touch operations on the screen of the electronic device 100 (such as click operations, long press operations, swipe up operations, swipe down operations, etc.), air gesture operations on the electronic device 100, and operations on the electronic device 100 based on one or more input devices such as a mouse, keyboard, and stylus. The embodiment of the present application does not limit the types of user operations that the human-computer interaction module can detect. For example, for touch operations on the screen of the electronic device 100, the human-computer interaction module can detect information such as the position coordinates and action time of the touch operation on the screen. The embodiment of the present application does not limit the specific method for the human-computer interaction module to detect user operations. The human-computer interaction module may also be referred to as a user input module or the like. In some embodiments, the human-computer interaction module may be located in the application framework layer shown in Figure 2A.

[0141] The multimedia processing module can be used to process and play multimedia content. The multimedia processing module may include an audio module and a video module. Among them, the audio module may include a software module located in the application framework layer for managing or processing audio. Optionally, the audio module may also include a software program located in the application layer for performing audio management or audio playback. The audio module may also include a hardware device in the electronic device 100 for playing or processing audio, such as the audio processing module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, etc. shown in Figure 1 above. The embodiments of the present application do not limit the content of the audio module.

[0142] The audio module can be used to receive audio events reported by one or more multimedia applications in the electronic device 100. The audio events can be referred to in the above description. The audio module can also receive audio data from the multimedia application and play audio according to the audio data.

[0143] The video module may include a software module for managing or processing videos located in the application framework layer shown in FIG2A . Optionally, the video module may also include a software program located in the application layer for managing or playing videos. The video module may also include a three-dimensional graphics processing library, a two-dimensional graphics engine, a media library, etc. located in the system library shown in FIG2A to enable drawing and display of video images. The video module may also include a hardware device in the electronic device 100 for playing or processing videos. The embodiments of the present application do not limit the content of the video module.

[0144] The video module can be used to receive video events reported by one or more multimedia applications in the electronic device 100. The video module can also receive video data from the multimedia application and play the video based on the video data. In some embodiments, after receiving an event indicating the start of video playback, if the video module has no video data to play for a preset duration, the video module can generate a data-out event for the video. The video module can then wait for subsequent playback data for the video. When video data is acquired again, the video module can generate a data-resumed event for the video.

[0145] The jam detection module can be used to detect jams in multimedia content playback. In some embodiments, the jam detection module can register user operation event monitoring and reporting with the human-computer interaction module. After detecting a user operation, the human-computer interaction module can send a user operation event to the jam detection module to indicate to the jam detection module information about the user operation on the electronic device 100. The jam detection module can also register audio event monitoring and reporting with the audio module. After receiving the audio event reported by the multimedia application, the audio module can send the audio event to the jam detection module. In this way, the jam detection module can detect whether the audio currently playing in the electronic device 100 is jammed based on the above-mentioned user operation event and audio event.

[0146] If the audio is not stuck, the stuck detection module can continue to detect and wait for new user operation events and audio events.

[0147] If the audio is stuck, the stuck detection module can instruct the network management module to adjust the network so that the electronic device 100 uses a network with better quality for communication.

[0148] In some embodiments, the jam detection module can register video event monitoring and reporting with the video module. After receiving the video event reported by the multimedia application, the video module can send the video event to the jam detection module. In this way, the jam detection module can detect whether the video currently playing on the electronic device 100 is jammed based on the above user operation events and video events.

[0149] As can be seen, the jam detection module can detect whether the multimedia content playback of any multimedia application in the electronic device 100 is jammed from a system level. This eliminates the need for developers of multimedia applications to perform additional operations to detect jams in multimedia content playback, saving development costs and improving development efficiency.

[0150] The network management module can be used to manage the network accessed by the electronic device 100. In some embodiments, the network management module can use one or more strategies such as Wi-Fi and cellular switching, cellular cell switching, primary and secondary card switching, multi-network switching, multi-network concurrency, and Wi-Fi roaming to adjust the network used by the electronic device 100 for data communication, so that the electronic device 100 uses a higher quality network for communication.

[0151] The above-mentioned Wi-Fi and cellular switching can mean that when the electronic device 100 is connected to both the Wi-Fi network and the cellular mobile network, the network management module can switch the network used for data communication with other devices between the Wi-Fi network and the cellular mobile network based on the quality of the Wi-Fi network and the cellular mobile network. When the quality of the Wi-Fi network is better than the quality of the cellular mobile network, the network management module can adjust the network used for data communication between the electronic device 100 and other devices to the Wi-Fi network. When the quality of the cellular mobile network is better than the Wi-Fi network, the network management module can adjust the network used for data communication between the electronic device 100 and other devices to the cellular network.

[0152] The primary and secondary card switching may mean that the electronic device 100 includes a primary SIM card and a secondary SIM card, and the network management module may switch the network for data communication with other devices between the cellular mobile networks corresponding to the primary SIM card and the secondary SIM card.

[0153] The network management module adjusts the network through which the electronic device 100 communicates from one network to another, which may include but is not limited to: adjusting from a Wi-Fi network to a cellular mobile network, adjusting from the cellular mobile network corresponding to the primary SIM card to the cellular mobile network corresponding to the secondary SIM card, adjusting from a cellular mobile network of one standard to a cellular mobile network of another standard (such as adjusting from a 4G cellular mobile network to a 5G cellular mobile network), adjusting the cellular mobile network from one resident cell to another resident cell, and the like.

[0154] The embodiment of the present application does not limit the method by which the network management module adjusts the network for communication of the electronic device 100.

[0155] In some embodiments, the freeze detection module and the network management module can both be located in the application framework layer shown in Figure 2A above.

[0156] In some embodiments, the jam detection module may also obtain information about activities in the electronic device 100 from the activity manager shown in FIG. 2A to determine whether the electronic device 100 currently contains an activity for a multimedia playback interface and whether the multimedia playback interface is currently displayed on the screen. The multimedia playback interface can be used to control the playback of multimedia content. The multimedia playback interface may include a video playback interface and an audio playback interface. Some multimedia content combines video and audio, playing the two together. When the electronic device 100 plays this multimedia content, it may simultaneously display the video playback interface and the associated audio. Other multimedia content is independent video, meaning that the video is not associated with audio. When the electronic device 100 plays this other multimedia content, it may display the video playback interface without playing the audio. Some audio content is independent and not associated with the video. When the electronic device 100 plays this audio content, it may display the audio playback interface.

[0157] In some embodiments, the electronic device 100 may further include a process management module (not shown in FIG. 2B ). The process management module may be used to manage processes in the electronic device 100. The process management module may create processes, allocate CPU time slice resources to processes, manage process status, and the like. The jam detection module may obtain information about running processes in the electronic device 100 from the process management module to determine whether a multimedia application in the electronic device 100 is in a running state, and whether the multimedia application is running in the foreground or background.

[0158] The following describes the multimedia content playback freeze detection scenario provided by this application.

[0159] 3A to 3H exemplarily illustrate some schematic diagrams of scenarios for detecting playback freezes of multimedia content.

[0160] Here, multimedia content that is a combination of audio and video is taken as an example for explanation.

[0161] As shown in Figure 3A, electronic device 100 can display user interface 310. User interface 310 can be a multimedia playback interface (i.e., an audio and video playback interface) of a multimedia application. User interface 310 may include a status bar 311, a video display area 312, a follow control 313, a like control 314, a comment control 315, and a favorite control 316.

[0162] The status bar 311 may include a signal strength indicator 311A, a Wi-Fi indicator 311B, a battery indicator 311C, and time information 311D.

[0163] Signal strength indicator 311A ​​may be used to indicate the signal strength of the electronic device 100 when accessing the cellular mobile network. In some embodiments, the status bar 311 may include one or more signal strength indicators. For example, if the electronic device 100 includes two SIM cards, the electronic device 100 may display two signal strength indicators in the status bar 311. These two signal strength indicators may respectively represent the signal strength of the electronic device 100 when accessing the cellular mobile network through the two SIM cards.

[0164] Wi-Fi identifier 311B may be used to indicate that electronic device 100 is connected to a Wi-Fi network. In some embodiments, the status bar 311 may display both signal strength indicator 311A ​​and Wi-Fi identifier 311B, indicating that electronic device 100 is currently communicating data with other devices via a Wi-Fi network.

[0165] The power level indicator 311C may be used to indicate the current power level of the electronic device 100 .

[0166] The time information 311D may be used to indicate the current time.

[0167] The video display area 312 can be used to display video images. When the video and audio are combined, the video and audio are integrated, and the electronic device 100 can play the audio and video synchronously. In some embodiments, when the audio freezes, the video played synchronously with the audio will also freeze. That is, the audio and video freezes simultaneously.

[0168] A follow control 313, a like control 314, a comment control 315, and a favorite control 316 may be displayed above the video screen. The follow control 313 can be used to follow the publisher of the audio or video displayed in the video display area 312. The like control 314 can be used to like the audio or video displayed in the video display area 312. The comment control 315 can be used to call up the comment interface for commenting on the audio or video displayed in the video display area 312. The favorite control 316 can be used to favorite the audio or video displayed in the video display area 312.

[0169] During the audio and video playback process shown in FIG3A , the electronic device 100 can simultaneously play the audio and video while acquiring subsequent audio and video playback data online through a network (e.g., a Wi-Fi network). The electronic device 100 can pre-cache audio and video playback data for a period of time following the current playback position.

[0170] In response to a click operation in the video display area 312 shown in FIG3A , the electronic device 100 can pause the audio and video and display the user interface 310 shown in FIG3B . The click operation for pausing the audio and video can be performed at any location in the video display area 312 except where the follow control 313 , like control 314 , comment control 315 , and favorite control 316 are located.

[0171] As shown in FIG3B , when the audio or video is paused, the electronic device 100 may display a play control 312A in the video display area 312 . The play control 312A may indicate that the audio or video is currently paused and may be used to trigger the electronic device 100 to continue playing the audio or video. In some embodiments, the electronic device 100 is not limited to triggering the electronic device 100 to continue playing the audio or video by operating the play control 312A. In the state where the audio or video is paused, the electronic device 100 may respond to a click operation at any location in the video display area 312 except for the location where the function controls (such as the follow control 313 , the like control 314 , the comment control 315 , and the favorite control 316 ) are located, and continue playing the audio or video.

[0172] In response to the operation of pausing the playback of audio and video as shown in Figure 3A, the multimedia application in the electronic device 100 can report a pause event to the audio module, and the user interaction module can send a user operation event corresponding to the operation to the jam detection module. The user operation event may include information such as the action position and action time of the operation on the screen. The audio module can also send the pause event to the jam detection module. The jam detection module can record the above-mentioned user operation event and the pause event. Optionally, when pausing the playback of audio and video, the multimedia application can also report a pause event to the video module to indicate that the video is paused. Then, the video module can send the pause event to the jam detection module.

[0173] In some embodiments, the jam detection module in the electronic device 100 can identify the user operation based on the received user operation event and determine whether the user operation is a preset operation. For example, the preset operation includes an upward sliding operation and a downward sliding operation, but does not include a click operation. The jam detection module can identify that the user operation shown in Figure 3A is a click operation based on the user operation event. The jam detection module can determine that the click operation is not a preset operation. In the case that the user operation indicated by the user operation event is not a preset operation, the jam detection module can continue to wait for new user operation events and audio events.

[0174] As shown in FIG3B , when the audio and video are in a paused state, in response to a click operation on the play control 312A, the electronic device 100 can continue to play the audio and video and display the user interface 310 shown in FIG3C .

[0175] As shown in Figure 3C, the electronic device 100 can display the video screen of the video continuing to play in the video display area 312, and continue to play the audio associated with the video. In response to the operation of playing audio shown in Figure 3B, the multimedia application in the electronic device 100 can report a start event to the audio module, and the user interaction module can send a user operation event corresponding to the operation to the jam detection module. The audio module can also send the start event to the jam detection module. It should be noted that the above-mentioned start event can be reported by the multimedia application after obtaining the playback data starting from the position where the playback continues in the audio and video corresponding to the content displayed in the video display area 312.

[0176] The freeze detection module can record the above user operation event and the start event. The freeze detection module can identify the user operation shown in FIG3B as a click operation based on the above user operation event, thereby determining that the click operation is not a preset operation.

[0177] When the electronic device 100 plays one audio or video, the electronic device 100 can switch to play another audio or video according to user operations.

[0178] As shown in Figure 3C, in response to an upward sliding operation in the video display area 312, the electronic device 100 may display the user interface 320 shown in Figure 3D. The user interface 320 may be an audio and video playback interface displayed by the electronic device 100 during the upward sliding operation.

[0179] As shown in FIG3D , user interface 320 may include video screen 321 and video screen 322. Video screen 321 and video screen 322 may be video screens corresponding to different audio and video. Video screen 321 may be the video screen displayed in video display area 312 shown in FIG3C .

[0180] In the process of the user's finger gradually sliding upward, the electronic device 100 can gradually move the video screen 321 and the video screen 322 upward. The video screen 321 gradually moves upward, and the content above the video screen 321 that exceeds the display area is hidden. The video screen 322 gradually moves upward, and the content below the video screen 321 that was originally hidden is gradually displayed. In other words, as the user's finger slides upward, the content displayed on the screen of the video screen 321 becomes less and less, and the display area it occupies becomes smaller and smaller, while the content displayed on the screen of the video screen 322 becomes more and more, and the display area it occupies becomes larger and larger.

[0181] In some embodiments, when the display area occupied by the video screen 322 exceeds the preset area size, when the swipe-up operation ends (i.e., the finger performing the swipe-up operation leaves the screen), the electronic device 100 can switch to playing the audio and video corresponding to the video screen 322 and display the user interface 330 shown in Figure 3E.

[0182] In some embodiments, the audio and video corresponding to the video screen 322 is the next audio and video to be played of the audio and video corresponding to the video screen 321. The electronic device 100 can pre-acquire and cache the playback data of the audio and video corresponding to the video screen 322. For example, based on the caching technology, when playing the audio and video corresponding to the video screen 321, the data obtained by the electronic device 100 online through the network can include all the playback data of the audio and video corresponding to the video screen 321, and can also include part or all of the playback data of the subsequent audio and video (i.e., the audio and video corresponding to the video screen 322). Alternatively, the electronic device 100 can also obtain the playback data of the audio and video corresponding to the video screen 322 online through the network in response to the upward sliding operation shown in Figure 3C. Among them, the electronic device 100 can start to obtain the playback data of the audio and video corresponding to the video screen 322 during the upward sliding operation, or it can also start to obtain the playback data of the audio and video corresponding to the video screen 322 after the upward sliding operation is completed.

[0183] In response to the swipe-up operation shown in Figures 3C and 3D , the multimedia application in electronic device 100 can report a stop event and a start event to the audio module, and the user interaction module can send the user operation event corresponding to the swipe-up operation to the jam detection module. The audio module can also send the above-mentioned stop event and start event to the jam detection module. The above-mentioned stop event can be used to instruct the audio and video corresponding to video screen 321 to stop playing. The above-mentioned start event can be used to instruct the audio and video corresponding to video screen 322 to start playing. It should be noted that the above-mentioned start event can be reported by the multimedia application when it obtains the playback data of the audio and video corresponding to video screen 322. In other words, when the audio and video corresponding to video screen 322 need to be played, the multimedia application will only report the start event if there is playback data capable of playing the audio and video corresponding to video screen 322. If the audio and video corresponding to video screen 322 need to be played, but the multimedia application has not obtained the playback data of the audio and video corresponding to video screen 322, the multimedia application will not report the start event.

[0184] After the jam detection module receives the user operation event corresponding to the above-mentioned swipe-up operation, it can determine that the user operation is a swipe-up operation based on the user operation event, thereby determining that the user operation shown in Figures 3C and 3D is a preset operation. When a preset operation is detected, the jam detection module can detect whether a start event is received within a preset time period starting from the preset operation action time. The preset time period can be 2s, or 3s, etc. The embodiment of the present application does not limit the length of the preset time period. If the jam detection module receives a start event within the preset time period, the jam detection module can continue to wait for new user operation events and audio events. If the jam detection module does not receive a start event within the preset time period, the jam detection module can instruct the network management module in the electronic device 100 to adjust the network so that the electronic device 100 uses a network with better quality for communication. In this way, the electronic device 100 can obtain the playback data of the audio and video corresponding to the video screen 322 as soon as possible to eliminate the jam of the audio and video.

[0185] Figures 3D and 3E illustrate exemplary scenarios in which the audio and video to be played after the switch is performed are not interrupted. In response to the upward swipe operation shown in Figures 3C and 3D , the multimedia application can quickly obtain the playback data of the next audio and video to be played after the switch, and then play the switched audio and video according to the playback data, displaying the video screen 322 shown in Figure 3D and the video display area 331 shown in Figure 3E .

[0186] As shown in Figure 3E, the user interface 330 may include a video display area 331, a follow control 332, a like control 333, a comment control 334, and a favorite control 335. The video screen displayed in the video display area 331 is the complete screen of the video screen 322 shown in Figure 3D. The audio and video corresponding to the video screen in the video display area 331 can be the next audio and video to be played after the audio and video corresponding to the video screen in the video display area 312 shown in Figure 3C. The follow control 332 can be used to follow the publisher of the audio and video in the video display area 331. The like control 333 can be used to like the audio and video played in the video display area 331. The comment control 334 can be used to call up the comment interface so that comments can be made on the audio and video played in the video display area 331. The favorite control 335 can be used to favorite the audio and video played in the video display area 331.

[0187] In some embodiments, when switching audio and video, the audio and video to be played after the switch may be stuck.

[0188] 3F to 3H exemplarily illustrate a scenario in which the audio and video playback is interrupted when switching between audio and video.

[0189] For example, during the audio and video playback process shown in FIG3C , in response to a swipe-up operation, if the electronic device 100 cannot obtain the playback data of the next audio or video to be played due to a poor network or other reasons, the electronic device 100 may display the user interface 340 shown in FIG3F . User interface 340 may be the audio and video playback interface displayed by the electronic device 100 during the swipe-up operation.

[0190] As shown in Figure 3F, the user interface 340 may include a video screen 341 and a display area 342. The video screen 341 can refer to the video screen 321 shown in Figure 3D above. The display area 342 can be used to play the next audio and video to be played corresponding to the video screen 341. Since the electronic device 100 has not obtained the playback data of the next audio and video to be played, the electronic device 100 can display the prompt information shown in Figure 3F in the display area 342. For example, the content of the prompt information can be "Network timeout, please wait...". The prompt information can indicate that the next audio and video to be played is stuck. This application does not limit the content of the prompt information.

[0191] When the swipe-up operation is completed, the electronic device 100 may display the user interface 350 shown in FIG3G . The user interface 350 may include a video display area 351. Since the electronic device 100 has not yet acquired the playback data for the audio or video to be played, the electronic device 100 may display a prompt message in the video display area 351. This prompt message can refer to the prompt message in the display area 342 shown in FIG3F .

[0192] In response to the swipe-up operation shown in Figures 3C and 3F, the user interaction module in the electronic device 100 can send a user operation event corresponding to the swipe-up operation to the jam detection module. The jam detection module can determine that the user operation is a swipe-up operation based on the user operation event, thereby determining that the user operation shown in Figures 3C and 3F is a preset operation. In the case of detecting a preset operation, the jam detection module can detect whether a start event is received within a preset time period starting from the preset operation action time. Since the multimedia application has not yet obtained the playback data of the next audio and video to be played, the jam detection module has not received a start event within the above-mentioned preset time period. The jam detection module can determine that the multimedia content playback is jammed, thereby instructing the network management module to adjust the network.

[0193] For example, the network management module may adjust the network used for data communication between the electronic device 100 and other devices from a Wi-Fi network to a cellular mobile network.

[0194] As shown in Figure 3G, a Wi-Fi logo 311B may be displayed in the user interface 350. The Wi-Fi logo 311B may indicate that the network on which the electronic device 100 is currently communicating data with other devices is a Wi-Fi network. When using a Wi-Fi network, the multimedia application may be unable to obtain the playback data of the next audio or video to be played. The network management module may adjust the network indication according to the jam detection module and search for a network with better quality than the Wi-Fi network. When it is found that the quality of the cellular mobile network accessed by the electronic device 100 is better than the Wi-Fi network, the network management module may adjust the network for data communication to a cellular mobile network.

[0195] In some embodiments, when the Wi-Fi network is adjusted to a cellular mobile network, the electronic device 100 may display the cellular mobile network identifier 311E shown in FIG3H . The cellular mobile network identifier 311E may indicate that the network on which the electronic device 100 currently communicates data with other devices is a cellular mobile network. The electronic device 100 may use a cellular mobile network to communicate with other devices (such as an audio and video server) to obtain playback data of audio and video to be played online. When the playback data is obtained, the electronic device 100 may play audio and video according to the playback data, and display the video screen shown in FIG3H in the video display area 351. Among them, after the multimedia application in the electronic device 100 obtains the playback data of the audio and video to be played, it may report a start event to the audio module. The audio module may then send the start event to the freeze detection module.

[0196] In some embodiments, after the jam detection module determines that the multimedia content playback is jammed, it can instruct the network management module to adjust the network. When a start event is received, the jam detection module can stop instructing the network management module to adjust the network.

[0197] The network adjustment scenarios shown in FIG3F to FIG3H are merely exemplary descriptions of the present application and should not be construed as limiting the present application. The network management module may also use other methods to adjust the network for communication of the electronic device 100.

[0198] It should be noted that the scenarios shown in Figures 3A to 3H above use the swipe operation as a preset operation and the click operation as an example to introduce the process of detecting multimedia content playback jams. The embodiments of the present application do not limit the type of the preset operation. In some embodiments, the click operation can also be a preset operation. The criteria for setting the above-mentioned preset operations may include: setting one or more operations that can trigger the multimedia application to play audio as preset operations.

[0199] As can be seen from the scenarios shown in Figures 3A to 3H above, in response to the preset operation for triggering the playback of multimedia content, the multimedia application in the electronic device 100 will report a start event when the playback data of the multimedia content to be played can be played. If the multimedia application fails to report the start event after the electronic device 100 receives the preset operation, it may indicate that the multimedia application has not obtained the playback data of the multimedia content to be played, and the playback of the multimedia content is stuck. Therefore, the present application can use a stuck detection module to detect whether a start event is received within a preset time period after the preset operation is performed, so as to accurately detect whether the multimedia content to be played is stuck in the scenario where the multimedia content playback is triggered by the preset operation.

[0200] In some embodiments, the jam detection module can also determine whether it is necessary to wait for a start event to detect jams in multimedia content playback based on the playback status of the multimedia content, the type of user operation, and the location of the operation. This is illustrated using a click operation as an example. Since there may be multiple functional controls on the multimedia playback interface, click operations at different locations on the multimedia playback interface may have different purposes. Furthermore, under different playback states of the multimedia content, click operations at the same location on the multimedia playback interface may also have different purposes.

[0201] For example, on the audio and video playback interface shown in FIG3A , a click operation on an area other than the functional controls (such as the follow control 313, the like control 314, the comment control 315, and the favorite control 316) in the video display area 312 can be used to play or pause the audio and video. When the audio and video are playing, a click operation on an area other than the functional controls in the video display area 312 can be used to pause the audio and video. When the audio and video are paused, a click operation on an area other than the functional controls in the video display area 312 can be used to play the audio and video.

[0202] When the audio is paused, the multimedia application can report a pause event. After receiving the pause event, if the jam detection module determines that the user operation is a click operation on a preset area on the screen based on the received user operation event, the jam detection module can detect whether a start event is received within a preset time period starting from the time of the above-mentioned click operation. For example, the above-mentioned preset area can be the area other than the location of the function controls in the video display area 312 shown in Figure 3A. Among them, if the jam detection module receives a start event within a preset time period starting from the time of the above-mentioned click operation, the audio is not jammed, and the jam detection module can continue to wait for new user operation events and audio events. If the jam detection module does not receive a start event within a preset time period starting from the time of the above-mentioned click operation, the audio is jammed, and the jam detection module can instruct the network management module to adjust the network so that the electronic device 100 uses a better quality network for communication.

[0203] The above-mentioned preset area can be determined according to the audio and video playback interface. The preset areas corresponding to different audio and video playback interfaces can be different. The electronic device 100 can determine whether the user operation is an operation that triggers audio playback based on whether the user operation acts on the preset area of ​​the audio and video playback interface.

[0204] In the above embodiment, the electronic device 100 can not only accurately detect whether the multimedia content to be played is stuck in the scenario where audio playback is triggered by a swipe up operation or a swipe down operation, but can also accurately detect whether the multimedia content to be played is stuck in the scenario where multimedia content playback is triggered by a click operation.

[0205] The above embodiments can more accurately identify the user operation used to trigger the playback of multimedia content by combining the playback status of the multimedia content, the type of user operation, and the action location, thereby detecting whether the multimedia content to be played is stuck in more scenarios where the playback of multimedia content is triggered by user operations.

[0206] In some embodiments, in addition to detecting whether the multimedia content is stuck when the multimedia content playback is triggered, the stuck detection module can also detect whether there is a stuck during the multimedia content playback process.

[0207] FIG. 4A and FIG. 4B exemplarily show schematic diagrams of other scenarios for detecting playback freezes of multimedia content.

[0208] As shown in FIG4A , electronic device 100 may display user interface 410. User interface 410 may be an audio and video playback interface for a multimedia application. User interface 410 may include a Wi-Fi identifier 411 and a video display area 412. Wi-Fi identifier 411 may indicate that the network currently used by electronic device 100 for data communication with other devices is a Wi-Fi network. Video display area 412 may display a video image.

[0209] Here, the multimedia content played by the electronic device 100 is still a combination of audio and video as an example for description. The electronic device 100 can play the video and the audio associated with the video synchronously. That is, when the electronic device 100 displays the video screen in the video display area 412 shown in Figure 4A, it is also playing the audio.

[0210] Among them, when starting to play audio and video, the multimedia application in the electronic device 100 can report a start event. The jam detection module can receive the start event. The electronic device 100 can play the corresponding video and audio based on the playback data of the audio and video to be played that has been acquired, while acquiring subsequent playback data. In the process of playing an audio and video, if the playback data acquired by the electronic device 100 has been played, and the electronic device 100 has not yet acquired subsequent playback data, then the electronic device 100 currently has no playback data to play the audio and video. At this time, both the audio and video are jammed. The electronic device 100 can display the prompt message 412A shown in Figure 4A in the video display area 412. The content of the prompt message 412A may include "Network timeout, please wait...". The prompt message 412A can be used to prompt the user that the audio and video are jammed. This application does not limit the content of the prompt message 412A.

[0211] Furthermore, if no playback data is available for audio or video playback after a preset time period, the audio module can generate a timeout event and send the timeout event to the jam detection module. Upon receiving the timeout event, the jam detection module can determine that the audio is jammed and instruct the network management module to adjust the network so that the electronic device 100 can communicate using a higher-quality network.

[0212] For example, the network management module may adjust the network used for data communication between the electronic device 100 and other devices from a Wi-Fi network to a cellular mobile network.

[0213] As shown in Figure 4B, the electronic device 100 can display a cellular mobile network logo 413. The cellular mobile network logo 413 can indicate that the network on which the electronic device 100 currently communicates data with other devices is a cellular mobile network. The electronic device 100 can use the cellular mobile network to communicate with other devices (such as an audio and video server) to obtain the playback data of the audio and video to be played online. When the playback data is obtained, the electronic device 100 can resume the playback of the audio and video according to the playback data, and display the video screen shown in Figure 4B in the video display area 412. In this way, the jamming of the audio and video is eliminated. Among them, after the multimedia application in the electronic device 100 obtains the playback data of the audio and video to be played, it can send the playback data to the audio module. After the audio module receives the playback data, it can generate a resume event and send the resume event to the jam detection module.

[0214] In some embodiments, after receiving the timeout event and before receiving the resume event, the jam detection module may instruct the network management module once or multiple times to adjust the network on which the electronic device 100 communicates. For example, when the timeout event is received, the jam detection module may instruct the network management module for the first time to adjust the network on which the electronic device 100 communicates. After instructing the network management module to adjust the network on which the electronic device 100 communicates, the jam detection module may wait for the resume event within a preset time period. If the resume event is not received within the preset time period after instructing the network management module to adjust the network on which the electronic device 100 communicates, the jam detection module may instruct the network management module again to adjust the network on which the electronic device 100 communicates. The jam detection module may instruct the network management module to adjust the network on which the electronic device 100 communicates multiple times according to the above method until the resume event is received.

[0215] It is understandable that after the network management module adjusts the network based on a single instruction from the jam detection module, the adjusted network may still be poor. The electronic device 100 may still be unable to communicate with devices such as audio and video servers based on the adjusted network and obtain the playback data for the audio and video to be played. Consequently, the audio module will be unable to generate a resume event for a long time. The jam detection module can again instruct the network management module to adjust the network used by the electronic device 100 for communication. The network management module can then adjust the network again so that the electronic device 100 uses a higher-quality network for communication. For example, the network management module can first adjust the network used by the electronic device 100 for data communication with other devices from a Wi-Fi network to a cellular mobile network. Upon receiving another instruction from the jam detection module to adjust the network, the network management module can use strategies such as cellular cell switching or primary / secondary card switching to enable the electronic device 100 to use a higher-quality network for communication. This allows the electronic device 100 to quickly obtain the playback data for the audio and video to be played, eliminating audio and video jams.

[0216] As shown in the scenarios of Figures 4A and 4B above, a timeout event can indicate that audio playback has experienced a pause. Based on the timeout event, electronic device 100 can accurately detect whether audio playback has experienced a pause and, if so, adjust the network to use a higher-quality network to eliminate the pause. This can further enhance the user's audio listening experience.

[0217] Based on the above-mentioned multimedia content playback jam detection scenario, the following specifically introduces the multimedia content playback jam detection method provided by this application. Here, an electronic device performs multimedia content playback jam detection based on user operation events and audio events as an example for introduction.

[0218] FIG5 exemplarily shows a flow chart of a method for detecting jamming in multimedia content playback.

[0219] As shown in Figure 5, electronic device 100 may include a freeze detection module 501, a human-computer interaction module 502, an audio module 503, a network management module 504, and a multimedia application 505. For details about freeze detection module 501, human-computer interaction module 502, audio module 503, and network management module 504, refer to the description of Figure 2B above. Multimedia application 505 may be an app on electronic device 100 that plays multimedia content. The electronic device 100 is not limited to multimedia application 505 and may include more multimedia applications.

[0220] S511 , the freeze detection module 501 detects whether a multimedia application is in a running state.

[0221] Multiple apps may be running simultaneously in the electronic device 100. Some apps may be running in the foreground. Other apps may be running in the background. When a multimedia application is running in the electronic device 100, the electronic device 100 may play multimedia content through the running app. Therefore, when a multimedia application is running, the electronic device 100 may experience jamming in the multimedia content playback. Then, the electronic device 100 can detect whether the multimedia content playback is jammed through the jam detection module 501.

[0222] In some embodiments, the jam detection module 501 can detect whether a multimedia application is in a running state by obtaining process information in the electronic device 100. A process can represent a program running in the system and is an abstraction of a running program. Each running program can constitute a process. If there is a process of a multimedia application (such as multimedia application 505) in the electronic device 100, the jam detection module 501 can determine whether a multimedia application is in a running state in the electronic device 100. Among them, a process can include a foreground process and a background process. The foreground process can include the process currently being displayed on the screen and some system processes. The foreground process can be a process that is interacting with the user. The background process can include a process that is currently invisible to the user, that is, a process running in the background. The electronic device 100 can determine whether the multimedia application is running in the foreground or in the background based on whether the process of the multimedia application is a foreground process or a background process.

[0223] The method of determining whether a multimedia application is in a running state is not limited to the process, and the jam detection module 501 can also determine whether a multimedia application is in a running state by other methods. The embodiment of the present application does not limit the method of determining whether a multimedia application is in a running state.

[0224] In some embodiments, when it is detected that a multimedia application is in running state, the electronic device 100 may execute the following steps S512 to S513.

[0225] S512 , the jam detection module 501 registers the user operation event monitoring and reporting to the human-computer interaction module 502 .

[0226] The human-computer interaction module 502 can be used to detect user operations on the electronic device 100. The human-computer interaction module 502 can receive user operation event monitoring reports registered by other modules or applications in the electronic device 100. When a user operation is detected, the human-computer interaction module 502 can send the user operation event to these software or applications that have registered for user operation event monitoring and reporting.

[0227] The freeze detection module 501 registers the user operation event monitoring and reporting to the human-computer interaction module 502, and can obtain the information of the user operation received by the electronic device 100.

[0228] S513 , the jam detection module 501 registers audio event monitoring and reporting with the audio module 503 .

[0229] The audio module 503 may receive audio events reported by multimedia applications, and send the audio events to other modules or applications that have registered audio event reporting with the audio module 503 .

[0230] The jam detection module 501 registers audio event monitoring and reporting with the audio module 503 to obtain the audio playback status.

[0231] In some embodiments, the detection is not limited to multimedia applications. When the freeze detection module 501 detects that at least one APP is running in the electronic device 100, steps S512 and S513 may be executed. This APP can be any APP.

[0232] In some embodiments, in step S511 , when it is detected that no multimedia application is in the running state, the electronic device 100 may execute step S511 in a loop to wait for the electronic device 100 to run at least one multimedia application.

[0233] Step S511 may be optional. For example, after the electronic device 100 is powered on and the system is started, the jam detection module 501 may execute steps S512 and S513. That is, the jam detection module 501 may not need to detect whether a multimedia application is running in the electronic device 100.

[0234] S514 , the user performs operation 1 on the electronic device 100 , where operation 1 is to play audio a1 .

[0235] In some embodiments, operation 1 may be an operation of switching audio a2 to audio a1 when the electronic device 100 is playing audio a2. Operation 1 may refer to the swipe-up operation shown in FIG3D .

[0236] In some embodiments, operation 1 may be an operation of triggering the electronic device 100 to continue playing the audio a1 when the audio a1 is in a paused playback state. Operation 1 may refer to the click operation shown in FIG3B .

[0237] This embodiment of the application does not limit operation 1.

[0238] Here, operation 1 is taken as an example of a user operation on the multimedia playback interface of the multimedia application 505 .

[0239] In some embodiments, the audio a1 may be associated with a video. When the electronic device 100 plays the audio a1, it may simultaneously play the video associated with the audio a1. That is, the multimedia content played by the electronic device 100 is a combination of the audio a1 and the video. When the audio a1 is playing, the electronic device 100 may display a video screen in the multimedia playback interface displayed on the screen. Alternatively, the audio a1 is independent and not associated with the video. And the multimedia content played by the electronic device 100 is an independent audio a1. This application does not limit the type of audio a1.

[0240] The human-computer interaction module 502 may detect operation 1 and then execute the following steps S515 and S516.

[0241] S515 , the human-computer interaction module 502 sends a user operation event corresponding to operation 1 to the freeze detection module 501 .

[0242] S516 , the human-computer interaction module 502 sends a user operation event corresponding to operation 1 to the multimedia application 505 .

[0243] The human-computer interaction module 502 can identify which APP in the electronic device 100 is operated on by Operation 1. Since Operation 1 is a user operation on the multimedia playback interface of the multimedia application 505, the human-computer interaction module 502 can send a user operation event corresponding to Operation 1 to the multimedia application 505 so that the multimedia application 505 can respond to Operation 1 and perform a corresponding operation.

[0244] The user operation event corresponding to the operation 1 may include information such as the position where the operation 1 is applied on the screen and the application time. The user operation event may be used to determine the operation type of the operation 1.

[0245] For example, the user operation event corresponding to Operation 1 may include: [InputEvent1: (x1, y1), 0-0-0, t1], [InputEvent2: (x2, y2), 1-0-0, t2]. In InputEvent1, 0-0-0 may indicate a finger press, (x1, y1) may indicate the coordinates of the finger on the screen in Operation 1, and t1 may indicate the time when the finger is pressed during Operation 1. In InputEvent2, 1-0-0 may indicate a finger release, (x2, y2) may indicate the coordinates of the finger on the screen in Operation 1, and t2 may indicate the time when the finger is released during Operation 1.

[0246] If the difference between x1 and x2 does not exceed the distance threshold, and the difference between y1 and y2 does not exceed the distance threshold, then the finger press and release positions in Operation 1 are substantially the same. Operation 1 is a click or long press. Furthermore, if the difference between t1 and t2 does not exceed the time threshold, then Operation 1 is a click. If the difference between t1 and t2 exceeds the time threshold, then Operation 1 is a long press.

[0247] If the difference between x1 and x2 does not exceed the distance threshold, and the difference between y1 and y2 exceeds the distance threshold, then operation 1 is an up-swipe operation or a down-swipe operation. Furthermore, if y1 is greater than y2, operation 1 can be an up-swipe operation. If y1 is less than y2, operation 1 can be a down-swipe operation.

[0248] If the difference between x1 and x2 exceeds the distance threshold, and the difference between y1 and y1 does not exceed the distance threshold, then operation 1 is a left swipe operation or a right swipe operation. Furthermore, if x1 is greater than x2, operation 1 is a left swipe operation. If x1 is less than x2, operation 1 is a right swipe operation.

[0249] The content of the above user operation events is only an illustrative description of this application and should not constitute a limitation to this application.

[0250] S517 : The multimedia application 505 obtains the playback data of the audio a1 in response to operation 1 .

[0251] The multimedia application 505 can identify the type of operation 1 (e.g., an upward swipe operation) based on the user operation event corresponding to operation 1. The multimedia application 505 can then determine that operation 1 is for playing audio a1. The multimedia application 505 can obtain the playback data of audio a1.

[0252] In some embodiments, before operation 1, the electronic device 100 obtains and caches the playback data of the audio a1. The multimedia application 505 can directly read the playback data of the audio a1 from the memory and play it.

[0253] In other embodiments, the electronic device 100 does not store the playback data for playing the audio a1. The multimedia application 505 can communicate with an audio and video server or other device via a network to obtain the playback data of the audio a1 online.

[0254] The embodiment of the present application does not limit the method by which the multimedia application 505 obtains the playback data of the audio a1.

[0255] S518 : The multimedia application 505 sends a start event of the audio a1 to the audio module 503 , instructing the audio module 503 to play the audio a1 .

[0256] When the playback data for playing audio a1 is obtained, the multimedia application 505 can send a start event of audio a1 to the audio module 503 and instruct the audio module 503 to play audio a1.

[0257] S519: The audio module 503 plays the audio a1.

[0258] The audio module 503 may start playing the audio a1 according to the playing data of the audio a1.

[0259] S520 , the audio module 503 sends a start event of the audio a1 to the freeze detection module 501 .

[0260] S521 , the freeze detection module 501 detects whether the current interface is a multimedia playback interface.

[0261] In some embodiments, after receiving the user operation event corresponding to Operation 1 (i.e., step S515), the jam detection module 501 can detect whether the current interface is a multimedia playback interface to determine whether Operation 1 is a user operation on the multimedia playback interface. The multimedia interface can be used to control the playback of multimedia content. For example, the multimedia playback interface may include, but is not limited to, a short video interface, a live video interface, and the like.

[0262] It is understandable that if operation 1 is an operation performed on the multimedia playback interface, then operation 1 is likely to be used to control the playback of multimedia content. The jam detection module 501 can further combine the information of operation 1 to determine whether it is necessary to wait for the start event. If operation 1 is not an operation performed on the multimedia playback interface, then the probability that operation 1 is used to control audio playback is low. To avoid false detection of audio jams, the jam detection module 501 can continue to wait for new user operation events without waiting for the start event after receiving the user operation event corresponding to operation 1.

[0263] In some embodiments, the freeze detection module 501 can obtain information about the activity corresponding to the current interface and, based on the activity, determine whether the current interface is a multimedia playback interface. If the activity is an activity of a multimedia playback interface in the multimedia application 505, then the current interface is a multimedia playback interface. Otherwise, the current interface is not a multimedia playback interface. The present embodiment of the application does not limit the method for detecting whether the current interface is a multimedia playback interface.

[0264] If the current interface is a multimedia playback interface, the freeze detection module 501 may execute the following step S522 .

[0265] If the current interface is not a multimedia playback interface, the freeze detection module 501 may execute the following step S525.

[0266] In some embodiments, step S521 may be performed before step S515.

[0267] S522: The freeze detection module 501 identifies whether operation 1 is a preset operation.

[0268] The preset operation may refer to an operation for playing multimedia content. For example, the preset operation may include a swipe up operation.

[0269] It is understood that if Operation 1 is a preset operation for playing multimedia content (such as audio), the multimedia application 505 needs to respond to Operation 1 to play the multimedia content, and the jam detection module 501 can wait for the start event to determine whether the multimedia content is jammed. If Operation 1 is not a preset operation, the probability of the multimedia application 505 reporting the start event after Operation 1 is low. To avoid false detection of multimedia content jamming, the jam detection module 501 does not need to wait for the start event after receiving the user operation event corresponding to Operation 1, and can continue to wait for new user operation events.

[0270] In some embodiments, different multimedia applications may correspond to different preset operations. It is understandable that the operations used to trigger multimedia content playback in different multimedia applications may be different. The electronic device 100 can store preset operations corresponding to different multimedia applications. The jam detection module 501 can determine the preset operation based on the application to which the activity of the current interface belongs, and then identify whether operation 1 is a preset operation. In this way, multimedia content playback jam detection can be performed more specifically on each multimedia application when playing multimedia content, thereby improving the accuracy of multimedia content playback jam detection.

[0271] In some embodiments, the jam detection module 501 can also identify whether operation 1 is a preset operation when the multimedia content is in a preset playback state and acts on a preset area of ​​the screen. It is understandable that an operation may have different purposes when the multimedia content is in different playback states and / or the action location is different. The jam detection module 501 can more accurately identify the user operation used to trigger the playback of the multimedia content by combining the playback state of the multimedia content, the type of user operation and the action location, so that it can detect whether the multimedia content to be played is jammed in more scenarios where the multimedia content playback is triggered by user operations.

[0272] If operation 1 is the default operation, the jam detection module 501 may execute the following step S523 .

[0273] If operation 1 is not a preset operation, the jam detection module 501 may execute the following step S525 .

[0274] S523: The freeze detection module 501 detects whether a start event is received within a preset time period starting from the start of operation 1.

[0275] The jam detection module 501 can determine the action time of operation 1 based on the user operation event corresponding to operation 1. The action time of operation 1 can be the moment when operation 1 ends (for example, the moment when the finger is released on the screen in operation 1). Alternatively, operation 1 continues to act for a period of time. For example, operations such as swiping up and long pressing will continue to act on the screen for a period of time. The jam detection module 501 can determine a moment in the process of operation 1 as the action time of operation 1. This application does not limit the method for determining the action time of operation 1.

[0276] In some embodiments, the jam detection module 501 can set a timer according to the action time of operation 1, and determine whether a start event is received before the timer times out. The timing duration of the timer can be the duration of the above-mentioned preset time period, for example, 1s, 2s, etc. The embodiment of the present application does not limit the value of the timing duration. Among them, the timer can start timing from the action time of operation 1. If the jam detection module 501 receives a start event before the timer times out, it can indicate that the multimedia content playback is not jammed. If the jam detection module 501 does not receive a start event before the timer times out, it can indicate that the multimedia content playback is jammed.

[0277] It should be noted that the sequence numbering of steps S521-S523 and steps S517-S520 does not limit their execution order. For example, steps S521-S523 and steps S517-S520 can be performed simultaneously. After receiving the user operation event corresponding to Operation 1, if the multimedia application 505 can quickly obtain the playback data of audio a1, the multimedia application 505 can quickly report the start event. In this way, the jam detection module 501 can receive the start event before the timer expires. After receiving the user operation event corresponding to Operation 1, if the multimedia application 505 needs to wait a long time to obtain the playback data of audio a1 or fails to obtain the playback data of audio a1, the multimedia application 505 reports the start event later. In this way, the jam detection module 501 will not receive the start event before the timer expires. The jam detection module 501 can accurately identify whether the multimedia content is jammed by combining the above user operation events and the start event.

[0278] If the jam detection module 501 receives a start event within a preset time period starting from the operation 1 action time, the jam detection module 501 may execute the following step S525 .

[0279] If the jam detection module 501 does not receive a start event within a preset time period starting from the operation 1 action time, the jam detection module 501 may execute the following step S524.

[0280] S524 , the freeze detection module 501 instructs the network management module 504 to adjust the network through which the electronic device 100 communicates.

[0281] The network management module 504 can adjust the network used by the electronic device 100 for data communication with other devices based on the instructions of the jam detection module 501, so that the electronic device 100 can use a higher-quality network for communication. In this way, the multimedia application 505 can obtain the playback data of audio a1 as quickly as possible, thereby eliminating jams and playing audio a1.

[0282] S525: The freeze detection module 501 continues to wait for user operation events and audio events to be reported.

[0283] If it is uncertain whether Operation 1 triggers multimedia content playback, the jam detection module 501 will not determine that the multimedia content is jammed if it does not receive a start event within a preset period of time after Operation 1 begins. Instead, it will continue to wait for subsequent user operation events and audio events. This can reduce the possibility of false detection of multimedia content jams and improve the accuracy of multimedia content jam detection.

[0284] If Operation 1 is determined to be an operation that triggers multimedia content playback, the jam detection module 501 receives a start event within a preset time period starting from Operation 1, and can determine that the multimedia content playback is not jammed, and then continue to wait for subsequent user operation events and audio events. The jam detection module 501 can then detect the subsequently received user operation events and audio events according to steps S521 to S525 to identify the situation of multimedia content playback jamming.

[0285] As can be seen from the method shown in FIG. 5 , the jam detection module 501 can obtain a user operation event corresponding to a user operation on the electronic device 100 and determine whether the user operation is a preset operation for triggering multimedia content playback. When it is detected that the user operation is a preset operation for triggering multimedia content playback, the jam detection module 501 can detect whether a start event is received within a preset time period after the preset operation is performed, thereby accurately detecting whether the multimedia content to be played is jammed in a scenario where multimedia content playback is triggered by a preset operation.

[0286] In some embodiments, the electronic device 100 can detect multimedia content playback jams based on user operation events and video events. The jam detection module 501 can register video event monitoring and reporting with the video module in the electronic device 100. When the video playback state changes, the multimedia application 505 can report the corresponding video event to the video module. The video module can then send the video event to the jam detection module 501.

[0287] For example, the human-computer interaction module 502 can send a user operation event corresponding to an operation for playing a video on the multimedia playback interface of the multimedia application 505 to the jam detection module 501. After the jam detection module 501 determines from the user operation event that the operation is a preset operation for playing a video on the multimedia playback interface, it can detect whether a video start playback event is received within a preset time period starting from the time the operation was applied. For details of the above process, please refer to steps S521 to S523 shown in Figure 5.

[0288] In response to the operation for playing the video on the multimedia playback interface in the multimedia application 505, the multimedia application 505 can report the start playback event of the video to be played to the video module after obtaining the playback data of the video to be played. The video module can send the start playback event to the jam detection module 501. Among them, if the jam detection module 501 receives the start playback event of the above-mentioned video to be played within a preset time period starting from the time of the above-mentioned operation for playing the video, the multimedia content (i.e., the video to be played) will not be jammed. Otherwise, the jam detection module 501 determines that the multimedia content playback is jammed, and can instruct the network management module 504 to adjust the network for the electronic device 100 to communicate.

[0289] FIG6 exemplarily shows a flow chart of another method for detecting jamming in multimedia content playback.

[0290] As shown in FIG. 6 , the method may include steps S611 to S615 .

[0291] S611 , the freeze detection module 501 records the start event of the audio a1 .

[0292] As can be seen from the method shown in Figure 5, when the electronic device 100 is triggered to play audio a1, the audio module 503 can forward the start event reported by the multimedia application 505 to the jam detection module 501. The jam detection module 501 can record the start event of audio a1. In particular, if the jam detection module 501 records the start event, the jam detection module 501 not receiving the pause event or stop event of audio a1 can indicate that audio a1 is in the playing state.

[0293] S612 : During the playback of audio a1 , the multimedia application 505 completes the playback of the acquired playback data of audio a1 , and fails to acquire subsequent playback data of audio a1 within a preset time period.

[0294] If the playback data of audio a1 is obtained by the electronic device 100 from another device via a network, the electronic device 100 can simultaneously play audio a1 while obtaining subsequent playback data of audio a1 from the other device. The electronic device 100 can also cache a portion of the subsequent playback data of audio a1, so that playback can continue for a period of time even if the network suddenly deteriorates.

[0295] In some embodiments, when the network is poor for a long time, after the playback data of audio a1 cached in the memory is played, if the electronic device 100 still does not obtain the subsequent playback data of audio a1 from other devices, the multimedia application 505 will have no playback data available. Audio a1 is stuck.

[0296] S613: The audio module 503 fails to obtain subsequent playback data of the audio a1 within a preset time period, and generates a timeout event for the audio a1.

[0297] As can be understood, the audio module 503 can obtain the playback data of audio a1 from the multimedia application 505. As can be seen from the above step S612, the multimedia application 505 fails to obtain the subsequent playback data of audio a1 after the preset duration. Therefore, the audio module 503 also fails to obtain the subsequent playback data of audio a1 after the preset duration, and thus continues to play audio a1. If the subsequent playback data of audio a1 is not obtained after the preset duration, the audio module 503 can generate a timeout event for audio a1.

[0298] The timeout event of audio a1 may indicate that audio a1 is stuck during playback.

[0299] S614 , the audio module 503 sends a timeout event of the audio a1 to the freeze detection module 501 .

[0300] S615 . The freeze detection module 501 instructs the network management module 504 to adjust the network for communication of the electronic device 100 according to the timeout event.

[0301] The network management module 504 can adjust the network used by the electronic device 100 for data communication with other devices based on the instructions of the jam detection module 501, so that the electronic device 100 can use a higher-quality network for communication. In this way, the multimedia application 505 can obtain the playback data of audio a1 as quickly as possible, thereby eliminating jams and playing audio a1.

[0302] As shown in the method shown in FIG6 , a timeout event can indicate that audio playback has experienced pauses. Based on the timeout event, electronic device 100 can accurately detect whether audio playback in any multimedia application has experienced pauses. When audio has experienced pauses, electronic device 100 can adjust the network quality to eliminate them, using a higher-quality network. This can improve the user's audio listening experience.

[0303] The method for the electronic device 100 to detect whether a video is stuck during playback may refer to the method for the electronic device 100 to detect whether an audio is stuck during playback shown in FIG. 6 .

[0304] In some embodiments, the electronic device 100 can also detect whether the multimedia content is stuck in combination with user operations on the electronic device 100 and data transmission indicators. The above-mentioned data transmission indicators can be used to evaluate network quality and are indicators that reflect network quality. For example, data transmission indicators may include round trip time (RTT). RTT may refer to the time it takes for a network request to go from the starting point to the destination and then back to the starting point. RTT can be used to diagnose the speed and reliability of the network connection. The larger the RTT, the worse the quality of the current network. The embodiments of the present application do not limit the specific content of the data transmission indicators. For example, in addition to RTT, data transmission indicators may also include indicators for evaluating network quality such as data flow of data transmission and packet interval size. RTT is specifically used as an example for explanation here.

[0305] When a user operation is received on the audio and video playback interface to trigger the playback of multimedia content, the electronic device 100 can detect the RTT within a preset time period after the user operation. If the RTT within the preset time period indicates that the network quality is worse than the preset network quality, the electronic device 100 can determine that the multimedia content is stuck and then adjust the network to use a higher quality network for communication.

[0306] Wherein, if the RTT within the above-preset time period is greater than the preset RTT threshold, it may indicate that the network quality is worse than the preset network quality. Alternatively, if the average RTT within the above-preset time period is greater than the preset RTT threshold, it may indicate that the network quality is worse than the preset network quality. This application does not limit the method by which the electronic device 100 determines whether the RTT within the above-preset time period indicates that the network quality is worse than the preset network quality.

[0307] This application does not limit the above-mentioned user operation for triggering the playback of multimedia content.

[0308] In some embodiments, the electronic device 100 can detect multimedia content freezes by combining user operations on the electronic device 100, multimedia playback events, and data transmission indicators. The data transmission indicators can improve the confidence level of the media content freeze detection results. Furthermore, the data transmission indicators can also be used to determine whether multimedia content freezes are caused by network quality.

[0309] Specifically, when receiving a user operation on the audio and video playback interface for triggering the playback of multimedia content, the electronic device 100 may obtain a start event reported by the multimedia application that plays the multimedia content after the user operation.

[0310] If a start event is received, the electronic device 100 can determine that the multimedia content is not stuck. When the multimedia content is not stuck based on the user operation and the multimedia playback event, if the RTT is less than the preset RTT threshold, it can be said that the current network quality is good, thereby increasing the confidence of the above-mentioned detection result of the multimedia content not being stuck.

[0311] If no start event is obtained, the electronic device 100 may determine that the multimedia content is stuck. When determining that the multimedia content is stuck based on user operations and multimedia playback events, if the RTT is greater than a preset RTT threshold, it can indicate that the current network quality is poor, thereby increasing the confidence level of the above-mentioned multimedia content stuck detection result.

[0312] When multimedia content freezes based on user operations and multimedia playback events, if the RTT is less than the preset RTT threshold, it can be said that the current network quality is good and the multimedia content freeze is not caused by poor network quality. It is understandable that multimedia content freezes may be caused by other reasons in addition to poor network quality. For example, the load of the electronic device 100 processor is too high, and there are too many background processes running in the background, which may also cause multimedia content freezes. Therefore, when the electronic device 100 determines that multimedia freezes based on user operations and multimedia playback events, and the RTT is less than the preset RTT threshold, the electronic device 100 can shut down one or more background processes to reduce the load on the processor without adjusting the network for the electronic device 100 to communicate.

[0313] Not limited to detecting whether the playback of multimedia content is stuck, the electronic device 100 can also detect whether the interface browsing is stuck in combination with user operations on the electronic device 100 and data transmission indicators. In some embodiments, in addition to being used to play audio and / or video and display an audio and video playback interface, the multimedia application can also display a graphic interface. The graphic interface may refer to a user interface that displays text and / or pictures. A graphic interface can be a long page. The electronic device 100 can slide the graphic interface up or down in response to a preset user operation (such as an upward sliding operation, a downward sliding operation) to display the content of different areas of the graphic interface on the screen. The electronic device 100 can also jump from one graphic interface to another graphic interface in response to a preset operation (such as a sliding operation to the left, a sliding operation to the right). Not limited to multimedia applications, the electronic device 100 may also include other APPs that can display graphic interfaces, such as browsers, news APPs, shopping APPs, reading APPs, etc.

[0314] The electronic device 100 can detect whether there is any freeze when the user browses the graphic interface of these APPs.

[0315] Specifically, upon receiving a user operation on the graphic interface for switching displayed content, the electronic device 100 may detect the RTT within a preset time period after the user operation. If the RTT within the preset time period indicates that the network quality is worse than the preset network quality, the electronic device 100 may determine that the graphic interface browsing is experiencing lag, and then adjust the network to use a higher quality network for communication.

[0316] The user operations for adjusting the displayed content may include, but are not limited to, user operations for switching the content of different areas of the graphic interface, user operations for jumping to other graphic interfaces, and the like. For example, user operations for switching the content of different areas of the graphic interface may include swiping up or swiping down. User operations for jumping to other graphic interfaces may include swiping left or swiping right.

[0317] It can be seen from the above embodiment that when a user operation acting on an interface is received, the electronic device 100 can determine whether this interface is a preset interface, and determine whether the user operation is a preset user operation. If it is a preset user operation acting on a preset interface, the electronic device 100 can detect the RTT within a preset time period after the user operation is performed. If the RTT within the preset time period indicates that the network quality is worse than the preset network quality, the electronic device 100 can determine that there is a freeze in browsing this interface, and then adjust the network to use a network with better quality for communication. In this way, the electronic device 100 can obtain the data required for the display interface as soon as possible through a network with better quality, thereby eliminating the freeze in browsing the interface as soon as possible and improving the user's experience of browsing the interface.

[0318] In some embodiments, the electronic device 100 may store a correspondence table between preset interfaces and preset user operations to detect whether a received user operation is a preset user operation acting on the preset interface. Optionally, the electronic device 100 may also obtain the correspondence table between the preset interfaces and preset user operations from other devices (e.g., a cloud server, etc.).

[0319] It is understood that the various user interfaces described in the embodiments of this application are merely exemplary interfaces and do not limit the scope of this application. In other embodiments, the user interface may adopt a different interface layout, include more or fewer controls, and add or remove other functional options. As long as they are based on the same inventive concept provided by this application, they are all within the scope of protection of this application.

[0320] It should be noted that, without causing any contradiction or conflict, any feature in any embodiment of the present application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of the present application.

[0321] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting jamming of multimedia content playback, characterized in that: The method is applied to an electronic device, the electronic device includes a first application, the first application is an application for playing multimedia content, and the method includes: The electronic device receives a first operation on a first interface, where the first interface is an interface in the first application for controlling the playback of multimedia content, and the first operation is for playing the first multimedia content; When the electronic device does not obtain the first start playing event reported by the first application after the first operation, the electronic device adjusts the network for data communication; wherein the first start playing event is used to indicate the start of playing of the first multimedia content.

2. The method according to claim 1, characterized in that The first operation matches a preset operation.

3. The method according to claim 1 or 2, characterized in that: The network for data communication is adjusted from the first network to the second network, and the method further includes: The first application in the electronic device obtains the playback data of the first media content by using the second network; The electronic device plays the first multimedia content according to the playback data of the first multimedia content.

4. The method according to claim 3, characterized in that After the electronic device adjusts the network for data communication, the method further includes: The electronic device changes a first identifier on the first interface to a second identifier, wherein the first identifier is used to indicate that a network for data communication is the first network, and the second identifier is used to indicate that a network for data communication is the second network.

5. The method according to claim 3 or 4, characterized in that: The electronic device plays the first multimedia content according to the playback data of the first multimedia content, specifically including: The electronic device plays the first audio and / or plays the first video according to the playback data of the first multimedia content.

6. The method according to any one of claims 3 to 5, characterized in that: The method further comprises: During the process of playing the first multimedia content, the electronic device encounters a freeze in playing the first multimedia content; The electronic device detects a first data exhaustion event and adjusts a network for data communication according to the first data exhaustion event; wherein the first data exhaustion event is used to indicate that the playback data used to play the first multimedia content in the electronic device is exhausted and no playback data is available.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: When the electronic device obtains the first start playing event reported by the first application after the first operation, the electronic device plays the first multimedia content; The playback data of the first multimedia content is obtained by the first application from a memory, or is obtained by the first application using a network for data communication.

8. The method according to any one of claims 2 to 7, characterized in that: The method further comprises: The electronic device receives a second operation on the first interface, and the second operation does not match the preset operation; or The electronic device receives a third operation on a second interface, where the second interface is not an interface for controlling playback of multimedia content; The electronic device waits for receiving a subsequent user operation.

9. The method according to any one of claims 1 to 8, characterized in that The first multimedia content is paused, and the method further includes: The electronic device receives a fourth operation on the first interface, where the fourth operation is used to play the first multimedia content; The electronic device recognizes that the fourth operation acts on the first area of ​​the first interface, and the fourth operation matches the preset operation, and the electronic device determines whether the first start playback event reported by the first application is obtained within a second time period; wherein the second time period is a preset time period starting from a second moment, and the second moment is the time when the second operation acts; When the electronic device does not obtain the first start playing event reported by the first application within the second time period, the electronic device adjusts the network for data communication; When the electronic device obtains the first start playing event reported by the first application within the second time period, the electronic device plays the first multimedia content.

10. The method according to any one of claims 1, characterized in that Before the electronic device adjusts the network for data communication, the method further includes: The electronic device determines, according to the activity information of the first interface, that the first interface is an interface for controlling the playback of multimedia content.

11. The method according to claim 1, characterized in that: Before the electronic device adjusts the network for data communication, the method further includes: The electronic device detects that a data transmission indicator after the first operation indicates that network quality is worse than a preset network quality.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: The electronic device receives a fifth operation on a second interface, where the second interface is an interface in the first application for controlling the playback of multimedia content, and the fifth operation is for playing second multimedia content; When the electronic device does not obtain the second start playback event reported by the first application after the fifth operation, and the data transmission indicator after the fifth operation indicates that the network quality is better than the preset network quality, the electronic device closes one or more processes, and the one or more processes are not processes of the first application, and the second start playback event is used to indicate the start of playback of the second multimedia content.

13. The method according to claim 11 or 12, characterized in that: The data transmission indicator is the round trip time RTT.

14. The method according to any one of claims 1 to 13, characterized in that When the electronic device fails to obtain the first start playing event reported by the first application after the first operation, the electronic device adjusts the network for data communication, specifically including: When the electronic device does not obtain the first start playing event within a first time period, the electronic device adjusts the network for data communication, the first time period is a preset time period starting from a first moment, and the first moment is the time when the first operation takes effect.

15. A method for detecting interface browsing freeze, characterized in that: The method is applied to an electronic device, the electronic device includes a first application, and the method includes: The electronic device receives a first operation on a first interface in the first application, where the first operation is used to switch display content on the screen of the electronic device within the first application; The electronic device acquires a data transmission indicator after the first operation, where the data transmission indicator is used to indicate a network quality of data communication performed by the electronic device; If the data transmission indicator after the first operation indicates that the network quality is worse than the preset network quality, the electronic device adjusts the network for data communication.

16. The method according to claim 15, characterized in that The first interface matches a preset interface, and the first operation matches a preset operation.

17. The method according to claim 15 or 16, characterized in that The first interface is a long page, and the first operation is used to switch the display content on the screen of the electronic device to the content of a different area on the first interface, or the first operation is used to switch the first interface to a second interface in the first application.

18. The method according to any one of claims 15 to 17, characterized in that: The data transmission indicator after the first operation includes the data transmission indicator within a first time period, the first time period is a preset time period starting from a first moment, and the first moment is the time when the first operation takes effect.

19. The method according to any one of claims 18, characterized in that The data transmission indicator is a round trip time RTT, and the data transmission indicator after the first operation indicates that the network quality is worse than the preset network quality, specifically including: The RTTs in the first time period are all greater than a preset RTT threshold, or an average value of the RTTs in the first time period is greater than the preset RTT threshold.

20. The method according to any one of claims 15 to 19, characterized in that The network for data communication is adjusted from the first network to the second network, and after the electronic device adjusts the network for data communication, the method further includes: The electronic device changes a first identifier on the first interface to a second identifier, wherein the first identifier is used to indicate that a network for data communication is the first network, and the second identifier is used to indicate that a network for data communication is the second network.

21. An electronic device, characterized in that: The electronic device comprises a memory and a processor; wherein the memory is used to store a computer program, and the processor is used to call the computer program, so that the electronic device executes the method according to any one of claims 1 to 14 or claims 15 to 20.

22. A computer-readable storage medium storing instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 14 or claims 15 to 20.

23. A computer program product, characterized in that The computer program product includes computer instructions. When the computer instructions are run on an electronic device, the electronic device executes the method according to any one of claims 1 to 14 or claims 12 to 20.

Images