Communication message processing method and display device

Abstract

The present disclosure relates to a communication message processing method and a display device, comprising: in the process of the first device casting screen to the second device, listening to the communication message of the first device; in the communication message is listened to, the state information of the follow-up video stream displayed by the second device is acquired, the state information comprises a follow-up state and a pause state, and the communication message represents the communication message sent by the third device to the first device; according to the state information of the follow-up video stream displayed by the second device, a control instruction is sent to the second device to control the second device to display notification content, different control instructions correspond to different notification content displayed by the second device, avoiding that when the user follows up the training video stream played by the second device, the notification content received by the first device is directly sent to the second device, the training video stream played by the second device is paused, and the follow-up video stream, so that the user in the follow-up state cannot make a timely response, and the use experience of the user is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of display technology, and in particular to a communication message processing method and a display device. Background Technology

[0002] With the development of terminal and internet technologies, multi-screen collaboration technology has been widely applied. Multi-screen collaboration technology refers to the collaborative work of multiple screens, including projecting the screen content of one device onto other devices for display. For example, projecting the screen content of a mobile phone onto a computer allows the user to view the content displayed on the mobile phone screen on the computer, thus achieving screen mirroring between the mobile phone and the computer.

[0003] In existing technology, during the process of screen mirroring from the first device to the second device, the second device displays the screen content of the first device. However, after the first device receives a communication message, the content displayed on the second device will pause to receive the communication message. However, in some usage scenarios, such as fitness scenarios, users will follow along with a training video played on the second device. The first device captures the user's training video and mirrors it to the second device, which then displays both the training video and the follow-up video. When the first device receives a communication message, it interrupts the playback of both the training video and the follow-up video on the second device, thus degrading the user experience. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides a communication message processing method and a display device.

[0005] In a first aspect, embodiments of this disclosure provide a communication message processing method, including:

[0006] During the process of the first device projecting its screen to the second device, the communication messages of the first device are monitored.

[0007] Upon receiving the communication message, the status information of the follow-up video stream displayed by the second device is obtained. The status information includes the follow-up status and the pause status. The communication message represents a communication message sent by the third device to the first device.

[0008] Based on the status information of the follow-up video stream displayed by the second device, a control command is sent to the second device to control the second device to display notification content. Different control commands correspond to different notification content displayed by the second device.

[0009] As one possible implementation, the notification content may include a first notification content and a second notification content, wherein the first notification content includes text and the second notification content includes text and audio.

[0010] The step of sending control commands to the second device based on the status information of the follow-up video stream displayed by the second device includes:

[0011] When the status information of the follow-up video stream displayed by the second device is follow-up status, a first control command is sent to the second device to control the second device to display the text corresponding to the first notification content;

[0012] When the status information of the follow-up video stream displayed by the second device is in a paused state, a second control command is sent to the second device to control the second device to display the text corresponding to the second notification content and to control the speaker to play the audio corresponding to the second notification content.

[0013] As one possible implementation, the method may further include:

[0014] When the status information of the follow-up video stream displayed by the second device is "follow-up status", in response to the pause follow-up instruction signal received by the second device, the display is controlled to show the text corresponding to the notification content, and the speaker is controlled to play the audio corresponding to the notification content.

[0015] As one possible implementation, the method may further include:

[0016] When the status information of the follow-up video stream displayed by the second device is in follow-up status, in response to the trigger operation received by the second device, a first instruction is sent to the first device to instruct the first device to open the application corresponding to the notification content; or, the display is controlled to pause the playback of the training video stream and the follow-up video stream, and the speaker is controlled to play the audio corresponding to the notification content.

[0017] As one possible implementation, after sending the first instruction to the first device, the method further includes:

[0018] Monitor the communication status of audio and video messages corresponding to the communication messages of the first device;

[0019] When the communication status of the audio and video corresponding to the communication message of the first device changes from the communication in progress state to the communication ended state, a screen projection communication channel is established between the first device and the second device.

[0020] As one possible implementation, the method may further include:

[0021] When the second device includes the application corresponding to the notification content, the notification content is opened through the application on the second device.

[0022] Monitor the communication status of the audio and video messages corresponding to the communication messages;

[0023] When the communication status of the audio and video corresponding to the communication message of the second device changes from the communication in progress state to the communication ended state, the display is controlled to continue playing the training video stream and the follow-up video stream.

[0024] As one possible implementation, the step of obtaining the status information of the follow-up video stream displayed by the second device includes:

[0025] When the duration of the communication message received by the first device is detected to be greater than or equal to a preset threshold, the status information of the follow-up video stream displayed by the second device is obtained.

[0026] In a second aspect, embodiments of this disclosure provide a display device, the display device comprising:

[0027] monitor;

[0028] The controller is configured as follows:

[0029] During the process of the first device projecting its screen to the second device, the communication messages of the first device are monitored.

[0030] Upon receiving the communication message, the status information of the follow-up video stream displayed by the second device is obtained. The status information includes the follow-up status and the pause status. The communication message represents a communication message sent by the third device to the first device.

[0031] Based on the status information of the follow-up video stream displayed by the second device, a control command is sent to the second device to control the second device to display notification content. Different control commands correspond to different notification content displayed on the display.

[0032] As one possible implementation, the controller is optionally configured as follows:

[0033] When the status information of the follow-up video stream displayed by the second device is follow-up status, a first control command is sent to the second device to control the second device to display the text corresponding to the first notification content;

[0034] When the status information of the follow-up video stream displayed by the second device is in a paused state, a second control command is sent to the second device to control the second device to display the text corresponding to the second notification content and to control the speaker to play the audio corresponding to the second notification content.

[0035] As one possible implementation method, optional,

[0036] The controller is also configured to:

[0037] When the status information of the follow-up video stream displayed by the second device is "follow-up status", in response to the pause follow-up instruction signal received by the second device, the display is controlled to show the text corresponding to the notification content, and the speaker is controlled to play the audio corresponding to the notification content.

[0038] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0039] The communication message processing method and display device provided in this disclosure, during the process of screen projection from the first device to the second device, monitor the communication messages of the first device. Upon monitoring a communication message, by obtaining the status information of the follow-up video stream displayed on the second device, it determines whether the user is following along with the training video stream played by the second device. Then, based on the obtained status information of the follow-up video stream displayed on the second device, a control command is sent to the second device to control the second device to display notification content. That is, by obtaining the status information of the follow-up video stream displayed on the second device, it determines whether the user is following along with the training video stream played by the second device, and then controls the second device to display notification content based on whether the user is following along with the training video stream played by the second device. This avoids directly sending the notification content received by the first device to the second device while the user is following along with the training video stream played by the second device, pausing the training video stream and the follow-up video stream played by the second device, which would prevent the user in the follow-up state from reacting in a timely manner and reduce the user experience. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0041] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1A This is a system network architecture diagram of a communication message processing method provided in an embodiment of this disclosure;

[0043] Figure 1B This is a schematic diagram of the structure of a terminal device provided in an embodiment of this disclosure;

[0044] Figure 2A This is a flowchart illustrating a communication message processing method provided in an embodiment of this disclosure;

[0045] Figure 2B This is a schematic diagram of the signal flow of a communication message processing method provided in an embodiment of the present disclosure;

[0046] Figure 3A This is a flowchart illustrating another communication message processing method provided in an embodiment of this disclosure;

[0047] Figure 3B This is a schematic diagram of the signal flow of another communication message processing method provided in this embodiment of the present disclosure;

[0048] Figure 4A This is a flowchart illustrating another communication message processing method provided in this embodiment of the present disclosure;

[0049] Figure 4B This is a schematic diagram of the signal flow of another communication message processing method provided in this embodiment of the present disclosure;

[0050] Figure 5A This is a flowchart illustrating another communication message processing method provided in this embodiment of the present disclosure;

[0051] Figure 5B This is a schematic diagram of the signal flow of another communication message processing method provided in this embodiment of the present disclosure;

[0052] Figure 6A This is a flowchart illustrating another communication message processing method provided in this embodiment of the present disclosure;

[0053] Figure 6B This is a schematic diagram of the signal flow of another communication message processing method provided in this embodiment of the present disclosure;

[0054] Figure 7A This is a flowchart illustrating another communication message processing method provided in this embodiment of the present disclosure;

[0055] Figure 7B This is a schematic diagram of the signal flow of another communication message processing method provided in this embodiment of the present disclosure;

[0056] Figure 8A This is a flowchart illustrating another communication message processing method provided in this embodiment of the present disclosure;

[0057] Figure 8B This is a schematic diagram of the signal flow of another communication message processing method provided in this embodiment of the present disclosure;

[0058] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0059] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0060] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0061] Figure 1A This diagram schematically illustrates a screen projection system network architecture diagram according to an embodiment of this application. The screen projection system network architecture diagram includes a first device 10, a second device 20, a router 30, and a server 40, etc. Specifically: The first device can serve as the sending end in the screen projection system network architecture, used to send multimedia content and various instructions to the second device. In exemplary embodiments, the first device includes, but is not limited to, smartphones, tablets, game consoles, laptops, etc. The second device can serve as the display end in the screen projection system network architecture, used to receive multimedia content and various instructions from the first device, and synchronously play the received multimedia content on its own display panel or the display panel of an associated device. The second device can be a relatively large-screen device such as a virtual reality device, set-top box, smart TV, vehicle terminal, or large outdoor display panel; of course, it can also be a relatively small-screen device such as a smartphone, tablet, or laptop.

[0062] A router can act as a gateway in the network architecture of a screen mirroring system, used to build a local area network (LAN) and provide internet services. A router can provide local connectivity between the first and second devices.

[0063] A server can serve as a cloud device in the network architecture of a screen mirroring system, providing cloud services such as cloud account management. Servers can be rack servers, blade servers, tower servers, or cabinet servers (including standalone servers or server clusters composed of multiple servers).

[0064] The first device can discover the second device through a first network or a second network, and establish a network connection with the second device based on the first network or the second network. Through this established network connection, the first device can project multimedia content to be displayed on the second device, which will then play it. The first network can be a local area network (LAN), which can be a wired LAN or a wireless LAN. Figure 1AAs shown, a wireless local area network (WLAN) can be a Wi-Fi network built by router 8. The second network can be a wide area network (WAN), such as the Internet. A WAN can include physical links, such as coaxial cable links, twisted-pair cable links, fiber optic links, and combinations thereof. A WAN can also include wireless links, such as cellular links and satellite links.

[0065] In an exemplary embodiment, such as Figure 1B As shown, the first device and the second device include at least one of the following: a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0066] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0067] The display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0068] The display 260 can be an LCD display, an OLED display, or a projection display, and can also be a projection device and a projection screen.

[0069] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0070] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote control).

[0071] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0072] The external device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0073] The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0074] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0075] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the monitor 260, the controller 250 can execute operations related to the object selected by the user command.

[0076] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0077] Users can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, users can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0078] A "user interface" is the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form that the user can accept. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0079] The above provides a brief introduction to the functions of the first and second devices.

[0080] The first device can project its screen to the second device.

[0081] However, in existing technologies, during the screen mirroring process from the first device to the second device, the second device displays the screen content of the first device. After the first device receives a communication message, the content displayed on the second device pauses to receive the communication message. However, in some usage scenarios, such as fitness scenarios, where users follow along with training videos played on the second device, the first device captures the user's workout video and mirrors it to the second device, which then displays both the workout video and the follow-up video. When the first device receives a communication message, it interrupts the playback of both the workout video and the follow-up video on the second device, degrading the user experience.

[0082] This disclosure provides a communication message processing method. During the process of a first device projecting its screen to a second device, the method listens for communication messages from the first device. Upon receiving a communication message, it obtains the status information of the follow-up video stream displayed on the second device, including a follow-up status and a pause status. Based on the status information of the follow-up video stream displayed on the second device, it sends a control command to the second device to control the second device to display notification content. Different control commands correspond to different notification content displayed on the display, ensuring that when the status information of the follow-up video stream displayed on the second device is in the follow-up status, the communication messages sent by the third device received by the first device will not affect the follow-up video stream and training video stream displayed on the second device, thereby improving the user experience.

[0083] Figure 2A This is a flowchart illustrating a communication message processing method provided in an embodiment of this disclosure. Figure 2B This is a schematic diagram of the signal flow of a communication message processing method provided in an embodiment of this disclosure, combined with... Figure 2A and Figure 2B Communication message processing methods include:

[0084] S100: During the process of the first device projecting its screen to the second device, the communication messages of the first device are monitored.

[0085] In a specific implementation, the first device includes an image acquisition module. For example, the image acquisition module can be a camera, and the user video data is acquired based on the image acquisition module of the first device.

[0086] In one specific implementation, the second device plays a training video stream, such as a fitness training video stream or a dance training video stream. The user follows along with the training video stream played by the second device, and the image acquisition module of the first device captures the user's follow-along video stream and displays it on the second device via screen projection.

[0087] Specifically, the distance between the first device and the second device is set to be less than a preset distance threshold, and the angle between the orientation of the image acquisition module of the first device and the orientation of the display screen of the second device is less than a preset angle threshold.

[0088] In this embodiment, when a user performs continuous physical movements, such as dance or gymnastic movements, by following a training video stream played on a second device, the image acquisition module of the first device captures the user's training video stream. Understandably, to keep the image acquisition module fixed and prevent shaking, the first device can be mounted on a fixed bracket, meaning the distance between it and the second device is less than a preset distance threshold, and the angle between the orientation of the first device's image acquisition module and the orientation of the second device's display screen is less than a preset angle threshold. This ensures that while the user follows the training video stream played on the second device, the image acquisition module of the first device can also capture the user's training video stream.

[0089] As an example, during the process of projecting screens from the first device to the second device, the second device can display the training video stream in the display window and the projected content (i.e., the follow-up video stream) from the first device in the projection window. For instance, the projection window is used to display the screen content of the first device, meaning that the content displayed in the projection window is the same as the content displayed on the screen of the first device.

[0090] It should be noted that screen casting typically includes push screen casting and screen mirroring. This disclosure is mainly applied to screen mirroring scenarios.

[0091] Push casting refers to projecting multimedia resources from one device onto another for playback. These multimedia resources include audio and video. For example, projecting a video from a mobile phone onto a smart TV for playback. This process does not affect the user's use of the phone; the user can still operate the phone. Mirror casting refers to projecting the screen content of one device onto another for display, essentially mirroring the entire screen of one device onto another. For example, projecting the screen content of a mobile phone onto a projector. In this process, whatever is displayed on the mobile phone will also be displayed in the projector's projection window.

[0092] In addition, during the process of the first device 10 projecting its screen to the second device 20, the second device 20 can also perform reverse control on the first device 10 based on the projection window in the second device 20.

[0093] In a specific implementation, during the process of the first device projecting its screen to the second device, in order to avoid the communication messages received by the first device affecting the training video stream and follow-up video stream played by the second device, the communication messages of the first device are first monitored during the process of the first device projecting its screen to the second device.

[0094] Communication refers to tools that enable communication, and the content of messages can be in the form of voice, video, telephone, text, etc.

[0095] The communication message indicates that the first device has received a communication message from a call request sent by the third device. The call request includes: a cellular network call request (requires a SIM card but not an application) or a non-cellular network call request (does not require a SIM card but requires an application). The non-cellular network call request can include: a carrier data network call request, an internet call request, or a local area network call request. The communication message type can be voice, video, telephone, text, etc. (there is an audio prompt message during the process of the third device sending the communication message to the first device).

[0096] S200: Upon receiving a communication message, obtain the status information of the follow-up video stream displayed on the second device.

[0097] Status information includes practice status and pause status.

[0098] The second device displays the follow-up video stream and the training video stream. The training video stream is the video stream played by the second device, and the follow-up video stream is the follow-up video data corresponding to the training video stream played by the first device, which is collected by the image acquisition module of the first device and then projected onto the second device for display.

[0099] When the server of the second device detects that the first device has received a communication message, the controller of the second device obtains the status information of the training video stream displayed in the projection window of the second device. Based on the status information of the training video stream displayed in the projection window of the second device, it can be determined whether the user is following along with the training video stream played by the second device. If the status information of the training video stream displayed by the second device is "following along", it can be determined that the user is following along with the training video stream played by the second device. If the status information of the training video stream displayed by the second device is "paused", it can be determined that the user is not following along with the training video stream played by the second device.

[0100] As a specific implementation method, the status information of the follow-up video stream can be determined by continuously acquiring image information of the follow-up video stream at preset frame values ​​and comparing the image information corresponding to adjacent frame values. For example, image information corresponding to three consecutive frames of the follow-up video stream is acquired, and the image information corresponding to the first frame is compared with the image information corresponding to the second frame, and the image information corresponding to the second frame is compared with the image information corresponding to the third frame. If the image information corresponding to the first frame is consistent with the image information corresponding to the second frame, and the image information corresponding to the second frame is consistent with the image information corresponding to the third frame, then the follow-up video stream can be determined to be in a paused state. If the image information corresponding to the first frame is inconsistent with the image information corresponding to the second frame, and / or the image information corresponding to the second frame is inconsistent with the image information corresponding to the third frame, then the follow-up video stream can be determined to be in a follow-up state.

[0101] S300: Based on the status information of the follow-up video stream displayed on the second device, send control commands to the second device to control the second device to display notification content.

[0102] The notification content displayed on the second device varies depending on the control command.

[0103] When the first device projects its screen to the second device and the first device receives a communication message, this embodiment of the present disclosure obtains the status information of the follow-up video stream displayed on the second device and sends a control command to the second device to control the second device to display notification content. This avoids the situation where the first device receives a communication message and then directly displays the notification content corresponding to the communication message on the second device during the screen projection process, which would interrupt the follow-up video stream and training video stream displayed on the second device and reduce the user experience.

[0104] The communication message processing method provided in this embodiment listens to the communication messages of the first device during the screen projection process from the first device to the second device. Upon receiving a communication message, the method obtains the status information of the follow-up video stream displayed on the second device to determine whether the user is following along with the training video stream played by the second device. Then, based on the obtained status information of the follow-up video stream displayed on the second device, the method sends a control command to the second device to control the second device to display notification content. That is, by obtaining the status information of the follow-up video stream displayed on the second device, the method determines whether the user is following along with the training video stream played by the second device, and then controls the second device to display notification content based on whether the user is following along with the training video stream played by the second device. This avoids directly sending the notification content received by the first device to the second device while the user is following along with the training video stream played by the second device, thus pausing the training video stream and the follow-up video stream played by the second device, which would prevent the user in the follow-up state from reacting in a timely manner and reduce the user experience.

[0105] Figure 3A This is a flowchart illustrating another communication message processing method provided in this embodiment of the present disclosure. Figure 3B This is a schematic diagram of the signal flow of another communication message processing method provided in this disclosure embodiment, combined with... Figure 3A and Figure 3B The implementation of step S300 includes:

[0106] S301. When the status information of the follow-up video stream displayed by the second device is follow-up status, a first control command is sent to the second device to control the second device to display the text corresponding to the first notification content.

[0107] In a specific implementation, when the controller of the second device detects that the first device has received a communication message and the status information of the follow-up video stream displayed by the second device is in the follow-up state, the controller of the second device sends a first control command to the second device to control the second device to display the text corresponding to the first notification content. That is, only the text corresponding to the notification content received by the first device is displayed, and the speaker does not play the audio corresponding to the notification content.

[0108] For example, if the first device opens a fitness application, the image acquisition module of the fitness application captures the user's follow-up video stream and projects it onto the second device for display. During the projection process from the first device to the second device, the fitness application on the first device listens for communication messages sent from the third device to the first device. For example, the first device receives a call from user A. At this time, the first device retrieves the caller information from the user's address book based on the caller information and calls the middleware of the first device to send it to the second device. After receiving the caller information sent by the middleware of the first device, the controller of the second device sends the text data corresponding to the caller information to the second device for projection display (the caller information is the notification content). That is, only the text of user A's call is displayed on the screen of the second device, and the audio of the call is not played. For example, the incoming call music set by the user on the first device is used to prevent the audio corresponding to the notification content of the first device from playing on the second device, so that the audio corresponding to the notification content interrupts the follow-up video stream and training video stream played on the second device.

[0109] S302. When the status information of the follow-up video stream displayed by the second device is in a paused state, a second control command is sent to the second device to control the second device to display the text corresponding to the second notification content and to control the speaker to play the audio corresponding to the second notification content.

[0110] In a specific implementation, when the controller of the second device detects that the first device has received a communication message and the status information of the follow-up video stream displayed by the second device is in a paused state, the controller of the second device sends a second control command to the second device, the text corresponding to the second notification content, and controls the speaker to play the audio corresponding to the second notification content. That is, not only is the text corresponding to the notification content received by the first device displayed, but the audio corresponding to the notification content is also played.

[0111] For example, if the first device opens a fitness application, the image acquisition module of the fitness application captures the user's workout video stream and projects it onto the second device for display. During the projection process from the first device to the second device, the fitness application on the first device listens for an incoming call from user A. At this time, the first device retrieves the caller information from the user's address book based on the caller information, calls the middleware of the first device and sends it to the second device. After receiving the caller information sent by the middleware of the first device, the server of the second device sends the text data corresponding to the caller information to the second device for projection display and plays the incoming call music set by the first device. This allows the user to display the text data corresponding to the communication message and play audio data through the second device when the workout is paused.

[0112] Figure 4A This is a flowchart illustrating another communication message processing method provided in this disclosure. Figure 4BThis is a schematic diagram of the signal flow of another communication message processing method provided in this disclosure embodiment. This disclosure embodiment is... Figure 3A Based on the corresponding embodiments, such as Figure 4A and Figure 4B As shown, the communication message processing method also includes:

[0113] S303. When the status information of the follow-up video stream displayed by the second device is follow-up status, in response to the pause follow-up instruction signal received by the second device, control the display to display the text corresponding to the notification content, and control the speaker to play the audio corresponding to the notification content.

[0114] In a specific implementation, when a communication message is detected and the status information of the follow-up video stream displayed on the second device is in follow-up mode, the display is controlled to show the text corresponding to the notification content to notify the user that a communication message has been received. This ensures that the user can receive communication messages normally without affecting the follow-up training video stream. The user can promptly receive the notification information based on the text corresponding to the notification content displayed on the display. At this time, the user can pause the training video stream displayed on the second device through voice control or by controlling the device. For example, if the user pauses the training video stream by voice control, the second device will respond to the user's voice control signal and pause the training video stream. At this time, the controller responds to the pause follow-up instruction received by the second device. That is, after receiving the signal from the second device to pause the training video stream, the controller controls the display to show the text corresponding to the notification content and controls the speaker to play the audio corresponding to the notification content. In other words, after receiving the user's instruction signal to pause the training video stream played by the second device, the controller controls the display to show the text corresponding to the notification content and controls the speaker to play the audio corresponding to the notification content, ensuring that the user can receive the notification content from a distance.

[0115] Figure 5A This is a flowchart illustrating another communication message processing method provided in this disclosure. Figure 5B This is a schematic diagram of the signal flow of another communication message processing method provided in this disclosure embodiment. This disclosure embodiment is... Figure 3A Based on the corresponding embodiments, such as Figure 5A and Figure 5B As shown, the communication message processing method also includes:

[0116] S304. When the status information of the follow-up video stream displayed by the second device is follow-up status, in response to the trigger operation received by the second device, a first instruction is sent to the first device to instruct the first device to open the application corresponding to the notification content.

[0117] Specifically, the triggering operation includes the opening operation, which can be based on the control device or on voice control.

[0118] When the status information of the follow-up video stream displayed on the second device is in the follow-up state, as one possible implementation, the user can trigger the notification content displayed on the second device through voice control or control device control. At this time, the controller responds to the user's trigger operation on the second device, sends a first instruction to the first device, and opens the application / function corresponding to the notification content on the first device.

[0119] For example, if the first device detects a communication message in application A, indicating that user A has received a video call from user B, and the status information of the follow-up video stream displayed on the second device is in follow-up status, the second device displays the text "User B's video call" corresponding to the notification content as a message prompt. At this time, if the user triggers the text "User B's video call" corresponding to the notification content displayed on the second device, the controller responds to the text corresponding to the notification content triggered by the user on the second device and sends a first instruction to the first device to instruct the first device to open application A.

[0120] S305: Control the display to pause the training video stream and follow-up video stream, and control the speaker to play the audio corresponding to the notification content.

[0121] When the status information of the follow-up video stream displayed on the second device is in follow-up status, as another possible implementation method, the user can trigger the text corresponding to the notification content displayed on the second device through voice control or control device control. At this time, the controller responds to the user's trigger operation on the second device, controls the display to pause the playback of the training video stream and the follow-up video stream, and controls the speaker to play the audio corresponding to the notification content.

[0122] For example, if the first device detects a communication message in application A, indicating that user A has received a video call from user B, and the status information of the follow-up video stream displayed on the second device is in follow-up status, the second device displays the text "User B's video call" corresponding to the notification content as a message prompt. At this time, if the user triggers the text "User B's video call" corresponding to the notification content displayed on the second device, the controller responds to the text corresponding to the notification content triggered by the user on the second device, controls the display to pause the playback of the training video stream and the follow-up video stream, and controls the speaker to play the audio corresponding to the notification content.

[0123] Figure 6A This is a flowchart illustrating another communication message processing method provided in this disclosure. Figure 6B This is a schematic diagram of the signal flow of another communication message processing method provided in this disclosure embodiment. This disclosure embodiment is... Figure 5ABased on the corresponding embodiments, when the communication message processing method includes step S304, the method further includes:

[0124] S306. Monitor the audio and video communication status corresponding to the communication messages of the first device.

[0125] S307. When the communication status of the audio and video corresponding to the communication message of the first device changes from the communication in progress state to the communication end state, a screen projection communication channel between the first device and the second device is established.

[0126] When the second device displays a follow-up video stream in a follow-up state, the user can trigger the notification text displayed on the second device. This triggers the controller to send a first instruction to the first device, causing the first device to open the application corresponding to the notification content. At this time, the user can answer communication messages on the first device's application. Since the first device has answered the communication message, it then exits the screen mirroring process to the second device. After the user answers a communication message on the first device, the controller monitors the audio and video status information corresponding to the first device's communication message, determining the time point when the first device ends the communication message. When the controller detects that the communication status of the audio and video corresponding to the first device's communication message has changed from "in communication" to "communication ended," the first device can send a screen mirroring request again. The controller responds to the screen mirroring request sent by the first device and sends the request to the second device, thus re-establishing the screen mirroring channel between the first and second devices.

[0127] Figure 7A This is a flowchart illustrating another communication message processing method provided in this disclosure. Figure 7B This is a schematic diagram of the signal flow of another communication message processing method provided in this disclosure embodiment. This disclosure embodiment is... Figure 5A Based on the corresponding embodiments, when the communication message processing method includes step S305, the method further includes:

[0128] S308. When the second device includes an application corresponding to the notification content, open the notification content through the application on the second device.

[0129] S309. Monitor the communication status of the audio and video messages corresponding to the communication messages.

[0130] S310. When the communication status of the audio and video corresponding to the communication message of the second device changes from the communication in progress state to the communication end state, control the display to continue playing the training video stream and the follow-up video stream.

[0131] When the status information of the follow-up video stream displayed on the second device is "follow-up," the user can trigger the opening of the text corresponding to the notification content displayed on the second device. This allows the user to control the second device to pause the playback of the training and follow-up video streams and play the audio corresponding to the notification content. At this time, if the second device includes the application corresponding to the notification content, the user can open the notification content on the second device to answer communication messages. Since the second device is answering communication messages, the follow-up and training video streams played on the second device are paused. After the user answers a communication message on the second device, the controller listens to the status information of the audio and video corresponding to the communication message on the second device and determines the time point when the second device ends the communication message. When the controller detects that the communication status of the audio and video corresponding to the communication message on the second device changes from "communication in progress" to "communication ended," it controls the display to continue playing the training and follow-up video streams.

[0132] Figure 8A This is a flowchart illustrating another communication message processing method provided in this disclosure. Figure 8B This is a schematic diagram of the signal flow of another communication message processing method provided in this disclosure embodiment. This disclosure embodiment is... Figure 2A Based on the corresponding embodiments, obtaining the status information of the follow-up video stream displayed by the second device includes:

[0133] S201. When the duration of the communication message received by the first device is greater than or equal to a preset threshold, the status information of the follow-up video stream displayed by the second device is obtained.

[0134] By monitoring the duration of communication message prompts, if the duration of the communication message prompts is greater than or equal to a preset threshold, the status information of the follow-up video stream displayed by the second device is obtained. If the duration of the communication message prompts is less than the preset threshold, the follow-up video stream projected by the first device is displayed normally on the second device. In this case, the communication message received by the first device is determined to be a useless communication message and there is no need to connect the communication message to the second device.

[0135] This disclosure provides a display device, which includes:

[0136] monitor;

[0137] The controller is configured as follows:

[0138] During the process of the first device projecting its screen to the second device, the communication messages of the first device are monitored.

[0139] Upon receiving a communication message, the system obtains the status information of the follow-up video stream displayed by the second device. The status information includes the follow-up status and the pause status. The communication message represents the communication message sent by the third device to the first device.

[0140] Based on the status information of the follow-up video stream displayed on the second device, control commands are sent to the second device to control the display of notification content. Different control commands correspond to different notification content displayed on the display.

[0141] The display device provided in this embodiment listens to the communication messages of the first device during the process of the first device projecting its screen to the second device. Upon listening to the communication messages, it obtains the status information of the follow-up video stream displayed by the second device, including the follow-up status and the pause status. Based on the status information of the follow-up video stream displayed by the second device, it sends control commands to the second device to control the second device to display notification content. Different control commands correspond to different notification content displayed on the display, ensuring that when the status information of the follow-up video stream displayed by the second device is in the follow-up status, the communication messages sent by the third device received by the first device will not affect the follow-up video stream and training video stream displayed by the second device, thereby improving the user experience.

[0142] As one possible implementation, the controller is optionally configured as follows:

[0143] When the status information of the follow-up video stream displayed by the second device is follow-up status, a first control command is sent to the second device to control the second device to display the text corresponding to the first notification content;

[0144] When the status information of the follow-up video stream displayed on the second device is paused, a second control command is sent to the second device to control the second device to display the text corresponding to the second notification content and to control the speaker to play the audio corresponding to the second notification content.

[0145] As one possible implementation method, optional,

[0146] The controller is also configured as follows:

[0147] When the status information of the follow-up video stream displayed on the second device is "follow-up status", in response to the pause follow-up instruction signal received by the second device, the display is controlled to show the text corresponding to the notification content, and the speaker is controlled to play the audio corresponding to the notification content.

[0148] This disclosure also provides an electronic device, including: a processor, the processor being configured to execute a computer program stored in a memory, the computer program being executed by the processor to implement the steps of the above method embodiments.

[0149] Figure 9 This is a schematic diagram of the structure of an electronic device provided in this disclosure. Figure 9 A block diagram is shown that is suitable for implementing embodiments of the present invention. Figure 9The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0150] like Figure 9 As shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: one or more processors 810, system memory 820, and bus 830 connecting different system components (including system memory 820 and processors).

[0151] Bus 830 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0152] Electronic device 800 typically includes a variety of computer system readable media. These media can be any media that can be accessed by electronic device 800, including volatile and non-volatile media, removable and non-removable media.

[0153] System memory 820 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 840 and / or cache memory 850. Electronic device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 860 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disk drives for reading and writing to removable non-volatile optical disks (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 830 via one or more data media interfaces. System memory 820 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0154] A program / utility 880 having a set (at least one) of program modules 870 may be stored, for example, in system memory 820. Such program modules 870 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 870 typically perform the functions and / or methods described in the embodiments of this invention.

[0155] The processor 810 executes various functional applications and information processing by running at least one of a plurality of programs stored in the system memory 820, such as implementing the method embodiments provided in the embodiments of the present invention.

[0156] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method embodiments.

[0157] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0158] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0159] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0160] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or wide area network (WAN) domain—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0161] This disclosure also provides a computer program product that, when run on a computer, causes the computer to perform the steps of the above-described method embodiments.

[0162] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0163] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A communication message processing method, characterized in that, include: During the process of the first device projecting its screen to the second device, the communication messages of the first device are monitored. Upon receiving the communication message, the status information of the follow-up video stream displayed by the second device is obtained. The status information includes the follow-up status and the pause status. The communication message represents a communication message sent by the third device to the first device. When the status information of the follow-up video stream displayed by the second device is in follow-up status, a first control command is sent to the second device to control the second device to display the text corresponding to the first notification content. This avoids directly sending the notification content received by the first device to the second device while the user is following up on the training video stream played by the second device, thus pausing the training video stream and the follow-up video stream played by the second device. The notification content displayed by the second device is different for different control commands. The notification content includes the first notification content, which includes text.

2. The method according to claim 1, characterized in that, The notification content also includes a second notification content, which includes text and audio; When the status information of the follow-up video stream displayed by the second device is in a paused state, a second control command is sent to the second device to control the second device to display the text corresponding to the second notification content and to control the speaker to play the audio corresponding to the second notification content.

3. The method according to claim 1, characterized in that, The method further includes: When the status information of the follow-up video stream displayed by the second device is "follow-up status", in response to the pause follow-up instruction signal received by the second device, the display is controlled to show the text corresponding to the notification content, and the speaker is controlled to play the audio corresponding to the notification content.

4. The method according to claim 1, characterized in that, The method further includes: When the status information of the follow-up video stream displayed by the second device is in follow-up status, in response to the trigger operation received by the second device, a first instruction is sent to the first device to instruct the first device to open the application corresponding to the notification content; or, the display is controlled to pause the playback of the training video stream and the follow-up video stream, and the speaker is controlled to play the audio corresponding to the notification content.

5. The method according to claim 4, characterized in that, After sending the first instruction to the first device, the method further includes: Monitor the communication status of audio and video messages corresponding to the communication messages of the first device; When the communication status of the audio and video corresponding to the communication message of the first device changes from the communication in progress state to the communication ended state, a screen projection communication channel is established between the first device and the second device.

6. The method according to claim 4, characterized in that, The method further includes: When the second device includes the application corresponding to the notification content, the notification content is opened through the application on the second device. Monitor the communication status of the audio and video messages corresponding to the communication messages; When the communication status of the audio and video corresponding to the communication message of the second device changes from the communication in progress state to the communication ended state, the display is controlled to continue playing the training video stream and the follow-up video stream.

7. The method according to claim 1, characterized in that, The step of obtaining the status information of the follow-up video stream displayed by the second device includes: When the duration of the communication message received by the first device is detected to be greater than or equal to a preset threshold, the status information of the follow-up video stream displayed by the second device is obtained.

8. A display device, characterized in that, The display device includes: monitor; The controller is configured as follows: During the process of the first device projecting its screen to the second device, the communication messages of the first device are monitored. Upon receiving the communication message, the status information of the follow-up video stream displayed by the second device is obtained. The status information includes the follow-up status and the pause status. The communication message represents a communication message sent by the third device to the first device. When the status information of the follow-up video stream displayed by the second device is in follow-up state, a first control command is sent to the second device to control the second device to display the text corresponding to the first notification content. This avoids directly sending the notification content received by the first device to the second device while the user is following up on the training video stream played by the second device, thus pausing the training video stream and the follow-up video stream played by the second device. The notification content displayed on the display is different for different control commands. The notification content includes the first notification content, which includes text.

9. The display device according to claim 8, characterized in that, The controller is specifically configured as follows: When the status information of the follow-up video stream displayed by the second device is in a paused state, a second control command is sent to the second device to control the second device to display the text corresponding to the second notification content and to control the speaker to play the audio corresponding to the second notification content.

10. The display device according to claim 8, characterized in that, The controller is also configured to: When the status information of the follow-up video stream displayed by the second device is "follow-up status", in response to the pause follow-up instruction signal received by the second device, the display is controlled to show the text corresponding to the notification content, and the speaker is controlled to play the audio corresponding to the notification content.

Images