Mirroring method, terminal device and storage medium

Abstract

Embodiments of the present disclosure provide a mirroring method, a terminal device and a storage medium. The mirroring method comprises: in the mirroring process of a terminal device, receiving a first operation on an always-on display interface or a screen-off interface of the terminal device; and executing a target mirroring control instruction triggered by the first operation.

Description

Screen projection method, terminal device and storage medium

[0001] This application claims priority to Chinese patent application No. 202310700863.9, filed on June 13, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of screen projection technology, and in particular to a screen projection method, a terminal device, and a storage medium. Background Art

[0003] With the development of electronic technology, screen mirroring has become widely used. Screen mirroring can be used to project multimedia files (such as videos and audio) from a terminal device (such as a mobile phone or tablet) onto another device (such as a smart TV or smart projector) for playback. For example, you can project a video from your mobile phone onto a smart TV for playback, allowing the user to view the video on their smart TV.

[0004] Summary of the Invention

[0005] In a first aspect, a screen projection method is provided, which is applied to a terminal device. The screen projection method includes: receiving a first operation on a screen-off display interface or a black screen interface of the terminal device during screen projection of the terminal device; and executing a target screen projection control instruction triggered by the first operation.

[0006] In a second aspect, a screen projection device is provided for use in a terminal device. The screen projection device includes: a receiving unit for receiving a first operation on a screen-off display interface or a black screen interface of the terminal device during screen projection of the terminal device; and a processing unit for executing a target screen projection control instruction triggered by the first operation.

[0007] In a third aspect, a terminal device is provided. The terminal device includes a processor and a memory. The memory stores instructions executable by the processor; when the processor is configured to execute the instructions, the terminal device implements the method provided in the first aspect.

[0008] In a fourth aspect, a computer-readable storage medium is provided, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the method provided in the first aspect.

[0009] In a fifth aspect, a computer program product comprising computer instructions is provided, which, when executed on a computer, causes the computer to execute the method provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings are used to provide a further understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation to the technical solution of the present disclosure.

[0011] FIG1 is a schematic diagram of a screen projection scenario according to an embodiment of the present disclosure.

[0012] FIG2 is a flow chart of a screen projection method according to an embodiment of the present disclosure.

[0013] FIG3 is a schematic diagram of a display interface of a terminal device according to an embodiment of the present disclosure.

[0014] FIG4 is a schematic diagram of an off-screen display interface according to an embodiment of the present disclosure.

[0015] FIG5 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure.

[0016] FIG6 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure.

[0017] FIG7 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure.

[0018] FIG8 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure.

[0019] FIG9 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure.

[0020] FIG10 is a flow chart of another screen projection method according to an embodiment of the present disclosure.

[0021] FIG11 is a flow chart of another screen projection method according to an embodiment of the present disclosure.

[0022] FIG12 is a flow chart of another screen projection method according to an embodiment of the present disclosure.

[0023] FIG13 is a structural diagram of a screen projection system according to an embodiment of the present disclosure.

[0024] FIG14 is a schematic diagram of a workflow of a terminal device according to an embodiment of the present disclosure.

[0025] FIG15 is a schematic diagram of a workflow of another terminal device according to an embodiment of the present disclosure.

[0026] FIG16 is a schematic diagram showing the composition of a screen projection device according to an embodiment of the present disclosure.

[0027] FIG17 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] To help those skilled in the art better understand the technical solutions of the embodiments of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0029] In the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0030] In order to facilitate understanding of the technical solutions of the present disclosure, the following is a brief introduction to the terms involved in the present disclosure.

[0031] Always on display (AOD) technology is a low-power display mode. This technology does not need to light up the entire screen of the mobile phone, but only partially lights up the screen, and the screen drawing frequency will also be lower than the normal frequency. Therefore, even if it is always in AOD mode, the display power consumption is relatively low. With AOD technology, users do not need to do too much operation. After the mobile phone screen is off, some important content (time, important information, etc.) can be displayed locally, and the power consumption is low. Furthermore, after the mobile phone turns on the screen-off display function, users can also check the date, time and other information through the mobile phone without turning on the screen, which is more convenient to operate.

[0032] The black screen state means that the terminal device is in standby state, but the screen of the terminal device is not lit, and the screen of the terminal device presents a black picture.

[0033] Screen mirroring technology connects the small screen of a device to the larger screen of a smart TV, allowing users to watch movies in their living room. Currently, all smart TVs support screen mirroring. There are three main wireless screen mirroring technologies: AirPlay (airplay), the Digital Living Network Alliance (DLNA), and Miracast (screen mirroring).

[0034] 1. AirPlay

[0035] AirPlay is a wireless technology developed by Apple that allows you to wirelessly stream images, audio, and video from iOS (Input Output System) devices like iPhone, iPad, and iPod touch to any AirPlay-enabled device via wireless fidelity (Wi-Fi). AirPlay also has a very special feature: AirPlay screen mirroring. This allows you to wirelessly stream the screen on your iPhone or iPad to your TV, allowing your TV to display the same content as your iPhone or iPad, rather than just images or videos.

[0036] DLNA

[0037] The DLNA protocol is an interoperability protocol between computers, mobile devices, and consumer electronics. DLNA can only transfer media data (i.e., music, photos, videos, and links), not other file types, nor can it enable mirroring (screen sharing). The phone or computer is responsible for pushing content, not playing it; the TV plays and displays it directly. DLNA lacks mirroring functionality similar to Apple TV's AirPlay.

[0038] 3. Miracast

[0039] Miracast is a wireless display standard developed by the Wi-Fi Alliance and based on Wi-Fi Direct. Any device that supports this protocol can share video images wirelessly. For example, a phone can use Miracast to play videos or photos directly on a TV or other device. Unlike DLNA, Miracast also has a mirroring function similar to AirPlay, allowing you to project your phone's screen content directly onto a TV screen.

[0040] When a user uses a terminal device for screen projection, the terminal device may not be used for a long time, causing the terminal device to enter the screen-off state or black screen state. However, based on current screen projection technology, the terminal device screen cannot respond to any operations in the screen-off state or black screen state. When the user needs to control the screen projection state, the user needs to manually wake up the terminal device screen, open the screen projection application to enter the screen projection control interface, and then control the screen projection state. This makes the operation of the terminal device during the screen projection process more complicated.

[0041] Based on this, the embodiments of the present disclosure provide a screen projection method, a terminal device, and a storage medium. During the screen projection process of the terminal device, the terminal device can receive an operation for controlling the current screen projection state on the screen off display interface or the black screen interface, and execute the target screen projection control instruction triggered by the operation, thereby completing the control of the current screen projection state. There is no need for the user to wake up the terminal device, the operation is convenient, the operation complexity of the terminal device during the screen projection process is reduced, and the power consumption of the terminal device during the screen projection process is reduced, which helps to improve the user experience.

[0042] The following describes the solutions of the embodiments of the present disclosure in conjunction with the accompanying drawings.

[0043] A screen projection method provided by an embodiment of the present disclosure can be applied to the screen projection scenario shown in Figure 1. Referring to Figure 1, the screen projection scenario includes a terminal device 11 and a display device 12. The terminal device 11 is a screen projection sending device, and the display device 12 is a screen projection receiving device. The terminal device 11 and the display device 12 are communicated with each other, for example, by a wired manner (USB (Universal Serial Bus), bus) or a wireless manner (such as Bluetooth, WiFi).

[0044] It should be noted that multimedia files can be projected between the terminal device 11 and the display device 12. For example, the terminal device 11 and the display device 12 both support a projection protocol (such as the DLNA protocol, the Miracast protocol, etc.).

[0045] In some embodiments, the terminal device 11 may be installed with a screen projection application for screen projection, and the terminal device 11 has the authority to use multimedia files. That is, the terminal device 11 can use / play multimedia files through the screen projection application. The screen projection application in the embodiment of the present disclosure may be an embedded application (i.e., a system application of the terminal device) installed in the terminal device 11 or a downloadable application. An embedded application is an application provided as part of the implementation of a terminal device (such as a mobile phone). A downloadable application is an application that can provide its own Internet Protocol Multimedia Subsystem (IMS) connection. The downloadable application may be an application pre-installed in the terminal, or a third-party application that can be downloaded and installed in the terminal device by the user; it may also be a quick application integrated in an embedded application or a downloadable application.

[0046] In some embodiments, the terminal device 11 can be a device with wireless transceiver capabilities, which can be deployed on land, including indoors or outdoors, handheld, wearable, or vehicle-mounted; it can also be deployed on the water (such as a ship); it can also be deployed in the air (for example, on an airplane, balloon, and satellite). The terminal device 11 can be a tablet computer, mobile phone, desktop computer, laptop, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, vehicle-mounted device, etc. The embodiments of the present disclosure do not impose any special restrictions on the form of the terminal device 11.

[0047] In some embodiments, the display device 12 may have network television functionality that provides broadcast reception and computer support. The display device 12 may be a digital television, network television, Internet Protocol Television (IPTV), or the like. The display device 12 may be a liquid crystal display, an organic light-emitting display, a projection device, or the like. The type, size, and resolution of the display device 12 are not limited. Those skilled in the art will appreciate that the performance and configuration of the display device 12 may be modified as needed.

[0048] It should be understood that Figure 1 is an exemplary structural diagram, and the number of devices included in the screen projection scene shown in Figure 1 is not limited, for example, the number of display devices is not limited. Moreover, in addition to the devices shown in Figure 1, the screen projection scene shown in Figure 1 may also include other devices, which is not limited by this disclosure.

[0049] The screen projection method provided by the present disclosure may be executed by a screen projection device, which may be the terminal device 11 shown in FIG1 . Furthermore, the screen projection device may also be a central processing unit (CPU) of the terminal device 11, or a control module for screen projection in the terminal device 11. In the embodiments of the present disclosure, the screen projection method provided by the embodiments of the present disclosure is described by taking the terminal device 11 executing the screen projection method as an example.

[0050] As shown in FIG2 , an embodiment of the present disclosure provides a screen projection method, which includes the following S101 and S102 .

[0051] In S101, during screen projection of a terminal device, a first operation is received on a screen-off display interface or a black screen interface of the terminal device.

[0052] In some embodiments, when a user needs to project the screen of a terminal device onto the screen of a display device, the user can use the screen projection function on the terminal device to project the screen of the terminal device onto the screen of the display device for display, that is, the content displayed on the screen of the terminal device can be projected onto the screen of the display device for display.

[0053] As an example, projecting the screen of a terminal device onto the screen of a display device may include at least one of the following implementations.

[0054] Implementation method 1: Based on the DLNA method, the terminal device sends the projected multimedia file to the display device, and the display device plays the multimedia file.

[0055] Implementation method 2: The terminal device projects the terminal device's screen to the display device for mirroring display.

[0056] Based on this, implementation 2 may further include at least one of the following implementations.

[0057] Implementation method 2-1: The terminal device projects its main screen to the display device.

[0058] Implementation method 2-2: The terminal device generates a virtual screen and then projects the virtual screen to the display device.

[0059] It should be understood that the above implementation is merely exemplary, and the present disclosure does not limit the implementation of the terminal device projecting the screen to the screen of the display device.

[0060] When a user uses a terminal device to project a screen, if the user needs to control the current projected screen state, the user can perform a first operation for controlling the current projected screen state on the screen-off display interface or the black screen interface of the terminal device. Correspondingly, the terminal device receives the first operation on its screen-off display interface or the black screen interface.

[0061] In some embodiments, the screen-off display interface may also have other names, for example, it may also be called the screen-off display interface.

[0062] As an example, the screen-off display function is enabled by default on the terminal device.

[0063] As another example, the terminal device periodically turns on the screen-off display function. For example, the terminal device turns on the screen-off display function between 07:00 in the morning and 23:00 in the evening, and turns off the screen-off display function between 23:00 in the evening and 07:00 in the morning.

[0064] As another example, the terminal device turns on or off the screen display function in response to a user instruction. For example, after receiving a user instruction to turn on the screen display function, the terminal device turns on the screen display function in response to the instruction. After receiving a user instruction to turn off the screen display function, the terminal device turns off the screen display function in response to the instruction.

[0065] Figure 3 shows a schematic diagram of the display interface of a terminal device according to an embodiment of the present disclosure. Referring to Figure 3 , after a user enters the control interface of the always-on display function, the control interface displays a schematic diagram of the always-on display interface. The schematic diagram of the always-on display interface may display information such as the time, date, and battery level for the user to view. The user can touch control More Styles 31 to view more of the always-on display interface. On the control interface of the always-on display function, the user can touch control 32 to turn the always-on display function on or off. After the user turns on the always-on display function, the user can select a time setting. For example, if the user touches control 33, the terminal device displays the always-on display interface for 10 seconds after receiving the user's on-screen operation. If the user touches control 34, the terminal device keeps the always-on display function on. If the user touches control 35, the terminal device turns the always-on display function on during the set time and turns it off outside of the set time.

[0066] As an example, the screen-off display interface includes a target screen projection control control, and receiving the first operation on the screen-off display interface of the terminal device may be receiving the first operation on the target screen projection control control on the screen-off display interface.

[0067] In some embodiments, the screen off display interface may include at least one projection control control, and the target projection control control is one of the at least one projection control control. The projection control instruction corresponding to the target projection control control is preconfigured according to the type of multimedia file being projected.

[0068] As an example, when the type of multimedia file projected is a video, the projection control instruction includes at least one of the following: an instruction for instructing the video to fast forward, an instruction for instructing the video to fast rewind, an instruction for instructing the play of the next video, an instruction for instructing the play of the previous video, an instruction for instructing the pause of the video, an instruction for instructing the end of projection, an instruction for instructing the increase of the volume of the video, and an instruction for instructing the decrease of the volume of the video.

[0069] As another example, when the type of multimedia file projected is a slideshow, the projection control instruction includes at least one of the following: an instruction for indicating to enlarge the projection ratio of the slideshow, an instruction for indicating to reduce the projection ratio of the slideshow, an instruction for indicating to play the next slideshow, an instruction for indicating to play the previous slideshow, and an instruction for indicating to end the projection.

[0070] In some embodiments, the slides may be presentation documents (PowerPoint, PPT).

[0071] It should be understood that the above examples of multimedia files including videos and slides for screen projection are merely illustrative, and the disclosed embodiments do not limit the types of multimedia files. For example, the type of multimedia files for screen projection may also be documents. The functions of the screen projection control instructions can be adjusted according to the actual application scenario, and this will not be described in detail.

[0072] In the case where the screen-off display interface includes a target projection screen control control, illustratively, taking the type of multimedia file projected as a video as an example, as shown in FIG4 , it is a schematic diagram of a screen-off display interface according to an embodiment of the present disclosure. Referring to FIG4 , the screen-off display interface of the terminal device includes four projection screen control controls, namely, projection screen control control 41, projection screen control control 42, projection screen control control 43, and projection screen control control 44. Projection screen control control 41 is used to indicate increasing the volume of the video, projection screen control control 42 is used to indicate playing the next video, projection screen control control 43 is used to indicate decreasing the volume of the video, and projection screen control control 44 is used to indicate playing the previous video.

[0073] In conjunction with the embodiment shown in Figure 4, during the process of projecting a video on the terminal device, when the user needs to control the current projection state, a first operation can be performed on the target projection control control on the screen-off display interface. Accordingly, the terminal device receives the first operation on the target projection control on the screen-off display interface. For example, when the user needs to play the next video, the user can operate the projection control 42, that is, the projection control 42 is the target projection control. For example, the user can instruct the terminal device to play the next video by clicking on the projection control 42. Accordingly, the terminal device receives the user's click operation on the projection control 42 on the screen-off display interface.

[0074] For example, taking the type of multimedia file projected as a slide as an example, as shown in FIG5 , it is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure. Referring to FIG5 , the screen-off display interface of the terminal device includes four screen-on control controls, namely, screen-on control control 51, screen-on control control 52, screen-on control control 53, and screen-on control control 54. Screen-on control control 51 is used to indicate the scale of the screen to enlarge the slide, screen-on control control 52 is used to indicate the play of the next slide, screen-on control control 53 is used to indicate the scale of the screen to reduce the slide, and screen-on control control 54 is used to indicate the play of the previous slide.

[0075] In conjunction with the embodiment shown in Figure 5, during the process of projecting a slide show on the terminal device, when the user needs to control the current projection state, a first operation can be performed on the target projection control control on the screen off display interface. Accordingly, the terminal device receives the first operation on the target projection control on the screen off display interface. For example, when the user needs to play the next slide, the user can operate the projection control 52, that is, the projection control 52 is the target projection control. For example, the user can instruct the terminal device to play the next slide by clicking on the projection control 52. Accordingly, the terminal device receives the user's click operation on the projection control 52 on the screen off display interface.

[0076] It should be noted that the example of the screen-off display interface including four projection control controls in Figures 4 and 5 is merely illustrative. The embodiments of the present disclosure do not limit the number of projection control controls included in the screen-off display interface, the display style of the projection control controls, and the functions corresponding to each projection control control. The number of projection control controls included in the screen-off display interface, the display style of the projection control controls, and the functions corresponding to each projection control control can be adjusted according to the actual application scenario, which will not be elaborated on.

[0077] The above embodiment is described using the example of including a projection control on the screen-off display interface. As another example, the screen-off display interface may include a command input area. The first operation received on the screen-off display interface of the terminal device may be the first operation received in the command input area.

[0078] For example, as shown in Figure 6, a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure is shown. Referring to Figure 6, the screen-off display interface includes a command input area 61. During screen projection on a terminal device, when a user needs to control the current projection state, the user can perform a first operation in the command input area 61 of the screen-off display interface. Accordingly, the terminal device receives the first operation in the command input area 61.

[0079] It should be understood that when the command input area is included on the screen-off display interface, user operations in non-command input areas will not be mistaken by the terminal device as operations related to screen projection control. Therefore, setting up a command input area on the screen-off display interface can prevent user misoperations from affecting the terminal device during the screen projection process and improve the accuracy of screen projection control.

[0080] As another example, the entire off-screen display interface or black screen interface of the terminal device can be used as the instruction input area. Receiving the first operation on the off-screen display interface or black screen interface of the terminal device can be receiving the first operation on the screen of the terminal device.

[0081] When the screen-off display interface of the terminal device includes a command input area or the entire screen-off display interface or the entire black screen interface of the terminal device is used as the command input area, the function corresponding to the first operation is related to the type of multimedia file projected.

[0082] As an example, when the type of multimedia file projected is a video, the first operation includes at least one of the following: an upward swipe operation to indicate the playback of the next video, a downward swipe operation to indicate the playback of the previous video, a right swipe operation to indicate the increase of the volume of the video, a left swipe operation to indicate the decrease of the volume of the video, a single-click operation to indicate the pause of the video, and a double-click operation to indicate the end of the projection.

[0083] As another example, when the type of multimedia file projected is a slideshow, the first operation includes at least one of the following: a right swipe operation for indicating an enlarged projection ratio of the slideshow, a left swipe operation for indicating a reduced projection ratio of the slideshow, an up swipe operation for indicating playing the next slideshow, a down swipe operation for indicating playing the previous slideshow, and a double-click operation for indicating the end of the projection.

[0084] It should be understood that the functions corresponding to the above different operations are merely exemplary and can be adjusted accordingly according to the application scenario without limitation. For example, when the type of multimedia file projected is a video, a right swipe operation can also be used to indicate video fast forward, and a left swipe operation can also be used to indicate video fast rewind, etc. The sliding operation can also have other names, for example, a left swipe operation can also be called a left swipe operation, etc., which will not be described in detail.

[0085] In some embodiments, the screen display interface includes prompt information, which is used to indicate the operation corresponding to the screen projection control instruction. It should be understood that the content of the prompt information is related to the type of multimedia file being projected.

[0086] Taking the example of a screen-off display interface including a command input area, when the type of multimedia file being projected is a video, FIG7 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure. Referring to FIG7 , the screen-off display interface displays the operation corresponding to each screen projection control instruction. For example, a right swipe increases the volume of the video, a left swipe decreases the volume of the video, an up swipe plays the next video, and a down swipe plays the previous video.

[0087] In the case where the type of multimedia file being projected is a slideshow, FIG8 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure. Referring to FIG8 , the screen-off display interface displays the corresponding operation for each screen projection control instruction. For example, a right swipe increases the projection ratio of the slideshow, a left swipe decreases the projection ratio of the slideshow, an up swipe plays the next slideshow, and a down swipe plays the previous slideshow.

[0088] Taking the entire screen-off display interface as an example of the command input area, when the type of multimedia file being projected is a video, FIG9 is a schematic diagram of another screen-off display interface according to an embodiment of the present disclosure. Referring to FIG9 , the screen-off display interface displays the operation corresponding to each screen projection control instruction. For example, a right swipe operation increases the volume of the video, a left swipe operation decreases the volume of the video, an up swipe operation plays the next video, and a down swipe operation plays the previous video.

[0089] It should be understood that there is prompt information on the screen-off display interface so that when the user needs to control the current projection status, he can determine the operation corresponding to the projection control instruction according to the prompt information displayed on the screen-off display interface, and then perform the correct operation to correctly control the projection status, which helps to improve the user experience.

[0090] In S102, the target screen projection control instruction triggered by the first operation is executed.

[0091] In some embodiments, after receiving the user's first operation, the terminal device determines the target screen projection control instruction corresponding to the first operation, and then sends the target screen projection control instruction to the projection application to instruct the projection application to execute the target screen projection control instruction triggered by the first operation.

[0092] As an example, at least one command mapping table is pre-stored in the terminal device. The command mapping table is the correspondence between operations and screen projection control instructions. Different command mapping tables correspond to different types of multimedia files.

[0093] After receiving the user's first operation, the terminal device can determine the instruction mapping table corresponding to the type of the multimedia file currently projected based on the type of the multimedia file currently projected, and then traverse the instruction mapping table based on the first operation to determine the target projection control instruction corresponding to the first operation.

[0094] For example, taking the type of multimedia file projected as video as an example, the following Table 1 shows an instruction mapping table provided in an embodiment of the present disclosure.

[0095] Table 1

[0096] It can be seen from Table 1 above that the screen projection control instruction corresponding to the upward swipe operation in the sliding operation and the click of the "Next" control in the control operation is to play the next video, the screen projection control instruction corresponding to the downward swipe operation in the sliding operation and the click of the "Previous" control in the control operation is to play the previous video, the screen projection control instruction corresponding to the left swipe operation in the sliding operation and the click of the "Volume-" control is to reduce the volume, and the screen projection control instruction corresponding to the right swipe operation in the sliding operation and the click of the "Volume+" control is to increase the volume.

[0097] For example, assuming that the user's first operation on the screen-off display interface of the terminal device is a swipe-up operation, after receiving the swipe-up operation, the terminal device uses the swipe-up operation as an index to traverse the above Table 1, and determines that the target screen projection control instruction is to play the next video. The terminal device then sends the target screen projection control instruction to the screen projection application of the terminal device to instruct the screen projection application to play the next video.

[0098] Based on the embodiment shown in Figure 2, during the screen projection process of the terminal device, the terminal device can receive a first operation for controlling the current screen projection state on the screen off display interface or the black screen interface, that is, during the screen projection process, the terminal device can act as a remote control that can control the current screen projection state in the screen off display state or the black screen state, and execute the target screen projection control instruction triggered by the user's first operation, thereby completing the control of the current screen projection state without the user waking up the terminal device, the operation is convenient, the complexity of the terminal device's operation during the screen projection process is reduced, and the power consumption of the terminal device during the screen projection process is reduced, which helps to improve the user experience.

[0099] In some embodiments, before receiving the first operation on the screen-off display interface of the terminal device, as shown in FIG10 , the screen projection method may further include the following S201 and S202 .

[0100] In S201, it is monitored whether the multimedia file projected by the terminal device meets the preset conditions.

[0101] In some embodiments, after a terminal device performs screen projection, in order to display the screen-off display interface at an appropriate time for the user to control the current projection state, the terminal device monitors whether the projected multimedia file meets a preset condition. The preset condition includes the completion of multimedia file playback. For example, if the type of the projected multimedia file is a video, the preset condition may be the completion of the projected video playback.

[0102] It should be understood that the preset conditions are related to the type of multimedia file being projected. Those skilled in the art will understand that the preset conditions may be adjusted according to the type of multimedia file being projected, and this is not limited to this.

[0103] In S202, when the multimedia file meets the preset conditions, the screen-off display interface is displayed.

[0104] It should be understood that after detecting that the playback of the currently projected multimedia file has ended, the terminal device automatically displays the screen-off display interface for the user to control the current projection status, which improves the intelligence of the terminal device during the projection process and helps to improve the user experience.

[0105] As an example, when it is detected that the multimedia file projected by the terminal device does not meet the preset conditions, the terminal device can enter a black screen state. That is, the screen of the terminal device is black, but the projection is still in progress.

[0106] It is understandable that the power consumption of the terminal device is lower in the black screen state. When it is detected that the multimedia file projected by the terminal device does not meet the preset conditions, the terminal device enters the black screen state, which helps to reduce the power consumption of the terminal device during the projection process.

[0107] In some embodiments, before receiving the first operation on the screen-off display interface of the terminal device, as shown in FIG11 , the screen projection method may further include the following S301 and S302 .

[0108] In S301, after the terminal device enters a black screen state, a second operation is received.

[0109] In S302, in response to the second operation, a screen-off display interface is displayed.

[0110] It should be understood that the embodiment shown in FIG. 10 above is to enhance the intelligence of the terminal device during screen projection by automatically displaying the screen-off display interface when the terminal device detects that the projected multimedia file meets preset conditions. In some embodiments, after the terminal device enters the black screen state, the terminal device may also receive a second user operation. Then, in response to the second operation, the screen-off display interface is displayed.

[0111] It should be understood that when the user's second operation is received, it means that the user needs to control the current projection state. Therefore, after receiving the second operation, in response to the second operation, the terminal device displays the screen-off display interface for the user to control the current projection state.

[0112] The second operation may be any one of a click operation, a sliding operation, and a touch operation of the user on the screen of the terminal device. The click operation may include a single-click operation, a double-click operation, etc., which is not limited.

[0113] In some embodiments, after the terminal device displays the screen-off display interface, if the user's third operation is not received within a preset time, the terminal device enters a black screen state to reduce the power consumption of the terminal device during the screen projection process. The preset time can be pre-set when the terminal device leaves the factory, or it can be user-defined, and there is no restriction on this. For example, the preset time is 10 seconds (second, S). That is, if the terminal device still does not receive the user's third operation after 10S of displaying the screen-off display interface, the terminal device enters a black screen state. For the description of the third operation, please refer to the above description of the second operation, which will not be repeated here.

[0114] In some embodiments, after the terminal device receives a first operation on the target screen projection control control on the screen-off display interface and executes the target screen projection control instruction triggered by the first operation, as shown in Figure 12, the screen projection method can also include the following S401 and S402.

[0115] In S401, when it is detected that the currently projected first multimedia file is switched to the second multimedia file, the projection control component corresponding to the second multimedia file is extracted.

[0116] In some embodiments, during screen projection on a terminal device, a user may replace a first multimedia file currently being projected with a second multimedia file. Upon detecting that the first multimedia file currently being projected has been switched to the second multimedia file, the terminal device extracts a screen projection control corresponding to the second multimedia file.

[0117] Taking the example of the first multimedia file currently being projected being a video and the second multimedia file being a slide, during the process of the terminal device projecting the video, the user chooses to stop projecting the video and choose to project the slide. When the terminal device detects that the currently projected multimedia file has switched from a video to a slide, it extracts the projection control controls corresponding to the slide.

[0118] In S402, the screen projection control control corresponding to the second multimedia file is used as the target screen projection control control.

[0119] It can be seen from the above description that the projection control instructions corresponding to the target projection control control are pre-configured according to the type of multimedia file being projected. In the case where it is detected that the first multimedia file currently being projected switches to the second multimedia file, if the target projection control control displayed on the screen-off display interface is not replaced, the user may not be able to correctly control the projection in the future, thereby affecting the user's experience. In this way, in the case where it is detected that the first multimedia file currently being projected switches to the second multimedia file, the projection control control corresponding to the second multimedia file is extracted, and the projection control control corresponding to the second multimedia file is used as the target projection control control, so that the user can correctly control the projection status, which helps to ensure the user's experience.

[0120] As an example, when it is detected that the first multimedia file currently being projected switches to the second multimedia file, the terminal device can first determine whether the type of the second multimedia file is the same as the type of the first multimedia file. If the type of the second multimedia file is different from the type of the first multimedia file, the projection control control corresponding to the second multimedia file is extracted, and the projection control control corresponding to the second multimedia file is used as the target projection control control. If the type of the second multimedia file is the same as the type of the first multimedia file, the terminal device may not extract the projection control control corresponding to the second multimedia file, and still use the projection control control corresponding to the first multimedia file as the target projection control. In this way, it helps to reduce the consumption of computing resources of the terminal device.

[0121] The following describes a screen projection method provided by an embodiment of the present disclosure with reference to examples.

[0122] FIG13 is a schematic diagram of a screen projection system according to an embodiment of the present disclosure. Referring to FIG13 , the screen projection system includes a terminal device 11 and a display device 12. The terminal device 11 includes an input module 111, a control module 112, a display module 113, and a screen projection application 114.

[0123] The input module 111 is used to collect user operations, for example, receive user operations on the screen of the terminal device 11, and send the collected operations to the control module 112. The user operations can be divided into the following two categories.

[0124] 1. Click operation (single-click operation, double-click operation), swipe left, swipe right, swipe up, swipe down, etc.

[0125] 2. The user clicks on the projection control on the screen-off display interface.

[0126] In some embodiments, the control module 112 is configured to control the terminal device 11 to enter a black screen state after the terminal device 11 enters the screen projection state; and, at an appropriate time, based on the type of the multimedia file currently being projected, to prompt the user of currently supported operations in the form of an off-screen display interface. If no user operation is received within a preset time, such as 10 seconds, the off-screen display interface is closed, causing the terminal device to enter a black screen state. The appropriate time includes receiving a user operation or the end of playback of the currently projected multimedia file.

[0127] The control module 112 is also used to receive user operations, convert the received user operations into corresponding screen projection control instructions according to the above-mentioned instruction mapping table, and then send the screen projection control instructions to the screen projection application to instruct the screen projection application to execute the screen projection control instructions triggered by the user operations.

[0128] In some embodiments, the display module 113 is used to display an off-screen display interface or a black screen interface.

[0129] In some embodiments, the display module 113 is further configured to display prompt information on the screen-off display interface.

[0130] In some embodiments, the display module 113 is also used to display the target screen projection control controls on the screen-off display interface.

[0131] In some embodiments, the display module 113 is further configured to display a command input area on the screen-off display interface.

[0132] For the relevant description of prompt information and target screen projection control controls, please refer to the corresponding description in S101 above, which will not be repeated here.

[0133] In some embodiments, the screen projection application 114 is used to receive the screen projection control instructions sent by the control module 112 and perform operations corresponding to the screen projection control instructions.

[0134] In some embodiments, the display device 12 is used to display multimedia files projected from the terminal device.

[0135] In conjunction with the screen projection system shown in Figure 13, an example is given of the terminal device displaying the screen-off display interface during the screen projection process. As shown in Figure 14, it is a schematic diagram of the workflow of a terminal device according to an embodiment of the present disclosure. Referring to Figure 14, after the terminal device enters the screen-off projection mode, after the input module collects the user's click operation on the screen-off display interface, the control module determines whether the type of the multimedia file currently projected is a video. In the case that the type of the multimedia file currently projected is a video, the display module displays video-related prompt information on the screen-off display interface to prompt the user for relevant operations. In the case that the type of the multimedia file currently projected is not a video, the display module displays slide-related prompt information on the screen-off display interface to prompt the user for relevant operations. After the display module displays the relevant prompt information on the screen-off display interface, if no user operation is received within 10 seconds, the display module closes the screen-off display interface after 10 seconds to enter a black screen state.

[0136] Screen-off mode refers to the screen-off display mode that the terminal device enters during the screen projection process.

[0137] In conjunction with the screen projection system shown in Figure 13, an example of the interaction between the terminal device and the user during the screen projection process is given. As shown in Figure 15, it is a schematic diagram of the workflow of another terminal device according to an embodiment of the present disclosure. Referring to Figure 15, taking the type of multimedia file projected by the terminal device as a video as an example, after the terminal device enters the screen-off projection mode, the input module receives the user's operation on the screen of the terminal device, such as a swipe-up operation, and then the input module sends the swipe-up operation to the control module. After receiving the swipe-up operation, the control module converts the swipe-up operation into a projection control instruction "next" that can be recognized by the projection application, and sends the projection control instruction "next" to the projection application. After receiving the projection control instruction "next", the projection application executes the projection control instruction "next" to play the next video.

[0138] The above mainly introduces the solution provided by the present disclosure from the perspective of the interaction between each node. It is understandable that each node, such as a terminal device, includes a hardware structure and / or software module corresponding to the execution of each function in order to realize the above functions. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present disclosure.

[0139] FIG16 is a schematic diagram showing the components of a screen projection device according to an embodiment of the present disclosure. As shown in FIG16 , the screen projection device 50 includes a receiving unit 501 and a processing unit 502. In some embodiments, the screen projection device 50 may further include a display unit 503.

[0140] The screen projection device 50 can be the terminal device or a chip in the terminal device. When the screen projection device 50 is used to implement the functions of the terminal device in the above embodiment, each unit is used to implement the following functions, for example.

[0141] The receiving unit 501 is used to receive a first operation on the screen-off display interface or the black screen interface of the terminal device during screen projection of the terminal device.

[0142] The processing unit 502 is used to execute the target screen projection control instruction triggered by the first operation.

[0143] In some embodiments, the screen-off display interface includes a target screen projection control widget, and the receiving unit 501 is, for example, used to receive a first operation on the target screen projection control widget on the screen-off display interface.

[0144] In some embodiments, the projection control instruction corresponding to the target projection control widget is preconfigured according to the type of multimedia file being projected.

[0145] In some embodiments, when the type of the multimedia file is a video, the screen projection control instruction includes at least one of the following: an instruction for instructing to fast forward the video, an instruction for instructing to fast rewind the video, an instruction for instructing to play the next video, an instruction for instructing to play the previous video, an instruction for instructing to pause the video, an instruction for instructing to end screen projection, an instruction for instructing to increase the volume of the video, an instruction for instructing to decrease the volume of the video; or, when the type of the multimedia file is a slideshow, the screen projection control instruction includes at least one of the following: an instruction for instructing to enlarge the screen projection ratio of the slideshow, an instruction for instructing to reduce the screen projection ratio of the slideshow, an instruction for instructing to play the next slideshow, an instruction for instructing to play the previous slideshow, an instruction for instructing to end screen projection.

[0146] In some embodiments, the screen-off display interface includes prompt information, which is used to indicate the operation corresponding to the screen projection control instruction.

[0147] In some embodiments, the screen-off display interface includes a command input area, and the receiving unit 501 is configured to receive a first operation in the command input area, for example.

[0148] In some embodiments, when the type of the multimedia file projected is a video, the first operation includes at least one of the following: an upward swipe operation for indicating the playback of the next video, a downward swipe operation for indicating the playback of the previous video, a right swipe operation for indicating the increase of the volume of the video, a left swipe operation for indicating the decrease of the volume of the video, a single-click operation for indicating the pause of the video, and a double-click operation for indicating the end of the screen projection; when the type of the multimedia file projected is a slide, the first operation includes at least one of the following: a right swipe operation for indicating the enlargement of the screen ratio of the slide, a left swipe operation for indicating the reduction of the screen ratio of the slide, an upward swipe operation for indicating the playback of the next slide, a downward swipe operation for indicating the playback of the previous slide, and a double-click operation for indicating the end of the screen projection.

[0149] In some embodiments, the processing unit 502 is further used to monitor whether the multimedia file projected by the terminal device meets preset conditions.

[0150] The display unit 503 is configured to display a screen-off display interface when the multimedia file meets a preset condition.

[0151] In some embodiments, the preset condition includes the completion of playback of the multimedia file.

[0152] In some embodiments, the receiving unit 501 is further configured to receive a second operation after the terminal device enters a black screen state.

[0153] The display unit 503 is further configured to display a screen-off display interface in response to a second operation.

[0154] In some embodiments, the processing unit 502 is further used to: extract the projection control control corresponding to the second multimedia file when monitoring the switching from the currently projected first multimedia file to the second multimedia file; and use the projection control corresponding to the second multimedia file as the target projection control control.

[0155] It should be noted that the units in Figure 16 may also be referred to as modules, for example, the sending unit may be referred to as a sending module. In addition, in the embodiment shown in Figure 16, the names of the units may not be the names shown in the figure, for example, the receiving unit may also be referred to as a communication unit.

[0156] If the various units in Figure 16 are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present disclosure is essentially or the part that contributes to some technologies or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present disclosure. The storage medium for storing computer software products includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0157] In the case where the projection device 50 implements the functions of the integrated modules in hardware, the present disclosure provides a schematic diagram of the structure of a terminal device. As shown in Figure 17, the terminal device 60 includes: a processor 602, a communication interface 603, and a bus 604. In some embodiments, the terminal device 60 may also include a memory 601.

[0158] Processor 602 may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 602 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof, and may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 602 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0159] The communication interface 603 is used to connect to other devices via a communication network, such as Ethernet, wireless access network, or wireless local area network (WLAN).

[0160] The memory 601 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0161] As an implementation, the memory 601 may exist independently of the processor 602, and the memory 601 may be connected to the processor 602 via a bus 604 for storing instructions or program codes. When the processor 602 calls and executes the instructions or program codes stored in the memory 601, the screen projection method provided in the embodiment of the present disclosure can be implemented.

[0162] In another implementation, the memory 601 may also be integrated with the processor 602 .

[0163] Bus 604 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 604 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG17 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0164] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal device can be divided into different functional modules to complete all or part of the functions described above.

[0165] The present disclosure also provides a computer-readable storage medium, which includes a non-transitory computer-readable storage medium. All or part of the processes in the above-mentioned method embodiments can be completed by computer instructions to instruct the relevant hardware, and the program can be stored in the above-mentioned computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The computer-readable storage medium can be any of the above-mentioned embodiments or memory. The above-mentioned computer-readable storage medium can also be an external storage device of the above-mentioned terminal device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the above-mentioned terminal device. Further, the above-mentioned computer-readable storage medium can also include both the internal storage unit of the above-mentioned terminal device and an external storage device. The above-mentioned computer-readable storage medium is used to store the above-mentioned computer program and other programs and data required by the above-mentioned terminal device. The above-mentioned computer-readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0166] The embodiments of the present disclosure also provide a computer program product, which includes a computer program. When the computer program product runs on a computer, it enables the computer to execute any one of the screen projection methods provided in the above embodiments.

[0167] In the disclosed embodiments, during screen projection on a terminal device, the current projection status can be adjusted directly on the terminal device's off-screen display interface or black screen interface, without waking up the terminal device. This facilitates operation and reduces the complexity of terminal device operations during projection. Furthermore, the power consumption of the terminal device in the off-screen display state or black screen state is lower than that in the always-on state, thereby reducing the complexity of terminal device operations during projection and the power consumption of the terminal device during projection, thereby improving the user experience.

[0168] Although the present disclosure is described herein in conjunction with various embodiments, in the process of implementing the disclosure for which protection is sought, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple components. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0169] Although the present disclosure has been described with reference to specific features and embodiments thereof, it will be apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present disclosure. Accordingly, this specification and the drawings are merely illustrative of the present disclosure as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present disclosure. Obviously, those skilled in the art may make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, the present disclosure is intended to encompass such modifications and variations if they fall within the scope of the claims of the present disclosure and their equivalents.

[0170] The above is only a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or replacements within the technical scope disclosed in the present disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A screen projection method, applied to a terminal device, comprising: During screen projection by the terminal device, receiving a first operation on a screen-off display interface or a black screen interface of the terminal device; Execute the target screen projection control instruction triggered by the first operation.

2. The method according to claim 1, wherein: The screen-off display interface includes a target screen projection control widget; the receiving a first operation on the screen-off display interface of the terminal device includes: Receive a first operation on the target screen projection control control on the screen off display interface.

3. The method according to claim 2, wherein: The screen projection control instruction corresponding to the target screen projection control control is preconfigured according to the type of multimedia file to be projected.

4. The method according to claim 3, wherein: In the case where the type of the multimedia file is a video, the screen projection control instruction includes at least one of the following: an instruction for instructing to fast forward the video, an instruction for instructing to fast rewind the video, an instruction for instructing to play the next video, an instruction for instructing to play the previous video, an instruction for instructing to pause playing the video, an instruction for instructing to end screen projection, an instruction for instructing to increase the volume of the video, and an instruction for instructing to decrease the volume of the video; or, In the case where the type of the multimedia file is a slideshow, the screen projection control instruction includes at least one of the following: an instruction for instructing to enlarge the screen projection ratio of the slideshow, an instruction for instructing to reduce the screen projection ratio of the slideshow, an instruction for instructing to play the next slideshow, an instruction for instructing to play the previous slideshow, and an instruction for instructing to end the screen projection.

5. The method according to claim 1, wherein: The screen-off display interface includes prompt information, and the prompt information is used to indicate the operation corresponding to the screen projection control instruction.

6. The method according to claim 1, wherein: The screen-off display interface includes a command input area; the receiving a first operation on the screen-off display interface of the terminal device includes: A first operation is received in the instruction input area.

7. The method according to claim 1, wherein: In the case where the type of the multimedia file projected is a video, the first operation includes at least one of the following: an upward swipe operation for indicating playing the next video, a downward swipe operation for indicating playing the previous video, a right swipe operation for indicating increasing the volume of the video, a left swipe operation for indicating decreasing the volume of the video, a single-click operation for indicating pausing the playing of the video, and a double-click operation for indicating ending the projection; When the type of the projected multimedia file is a slideshow, the first operation includes at least one of the following: a right swipe operation for indicating to enlarge the projection ratio of the slideshow, a left swipe operation for indicating to reduce the projection ratio of the slideshow, an up swipe operation for indicating to play the next slideshow, a down swipe operation for indicating to play the previous slideshow, and a double-click operation for indicating to end the projection.

8. The method according to claim 1, wherein: Before receiving a first operation on the screen-off display interface of the terminal device, the method further includes: Monitoring whether the multimedia file projected by the terminal device meets the preset conditions; When the multimedia file meets the preset condition, the screen-off display interface is displayed.

9. The method according to claim 8, wherein: The preset condition includes that the playing of the multimedia file is finished.

10. The method according to claim 1, wherein: Before receiving a first operation on the screen-off display interface of the terminal device, the method further includes: After the terminal device enters a black screen state, receiving a second operation; In response to the second operation, the screen-off display interface is displayed.

11. The method according to claim 2, further comprising: When it is detected that the current screen projection is switched from the first multimedia file to the second multimedia file, extracting the screen projection control widget corresponding to the second multimedia file; The screen projection control control corresponding to the second multimedia file is used as the target screen projection control control.

12. A terminal device, comprising: Processor and memory; Wherein, the memory stores instructions executable by the processor; When the processor is configured to execute the instructions, the terminal device implements the method according to any one of claims 1 to 11.

13. A computer-readable storage medium, wherein: The computer-readable storage medium comprises computer instructions, which, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 11.

Images