Application window projection methods, projection devices, smart devices, projection systems, and storage media

Abstract

This invention provides a method, device, smart device, system, and storage medium for casting application windows. The method includes the following steps: in response to a casting operation, adding a casting option to the menu options of a third-party application via a Hook mechanism, and monitoring whether the casting option is selected; in response to a user's selection of the casting option in the menu options of the third-party application, obtaining the corresponding application window display data of the third-party application; and transmitting the application window display data to a display device for display. This invention's method allows users to directly select the casting option from the menu of the third-party application to be cast, is simple to operate, and reduces the likelihood of erroneous casting.

Description

Technical Field

[0001] This invention relates to the field of screen projection technology, and in particular to a method, device, smart device, system, and storage medium for projecting application windows. Background Technology

[0002] Application window casting, also known as application window sharing, refers to transmitting the display screen of a third-party application window running on one smart device to another smart device for real-time display. When casting an application window, the user opens the casting application, selects the window of the third-party application to be cast on the sending smart device (e.g., a PowerPoint presentation window), and then the casting application acquires the display screen of the window, encodes it, and transmits it to the receiving smart device. The receiving application then decodes and displays the encoded display data. However, this method requires the casting application to generate a list of windows for various third-party applications. The user must then select the corresponding application from this list to cast the application window, making the process relatively complex. Furthermore, users may select the wrong application window, leading to incorrect casting. Summary of the Invention

[0003] To overcome the problems existing in related technologies, embodiments of the present invention provide an application window projection method, projection device, smart device, projection system, and storage medium. The application window projection method allows users to directly select projection options from the window menu of the third-party application to be projected, which is simple to operate and less prone to erroneous projection.

[0004] According to a first aspect of the present invention, a method for connecting a smart device to the network is provided, comprising the following steps:

[0005] In response to the screen casting operation, the screen casting option is added to the menu options of the third-party application through the Hook mechanism, and the system listens for whether the screen casting option is selected.

[0006] In response to the user's selection of the screen mirroring option in the menu options of the third-party application, the application window display data of the corresponding third-party application is obtained;

[0007] The application window display data is transmitted to a display device for display.

[0008] According to a second aspect of the present invention, an application window projection device is provided, comprising:

[0009] The screen casting option addition module is used to respond to screen casting operations by adding screen casting options to the menu options of third-party applications through a Hook mechanism, and to listen for whether the screen casting option is selected.

[0010] The display data acquisition module is used to acquire the application window display data of the corresponding third-party application in response to the user's selection of the screen casting option in the menu options of the third-party application;

[0011] The display module is used to transmit the application window display data to the display device for display.

[0012] According to a third aspect of the present invention, a smart device is provided, including a display, a processor, and a memory; the memory stores a computer program adapted to be loaded by the processor and executed as described in any of the above embodiments.

[0013] According to a fourth aspect of the present invention, an application window projection system is provided, comprising a first smart device and a second smart device electrically connected to each other;

[0014] In response to a screen mirroring operation, the first smart device adds the screen mirroring option to the menu options of a third-party application via a Hook mechanism and listens to whether the screen mirroring option is selected; in response to the user's selection of the screen mirroring option in the menu options of the third-party application, it obtains the application window display data of the corresponding third-party application; and transmits the application window display data to the display device for display.

[0015] The second smart device receives and displays the application window display data sent by the first smart device.

[0016] According to a fifth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the application window projection method as described in any of the above embodiments.

[0017] By applying the above technical solution of the present invention, the screen casting option is added to the menu options of a third-party application through the Hook mechanism of the Windows operating system, and the selection of the screen casting option is monitored. When the screen casting option is selected by the user, the application window display data of the third-party application corresponding to the screen casting option selected by the user is obtained and transmitted to the display device for display, thereby realizing convenient screen casting of application windows. The operation is simple and not prone to errors.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention.

[0019] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is an application scenario diagram illustrating an embodiment of the application window projection method of this application;

[0022] Figure 2 A flowchart illustrating an embodiment of this application window projection method;

[0023] Figure 3 A flowchart illustrating a method for adding a screen mirroring option according to one embodiment of this application;

[0024] Figure 4 This is a diagram illustrating the effect of adding the screen mirroring option in one embodiment of this application;

[0025] Figure 5 A flowchart illustrating the acquisition of application window display data is shown in one embodiment of this application;

[0026] Figure 6 This is a structural block diagram of an application window projection device according to one embodiment of this application;

[0027] Figure 7 This is a structural block diagram of a smart device shown in one embodiment of this application. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0030] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0031] Please see Figure 1 , Figure 1 This is an application scenario diagram illustrating an embodiment of the application window projection method of this application.

[0032] The application window projection method is applied in an application window projection system, which includes a first smart device 100 and a second smart device 200 electrically connected to each other. The first smart device 100 can be a computer, tablet, or mobile phone, etc., and has a projection application installed on it. Users can perform projection operations on the projection application to send the application window display data of a third-party application running on the first smart device 100 to the second smart device 200 for display. The second smart device 200 can also be a computer, tablet, or mobile phone, etc., and has a projection application installed on it. The projection application receives and decodes the application window display data of the third-party application to be projected from the first smart device 100, and displays it.

[0033] The typical method for casting application windows involves the casting application on the First Smart Device 100 generating a list of windows for each third-party application. Users can then select the corresponding third-party application to be cast from the window list to achieve the casting of the application window. This process is relatively complicated, and users may select the wrong application window when choosing the target casting application, resulting in incorrect casting.

[0034] Based on the above problems, in this embodiment of the invention, a screen casting application runs on the first smart device 100. The screen casting option is added to the menu options of the third-party application through the Hook mechanism of the Windows operating system, and the system listens for whether the screen casting option is selected. When the screen casting option is selected by the user, the system obtains the application window display data of the third-party application corresponding to the screen casting option selected by the user, and transmits it to the second smart device 200 for display, thereby realizing convenient screen casting of application windows. The operation is simple and not prone to errors.

[0035] In addition, the application window projection method can also be used in an environment that includes a processing device independent of the electronic device. The processing device can implement the application window projection method of this application after being connected to the electronic device.

[0036] The following is an illustration through specific examples.

[0037] According to a first aspect of the present invention, an application window projection method is disclosed, which is applied to a smart device. The smart device can implement the application window projection method of this application purely in software, or it can be implemented using a combination of software and hardware. The smart device can run an application for executing the application window projection method. The application can be presented in a form adapted to the electronic device, such as an APP application, a background program, or in some examples, a system plugin, a web page plugin, etc. The following description uses the aforementioned smart device as a first smart device, which is a computer device.

[0038] Please see Figure 2 , Figure 2 This is a flowchart illustrating an application window projection method according to an embodiment of the present invention.

[0039] The application window casting method, applied to the first smart device, includes the following steps:

[0040] S1: In response to the screen casting operation, the screen casting option is added to the menu options of the third-party application through the Hook mechanism, and the system listens for whether the screen casting option is selected.

[0041] In this embodiment, by adding a screen casting option to the menu of a third-party application, users can directly select the screen casting option in the window of the already opened third-party application, without having to select it from the list of windows generated by the screen casting application. This makes the operation convenient and prevents the wrong application from being selected for screen casting.

[0042] Please also refer to Figure 3 and Figure 4 , Figure 3A flowchart illustrating a method for adding a screen mirroring option according to one embodiment of this application; Figure 4 This application illustrates the effect of adding the screen mirroring option in one embodiment.

[0043] Step S1 includes the following steps:

[0044] S11: In response to the user's input screen mirroring operation, call the Windows operating system's Hook mechanism to obtain the interface of the third-party application and establish a connection with the third-party application.

[0045] In this embodiment, the screen casting operation input by the user can be the launch operation of the screen casting application, or other screen casting input operations after the screen casting application is launched, such as touch operation, or voice wake-up, etc.

[0046] The Windows operating system is installed on the first smart device 100. The Hook mechanism: Hook is a message processing mechanism in the Windows operating system, a system mechanism provided by Windows to replace the "interrupt" in DOS. After hooking a specific system event, once the hooked event occurs, the program that hooked the event will receive a system notification, allowing the program to respond to the event immediately. By obtaining the interface of third-party applications through the Hook mechanism, it is possible to connect with third-party applications and achieve further interaction.

[0047] S12: Obtain the screen casting function and inject the screen casting function into the third-party application through the Hook mechanism.

[0048] In this embodiment, the screen mirroring function is a user-written function based on user needs. This function can be injected into a third-party application via a Hook mechanism. The third-party application calls and executes this function, thereby enabling smart screen mirroring from the smart device. Specifically, this screen mirroring function can at least add menu options to the third-party application and monitor whether the screen mirroring option is selected in the third-party application.

[0049] S13: The screen casting option is generated through the screen casting function and added to the menu options of the third-party application.

[0050] The screen mirroring application generates corresponding screen mirroring options by calling the injected screen mirroring function and adds these options to the menu options of the third-party application. Specifically, the screen mirroring options can be added to the menu options on the menu bar of the third-party application, allowing users to select them directly from the menu bar; or they can be added to the pop-up menu options of the third-party application, allowing users to select screen mirroring options by right-clicking to bring up the pop-up menu. A pop-up menu (also known as a context, shortcut, or pop-up) is a graphical user interface (GUI) that provides convenience for user interaction.

[0051] S14: Listen to whether the screen casting option of the third-party application is selected through the screen casting function.

[0052] If the screen mirroring option in the menu of a third-party application is selected, screen mirroring can be initiated to enable real-time screen mirroring of the application window.

[0053] S2: In response to the user's selection of the screen mirroring option in the menu options of the third-party application, obtain the application window display data of the corresponding third-party application.

[0054] Please see Figure 5 , Figure 5 This is a flowchart illustrating the acquisition of application window display data according to one embodiment of this application.

[0055] Step S2 includes the following steps:

[0056] S21: In response to the user's selection of the screen casting option in the menu options of the third-party application, determine the identifier of the corresponding third-party application to be cast and generate a screen casting command.

[0057] In this embodiment, when the screen mirroring option of a third-party application is selected, the screen mirroring application responds to the user's selection of the screen mirroring option. Each third-party application is assigned a corresponding identifier. The identifier of the third-party application to be mirrored corresponding to the screen mirroring option is determined, so that the Windows operating system can identify the third-party application to be mirrored and obtain the display data of the corresponding application window.

[0058] S22: Establish a connection with the application programming interface (API) of the Windows operating system, and send the screen casting command and the identifier of the third-party application to be cast to the Windows operating system through the API.

[0059] An Application Programming Interface (API) is a set of predefined interfaces (such as functions or HTTP interfaces) or conventions that connect different components of a software system. It provides applications and developers with a set of routines that allow them to access certain software or hardware without needing access to the source code or understanding the details of the internal workings.

[0060] The screen casting command and the identifier of the third-party application to be cast are sent to the Windows operating system, which then searches for the display data of the corresponding third-party application's screen casting window.

[0061] S23: The Windows operating system obtains the application window display data of the third-party application to be cast, based on the casting command and the identifier of the third-party application to be cast. The casting command tells the Windows operating system the business task to be performed, while the identifier is used to locate the third-party application to be cast.

[0062] S3: Transmit the application window display data to the display device for display.

[0063] In this embodiment, the first smart device 100 and the second smart device 200 can be connected via a network or other means. If the first smart device 100 and the second smart device 200 are connected via a network, remote screen mirroring of application windows can be achieved. In this case, the application window display data is encoded by the Windows operating system and then transmitted to the display device for decoding and display.

[0064] When the second smart device 200 receives the encoded application window display data, it needs to decode the application window display data before it can be displayed.

[0065] By applying the above technical solution of the present invention, the screen casting option is added to the menu options of a third-party application through the Hook mechanism of the Windows operating system, and the selection of the screen casting option is monitored. When the screen casting option is selected by the user, the application window display data of the third-party application corresponding to the screen casting option selected by the user is obtained and transmitted to the display device for display, thereby realizing convenient screen casting of application windows. The operation is simple and not prone to errors.

[0066] According to a second aspect of the present invention, an application window projection device is disclosed. This device can be used to execute the application window projection method of the corresponding embodiment of this application, and has corresponding functions and beneficial effects. For details not disclosed in the embodiments of the application window projection device of this application, please refer to the content of the application window projection method of this application.

[0067] Please see Figure 6 This is a structural block diagram of an application window projection device provided in one embodiment of this application.

[0068] The application window projection device 600 includes:

[0069] The screen casting option addition module 601 is used to respond to the screen casting operation by adding the screen casting option to the menu options of the third-party application through the Hook mechanism, and to listen for whether the screen casting option is selected.

[0070] The display data acquisition module 602 is used to acquire the application window display data of the corresponding third-party application in response to the user's selection of the screen casting option in the menu options of the third-party application.

[0071] Display module 603 is used to transmit the application window display data to a display device for display.

[0072] The screen casting option addition module 601 of this application embodiment adds the screen casting option to the menu options of the third-party application through the Hook mechanism of the Windows operating system, and listens to whether the screen casting option is selected; when the screen casting option is selected by the user, the display data acquisition module 602 acquires the application window display data of the third-party application corresponding to the screen casting option selected by the user; the display module 603 transmits the application window display data of the third-party application to be cast to the display device for display, thereby realizing convenient screen casting of application windows, which is simple to operate and less prone to errors.

[0073] It is worth noting that in the embodiments of the above application window projection device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of this application.

[0074] According to a third aspect of this application, a smart device is also provided. The smart device can be any smart terminal with a display screen, such as a computer, mobile phone, tablet computer, interactive flat panel, etc. In the embodiments of this application, the smart device is a computer.

[0075] Please see Figure 7 This is a structural block diagram of a smart device provided in one embodiment of this application. The smart device 700 includes at least one processor 710, at least one network interface 720, a user interface 730, a memory 740, and at least one communication bus 750. The communication bus 750 is used to enable communication between these components.

[0076] The user interface 730 may include an interface for connecting to a display screen, and optionally, the user interface 730 may also include a standard wired interface or a wireless interface.

[0077] The network interface 720 can optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0078] The processor 710 may include one or more processing cores. The processor 710 connects to various parts within the interactive flat panel 700 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored within the processor 710, and by accessing data stored in the memory 740. Optionally, the processor 710 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 710 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 710.

[0079] The memory 740 may include random access memory (RAM) or read-only memory. Optionally, the memory 740 may include a non-transitory computer-readable storage medium. The memory 740 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 740 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 740 may also be at least one storage device located remotely from the aforementioned processor 710. Figure 7 As shown, the memory 740, which serves as a computer storage medium, may include a Windows operating system, a network communication module, a user interface module, a screen mirroring application, and other applications.

[0080] exist Figure 7 In the smart device 700 shown, the user interface 730 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 710 can be used to call the smart device's operating application stored in the memory 740 and execute the relevant operations in the application window projection in the above embodiment.

[0081] The aforementioned smart device can be used to execute the application window projection method of the corresponding embodiment of this application, and has corresponding functions and beneficial effects.

[0082] According to a fourth aspect of this application, an application window projection system is also provided; please refer to [link to relevant documentation]. Figure 1 The application window projection system includes a first smart device 100 and a second smart device 200 that are electrically connected to each other; wherein the first smart device 100 and the second smart device 200 can be connected via a network.

[0083] In response to a user's input of a screen mirroring operation, the first smart device 100 adds the screen mirroring option to the menu options of a third-party application through a Hook mechanism and listens to whether the screen mirroring option is selected; in response to the user's selection of the screen mirroring option in the menu options of the third-party application, it obtains the application window display data of the corresponding third-party application; and transmits the application window display data to a display device for display.

[0084] The second smart device 200 receives and displays the application window display data sent by the first smart device.

[0085] This application embodiment runs a screen mirroring application on a first smart device. The screen mirroring option is added to the menu options of a third-party application using the Windows operating system's hook mechanism. The system monitors whether the screen mirroring option is selected. When the user selects the option, it acquires the application window display data of the third-party application corresponding to the selected option and transmits it to the display device for display. A second smart device 200 receives the application window display data of the third-party application to be mirrored, decodes it, and displays it. This application window screen mirroring method allows users to directly select the mirroring option from the window menu of the third-party application to be mirrored, is simple to operate, and is less prone to incorrect mirroring.

[0086] According to a fifth aspect of this application, a storage medium containing computer-executable instructions is also provided. When executed by a computer processor, these computer-executable instructions are used to perform relevant operations in the application window projection method provided in any embodiment of this application, and possess corresponding functions and beneficial effects. The computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium, which can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0088] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for projecting an application window, characterized in that, Includes the following steps: In response to the user's input screen mirroring operation, the Windows operating system's Hook mechanism is invoked to obtain the interface of the third-party application and establish a connection with the third-party application; Obtain the screen mirroring function and inject the screen mirroring function into a third-party application through the Hook mechanism; The screen casting function generates screen casting options and adds them to the menu options of third-party applications; The screen casting function listens to whether the screen casting option of a third-party application is selected, wherein the third-party application includes all currently running third-party applications; In response to the user's selection of the screen mirroring option in the menu options of the third-party application, the application window display data of the corresponding third-party application is obtained; The application window display data is transmitted to a display device for display.

2. The application window projection method according to claim 1, characterized in that, The screen casting option is generated through the screen casting function and added to the menu options of the pop-up menu of the third-party application.

3. The application window projection method according to claim 1, characterized in that, The step of obtaining the corresponding application window display data of the third-party application in response to the user's selection of the screen mirroring option in the menu options of the third-party application includes: In response to the user's selection of the screen casting option in the menu options of a third-party application, the identifier of the third-party application to be cast to corresponding to the screen casting option is determined, and a screen casting command is generated. A connection is established with the application programming interface of the Windows operating system, and the screen casting command and the identifier of the third-party application to be cast are sent to the Windows operating system through the application programming interface; The Windows operating system obtains the application window display data of the third-party application to be cast based on the casting command and the identifier of the third-party application to be cast.

4. The application window projection method according to claim 3, characterized in that, The method for transmitting the application window display data to the display device for display is as follows: The application window display data is encoded by the Windows operating system and then transmitted to the display device for decoding and display.

5. An application window projection device, characterized in that, include: The screen casting option module is used to respond to user input screen casting operations, call the Windows operating system's Hook mechanism to obtain the interface of the third-party application and establish a connection with the third-party application; obtain the screen casting function, and inject the screen casting function into the third-party application through the Hook mechanism; The screen casting option is generated using the screen casting function and added to the menu options of the third-party application. The screen casting function listens to whether the screen casting option of a third-party application is selected, wherein the third-party application includes all currently running third-party applications; The display data acquisition module is used to acquire the application window display data of the corresponding third-party application in response to the user's selection of the screen casting option in the menu options of the third-party application; The display module is used to transmit the application window display data to the display device for display.

6. A smart device, comprising a display, a processor, and a memory; characterized in that: The memory stores a computer program adapted to be loaded by the processor and executed as described in any one of claims 1 to 4.

7. An application window projection system, characterized in that: This includes a first intelligent device and a second intelligent device that are electrically connected to each other. The first smart device responds to the user's input screen mirroring operation by calling the Hook mechanism of the Windows operating system to obtain the interface of the third-party application and establish a connection with the third-party application; it obtains the screen mirroring function and injects the screen mirroring function into the third-party application through the Hook mechanism. The screen casting function generates screen casting options and adds them to the menu options of third-party applications; The screen casting function listens to whether the screen casting option of a third-party application is selected, wherein the third-party application includes all currently running third-party applications; in response to the user's selection of the screen casting option in the menu of the third-party application, the corresponding application window display data of the third-party application is obtained. The application window display data is transmitted to a display device for display. The second smart device receives and displays the application window display data sent by the first smart device.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the application window projection method as described in any one of claims 1 to 4.

Images