Extended screen display control method and apparatus, head-mounted device, and readable storage medium

Abstract

The present application provides an extended screen display control method and apparatus, a head-mounted device, and a computer-readable storage medium. The method comprises: after a connection is established between a head-mounted device and an electronic device, establishing a first window management area on the head-mounted device, the first window management area being used for synchronizing display content of a second window management area on the electronic device; and receiving target window content sent by the electronic device and displaying the target window content in the first window management area, the target window content being window content located in the second window management area.

Description

Extended screen display control method, device, head-mounted device, and readable storage medium

[0001] This disclosure claims priority to Chinese Patent Application No. 202411901014.0, filed on December 20, 2024, entitled "Extended Screen Display Control Method, Apparatus, Head-Mounted Device and Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of multi-screen display technology, specifically to an extended screen display control method, device, head-mounted device, and computer-readable storage medium. Background Technology

[0003] With the continuous development and advancement of computer network technology, using electronic devices such as computers to display content (e.g., documents, videos, or images) has become commonplace in users' daily lives and work. In related technologies, to improve display and operational efficiency, multiple screens are often added to achieve multi-screen display of electronic device content. However, adding a screen to achieve multi-screen display is costly. Summary of the Invention

[0004] This application provides an extended screen display control method, device, head-mounted device, and computer-readable storage medium, which can realize multi-screen display of electronic devices by using the head-mounted device as an extended screen, thereby reducing the cost of multi-screen display of electronic devices.

[0005] In a first aspect, this application provides an extended screen display control method, which is applied to a head-mounted device, the method comprising:

[0006] After the head-mounted device and the electronic device establish a connection, a first window management area is established on the head-mounted device. The first window management area is used to synchronize the display content of the second window management area on the electronic device.

[0007] Receive target window content sent by the electronic device, wherein the target window content is the window content located in the second window management area;

[0008] Display the content of the target window in the first window management area.

[0009] Secondly, this application provides an extended screen display control method, which is applied to an electronic device, and the method includes:

[0010] After the head-mounted device and the electronic device establish a connection, a second window management area is created on the electronic device, and the display content of the second window management area is synchronized with the display content of the first window management area on the head-mounted device.

[0011] In response to a drag operation targeting the content of a target window on the electronic device, the content of the target window is moved to the second window management area;

[0012] Once the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and the target window content is loaded into the second window management area for display.

[0013] Thirdly, this application provides an extended screen display control device, the extended screen display control device comprising:

[0014] A creation unit is used to create a first window management area on the head-mounted device after a connection is established between the head-mounted device and the electronic device. The first window management area is used to synchronize the display content of the second window management area on the electronic device.

[0015] A receiving unit is configured to receive target window content sent by the electronic device, wherein the target window content is window content located in the second window management area;

[0016] A control unit is used to display the content of the target window in the first window management area.

[0017] Fourthly, this application also provides a head-mounted device, which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes any of the extended screen display control methods provided in this application when it calls the computer program in the memory.

[0018] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to execute the extended screen display control method.

[0019] In this application, firstly, after establishing a connection between the head-mounted device and the electronic device, a first window management area is established on the head-mounted device. The first window management area is used to synchronize the display content of the second window management area on the electronic device. In this way, the head-mounted device is essentially set as an extended screen of the electronic device. The head-mounted device receives the target window content sent by the electronic device and displays the target window content in the first window management area, realizing multi-screen display of the electronic device. This allows the electronic device to achieve multi-screen display without adding a display screen, reducing the cost of multi-screen display of the electronic device. Secondly, since the content is only sent to the head-mounted device in the second window management area, it is convenient to manage the content synchronization when the head-mounted device is used as an extended screen. In this way, all target window content that needs to be shared to the head-mounted device can be quickly shared to the head-mounted device by adding the second management area, improving the synchronization efficiency of the target window content. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 is a schematic block diagram of a head-mounted device provided in an embodiment of this application;

[0022] Figure 2 is a flowchart illustrating an extended screen display control method for head-mounted devices provided in an embodiment of this application.

[0023] Figure 3 is a schematic diagram illustrating an embodiment of data synchronization between the first window management area and the second window management area in this application.

[0024] Figure 4 is a schematic flowchart of an embodiment of the extended screen display control process provided in this application;

[0025] Figure 5 is a flowchart of another embodiment of the extended screen display control process provided in this application;

[0026] Figure 6 is a schematic flowchart of an extended screen display control method for electronic devices provided in an embodiment of this application;

[0027] Figure 7 is a schematic diagram of an embodiment of the extended screen display control device provided in this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0030] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0031] To enable any person skilled in the art to implement and use this application, the following description is provided. In this description, details are set forth for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known processes will not be described in detail to avoid obscuring the description of the embodiments of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in the embodiments of this application.

[0032] This application provides an extended screen display control method, apparatus, head-mounted device, and computer-readable storage medium. The extended screen display control apparatus can be integrated into the head-mounted device. The head-mounted device can be smart glasses, a smart helmet, etc. The smart glasses can be AR (augmented reality) glasses, VR (virtual reality) glasses, MR (mixed reality) glasses, XR (eXtended Reality) glasses, etc., and the smart helmet can be an AR helmet, etc.

[0033] The execution subject of the extended screen display control method in this application embodiment can be the extended screen display control device provided in this application embodiment, or a head-mounted device that integrates the extended screen display control device. The extended screen display control device can be implemented in hardware or software.

[0034] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0035] Figure 1 is a schematic block diagram of a head-mounted device provided in an embodiment of this application.

[0036] As shown in Figure 1, the head-mounted device 100 includes a processor 101 and a memory 102, which are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0037] Specifically, processor 101 provides computing and control capabilities to support the operation of the entire head-mounted device 100. Processor 101 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0038] Specifically, the memory 102 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0039] Those skilled in the art will understand that the structure shown in FIG1 is merely a block diagram of a portion of the structure related to the embodiments of this application, and does not constitute a limitation on the head-mounted device to which the embodiments of this application are applied. A specific head-mounted device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0040] The processor 101 is configured to run a computer program stored in the memory 102, and implement any of the extended screen display control methods provided in this application embodiment when executing the computer program. For example, the processor 101 is configured to run a computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0041] After the head-mounted device and the electronic device establish a connection, a first window management area is established on the head-mounted device. The first window management area is used to synchronize the display content of the second window management area on the electronic device. The target window content sent by the electronic device is received, wherein the target window content is the window content located in the second window management area. The target window content is displayed in the first window management area.

[0042] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the head-mounted device described above can be referred to the corresponding process in the following extended screen display control method embodiment, and will not be repeated here.

[0043] The following describes an extended screen display control method applied to head-mounted devices. Please refer to Figure 2, which is a flowchart illustrating an extended screen display control method provided in an embodiment of this application. This extended screen display control method for head-mounted devices includes steps 201 to 203, wherein:

[0044] 201. After the head-mounted device and the electronic device establish a connection, a first window management area is established on the head-mounted device. The first window management area is used to synchronize the display content of the second window management area on the electronic device.

[0045] Among them, head-mounted devices can be smart glasses, smart helmets, etc.; smart glasses can be AR glasses, VR glasses, MR glasses, XR glasses, etc.; and smart helmets can be AR helmets, etc.

[0046] The electronic device can be a desktop computer, laptop computer, mobile phone, tablet computer, etc. The electronic device displays content (such as the secondary window management area or target window content) through its main display screen (the default primary display screen of the electronic device). This main display screen can be a screen integrated into the electronic device, such as the screen of a laptop computer, or an external display screen, such as the external display screen of a desktop computer. For ease of description, this document uses the term "display on an electronic device," and unless otherwise specified, the "display on an electronic device" described below refers to the display on the main display screen of the electronic device.

[0047] Before establishing the first window management area on the head-mounted device, a connection needs to be established between the head-mounted device and the electronic device. In some embodiments, the head-mounted device and the electronic device can be connected via a wired connection, such as a USB-C cable. In other embodiments, they can be connected wirelessly, such as via Wi-Fi or Bluetooth. Furthermore, an automatic connection mechanism can be set up. For example, the electronic device can automatically detect nearby head-mounted devices (such as AR glasses) and display a connection prompt, asking the user whether they want the head-mounted device to connect as an extended screen for the electronic device. This automatic connection mechanism can reduce manual connection intervention by the user. Furthermore, a connection success notification can be dynamically displayed on the icon in the electronic device's status bar or displayed via sound (such as a cheerful "connection successful" sound). This enhances the user's connection experience through visual and auditory feedback. Furthermore, when connecting wirelessly, an automatic search function can be provided for quick connection. For example, the electronic device can automatically search for nearby Bluetooth devices, and the user can quickly complete pairing by clicking the name of the head-mounted device on the electronic device, simplifying the connection process between the head-mounted device and the electronic device to some extent. Once the head-mounted device is successfully connected to the electronic device, it will be automatically set as an extended screen for the electronic device (at this time, the head-mounted device acts as an extended virtual display) to synchronize the display content of the electronic device.

[0048] The first window management area is a window management area created on the head-mounted device. This first window management area can occupy the entire display area of ​​the head-mounted device. For example, as shown in Figure 3, the first window management area is presented as the entire visible area of ​​the head-mounted device.

[0049] The second window management area is a window management area created on the electronic device. This second window management area occupies a portion of the main display screen of the electronic device. For example, as shown in Figure 3, the second window management area is presented as a small window.

[0050] Once the connection between the head-mounted device and the electronic device is successful, a small, movable, and resizable window will be generated in a certain area of ​​the main display screen of the electronic device as a second window management area, and a virtual window will also be generated on the head-mounted device as a first window management area.

[0051] 202. Receive the target window content sent by the electronic device, wherein the target window content is the window content located in the second window management area.

[0052] The target window content can be an application window, a file, or an image, etc. There are no restrictions on the specific form of the target window content.

[0053] On electronic devices, users can drag and drop target window content, such as application windows, files, or images, to the second window management area. Once the target window content is dragged to the second window management area, the electronic device's main display will show the target window content within that area; simultaneously, the head-mounted device will also display the target window content synchronously within the first window management area. To ensure that the head-mounted device can synchronously display the target window content from the electronic device, when the user moves the target window content to the second window management area via dragging, the electronic device sends the target window content to the head-mounted device, which can then receive this sent content.

[0054] Furthermore, when a user drags and drops target window content such as an application window, file, or image, the electronic device can preview the target window content that will be transmitted to the head-mounted device to ensure the accuracy of the transmitted content.

[0055] 203. Display the content of the target window in the first window management area.

[0056] In step 203, there are multiple ways to display the content of the target window within the first window management area. For example, these include the following methods ① and ②:

[0057] ① In some embodiments, the loading process of the target window content is displayed using a transition effect. In this case, as shown in Figure 4, step 203 may specifically include the following step 203A:

[0058] 203A. Display the content of the target window in the first window management area according to preset display transition parameters, wherein the preset display transition parameters include at least one of parameters that control the content to be gradually enlarged and parameters that control the color to be gradually darkened.

[0059] On electronic devices, users can drag and drop target window content, such as application windows, files, or images, into the second window management area. Once the target window content is dragged into the second window management area, the electronic device can display it according to preset display transition parameters. For example, the color of the target window content gradually changes from light to dark, returning to its original color; the target window content gradually enlarges until it fills the entire second window management area. Simultaneously, the electronic device sends the target window content to a head-mounted device, allowing the head-mounted device to display the target window content synchronously.

[0060] After receiving the target window content sent by the electronic device, the head-mounted device can display the target window content in the first window management area according to the preset display transition parameters. For example, the color of the target window content gradually changes from light to dark to restore the original color of the target window content, and the target window content gradually increases in size until it fills the entire first window management area.

[0061] ② In some embodiments, the loading process of the target window content is displayed using a window adaptive layout effect. In this case, as shown in Figure 5, step 203 may specifically include the following step 203B:

[0062] 203B. Display the target window content in the first window management area according to the first window layout parameters corresponding to the content type of the target window content, wherein the window layout parameters include at least one of window size, display ratio, and window position.

[0063] The window layout parameters may include, but are not limited to, window size, display ratio, and window position. For example, a window layout parameter can be set for each content type. For instance, taking the window ratio as an example, an application window can be set to a 16:9 window ratio, and an image can be set to a 4:3 window ratio.

[0064] The first window layout parameter refers to the window layout parameter used on the head-mounted device, and the second window layout parameter refers to the window layout parameter used on the electronic device. The specific settings of the first window layout parameter and the second window layout parameter can be exactly the same, partially the same, or completely different.

[0065] On electronic devices, users can drag and drop target window content, such as application windows, files, or images, into the second window management area. Once the target window content is dragged into the second window management area, the electronic device can display it according to the second window layout parameters corresponding to the content type of the target window content. For example, the target window content might be centered in the second window management area with a height ratio of 3:2 and a length ratio of 4:3. Simultaneously, the electronic device will send the target window content to a head-mounted device so that the head-mounted device can simultaneously display the target window content.

[0066] After receiving the target window content sent by the electronic device, the head-mounted device can display the target window content in the first window management area according to the first window layout parameters corresponding to the content type of the target window content. For example, the target window content is displayed on the left side of the first window management area according to the height ratio of the target window content to the first window management area being 3:2 and the length ratio being 4:3.

[0067] Furthermore, in some embodiments, after the target window content is loaded in the first window management area, the head-mounted device can also provide visual effect prompts through the edges of the first window management area according to first preset effect parameters. The first preset effect parameters include at least one of animation parameters, light effect change parameters, and color change parameters. Similarly, the second window management area on the electronic device will also provide synchronized visual effect prompts. After the target window content is loaded in the second window management area, the electronic device can also provide visual effect prompts through the edges of the second window management area according to the first preset effect parameters. This allows for dynamic loading of the target window content and enables the target window content to be automatically aligned to the small window edge of the second window management area via an intelligent "snap" function. Thus, the head-mounted device and the electronic device can provide synchronized visual effect prompts after the target window content is loaded.

[0068] Furthermore, in order to improve the management efficiency of the window content displayed on the head-mounted device, thereby improving the user's work efficiency in the virtual environment, the extended screen display control method further includes the following step 204 (not shown in the accompanying drawings):

[0069] 204. Based on the target operation data of the target window content, update the display content of the first window management area.

[0070] The target operation data refers to the data used to manipulate the content of the target window. The target operation data may include at least one of the first operation data and the second operation data.

[0071] The first operation data refers to the data used to manipulate the content of the target window in the first window management area on the head-mounted device. Operations on the target window content in the first window management area can include editing, zooming, annotation, etc. In some embodiments, clicking on the corresponding content in the first window management area on the head-mounted device can bring up an input method keyboard, or sliding the touch panel of the head-mounted device can control the keyboard to pop up or hide, thereby enabling editing operations on the target window content.

[0072] The second operation data refers to the data used to manipulate the content of the target window in the second window management area on the electronic device. Operations on the target window content in the second window management area can include editing, zooming, annotation, etc.

[0073] There are several ways to implement step 204, including, for example:

[0074] (1) In some embodiments, the target operation data is the first operation data, in which case step 204 may specifically include the following step 2041A:

[0075] 2041A. When first operation data of operating on the target window content on the head-mounted device is detected, the display content of the first window management area is updated based on the first operation data.

[0076] On a head-mounted device, users can operate on the content of the target window in the first window management area, such as performing document editing, image zooming, and other operations. At this time, the head-mounted device will update the displayed content of the first window management area based on the first operation data of the operation on the target window content on the head-mounted device.

[0077] On the other hand, in order to synchronize the displayed content between the electronic device and the head-mounted device, when the head-mounted device detects the first operation data of the target window content, it will also synchronize the first operation data of the target window content to the electronic device. This allows the electronic device to update the displayed content of the second window management area based on the first operation data. In this way, by synchronizing the first operation data of the target window content to the electronic device, the head-mounted device can synchronize the operation data of the target window content, such as editing, zooming, and annotation, to the electronic device, ensuring the real-time interaction of window data between the head-mounted device and the electronic device.

[0078] Therefore, firstly, after establishing a connection between the head-mounted device and the electronic device, a first window management area is created on the head-mounted device. This first window management area is used to synchronize the display content of the second window management area on the electronic device. In this way, the head-mounted device is essentially set as an extended screen for the electronic device. When the target window content of the electronic device is dragged to the second window management area created on the electronic device, the head-mounted device receives the target window content sent by the electronic device and displays the target window content in the first window management area. In this way, all target window content that needs to be shared from the electronic device to the head-mounted device can be quickly shared to the head-mounted device by dragging and dropping, improving the synchronization efficiency of the target window content and realizing the synchronization between the electronic device and the head-mounted device. The first aspect is multi-window interaction between wearable devices; secondly, by updating the display content of the first window management area based on the target operation data of the target window content, whether the target window content is modified and managed on the head-mounted device or on other electronic devices, the target window content displayed in the first window management area of ​​the head-mounted device can be updated in real time. There is no need to synchronize the modified content to the head-mounted device after the modification is completed on the electronic device. Thus, the target window content can be managed synchronously on the electronic device or the head-mounted device, improving the management efficiency of the target window content displayed on the head-mounted device, and thereby improving the user's work efficiency in the virtual environment.

[0079] (2) In some embodiments, the target operation data is the second operation data, in which case step 204 may specifically include the following step 2041B:

[0080] 2041B. When receiving second operation data of the target window content sent by the electronic device, the display content of the first window management area is updated based on the second operation data, wherein the second operation data is data of operation on the target window content on the electronic device.

[0081] On an electronic device, a user can perform operations on the target window content in the second window management area, such as document editing, image zooming, etc. At this time, the electronic device will update the displayed content of the second window management area based on the second operation data of the operation on the target window content of the second window management area on the electronic device.

[0082] On the other hand, in order to synchronize the displayed content between the electronic device and the head-mounted device, when the second operation data of the target window content is detected on the electronic device, the electronic device will also synchronize the second operation data of the target window content to the head-mounted device. This allows the head-mounted device to update the displayed content of the first window management area based on the second operation data. In this way, by synchronizing the second operation data of the target window content to the head-mounted device, the electronic device can synchronize the target window content with the head-mounted device when the electronic device performs operations such as editing, zooming, and annotation, ensuring the real-time interaction of window data between the head-mounted device and the electronic device.

[0083] Furthermore, a toolbar can be set on the first window management area, and the toolbar on the first window management area can be displayed in a floating manner. In some embodiments, the method may further include: displaying the toolbar in the first window management area in response to an operation at the edge of the first window management area, wherein the toolbar of the first window management area may include at least one of window resizing controls, window closing controls, window minimizing controls, and editing controls. Here, edge operation refers to operation at the edge of the first window management area (e.g., moving the cursor of the head-mounted device to the edge of the first window management area through gestures, interactive control devices, etc.). In this way, the toolbar of the first window management area can be hidden and displayed, and the user can wake up the toolbar of the first window management area through some operation, such as the toolbar can automatically pop up when the user operates the edge of the first window management area, thus providing a portable toolbar function entry. Therefore, when the user needs to perform operations such as resizing, closing, minimizing, and editing content through the head-mounted device, he / she can move the cursor of the head-mounted device to the edge of the first window management area through gestures, interactive control devices, etc., thereby triggering the waking up of the toolbar of the first window management area, realizing operations such as resizing, closing, minimizing, and editing content through the head-mounted device.

[0084] Similarly, a toolbar can also be set on the second window management area, and the toolbar on the second window management area can be displayed in a floating manner. In some embodiments, the method may further include: displaying the toolbar in the second window management area in response to an operation at the edge of the second window management area, wherein the toolbar of the second window management area may include at least one of window resizing controls, window closing controls, window minimizing controls, and editing controls, where the edge operation is performed at the edge of the second window management area (e.g., moving the cursor of the electronic device to the edge of the second window management area with the mouse). In this way, the toolbar of the second window management area can be hidden and displayed, and the user can wake up the toolbar of the second window management area through some operation, such as the toolbar automatically popping up when the user operates on the edge of the second window management area, thus providing a portable toolbar function entry point. Therefore, when the user needs to perform operations such as resizing, closing, minimizing, and editing content on the electronic device, he / she can move the cursor of the electronic device to the edge of the second window management area with the mouse, thereby triggering the waking up of the toolbar of the second window management area, realizing operations such as resizing, closing, minimizing, and editing content on the electronic device.

[0085] Furthermore, the size or position of the first window management area can be adjusted. In some embodiments, the size or position of the first window management area can be adjusted using a head-mounted device. In this case, the method may further include: when first adjustment data of the adjustment of the first window management area on the head-mounted device is detected, updating the size or position of the first window management area based on the first adjustment data. On the head-mounted device, the user can adjust the first window management area, such as adjusting the size or position of the window. At this time, the head-mounted device will update the size or position of the first window management area based on the first adjustment data of the adjustment of the first window management area on the head-mounted device; thus, the size or position of the first window management area can be adjusted using a head-mounted device.

[0086] In some embodiments, the size or position of the first window management area can be adjusted via an electronic device. The method may further include: when receiving second adjustment data for the first window management area sent by the electronic device, updating the size or position of the first window management area based on the second adjustment data, wherein the second adjustment data is the data used by the electronic device to adjust the first window management area. On the electronic device, a user can perform adjustment operations on the first window management area, such as adjusting its size or position by dragging with a mouse. The electronic device can then send the second adjustment data for the first window management area to a head-mounted device, so that the head-mounted device, upon receiving the second adjustment data, updates the size or position of the first window management area based on the second adjustment data. Thus, the size or position of the first window management area can be adjusted via the electronic device. A cursor control device (such as a mouse) can be used to adjust the first window management area on the electronic device. During the adjustment process via the cursor control device, the cursor size of the cursor control device changes with the size of the first window management area to facilitate adjustment.

[0087] Furthermore, on the head-mounted device, the first window management area or its displayed content can be controlled based on gestures or voice. In some embodiments, in response to a gesture operation on the head-mounted device targeting the first window management area, the head-mounted device adjusts the position or size of the first window management area. This allows users to directly manipulate the first window management area via gestures or voice, improving the management flexibility of the first window management area. Thus, by combining wireless connectivity, real-time synchronization of target window content and operation data, and gesture operations, seamless connection and multi-window interaction between the electronic device and the head-mounted device are achieved, enabling users to flexibly manage multiple windows in a virtual environment.

[0088] In some embodiments, in response to a gesture operation on the head-mounted device targeting the display content of the first window management area, the head-mounted device updates the display content of the first window management area based on the content operation data corresponding to the gesture operation; on the other hand, the head-mounted device also synchronously sends the content operation data corresponding to the gesture operation to the electronic device, so that the electronic device can update the display content of the second window management area based on the content operation data corresponding to the gesture operation.

[0089] In some embodiments, the position or size of the first window management area is adjusted in response to a voice command triggered on the head-mounted device. For example, using voice commands such as "close the current window" or "enlarge the window to full screen" not only enhances operational flexibility but also reduces the need for physical input devices. Thus, by combining wireless connectivity, real-time synchronization of target window content and operation data, and voice commands, seamless connectivity and multi-window interaction between the electronic device and the head-mounted device are achieved, enabling users to flexibly manage multiple windows in a virtual environment.

[0090] In some embodiments, in response to a voice command triggered on the head-mounted device for the display content of the first window management area, on the one hand, the head-mounted device updates the display content of the first window management area according to the content operation data corresponding to the voice command; on the other hand, the head-mounted device also synchronously sends the content operation data corresponding to the voice command to the electronic device, so that the electronic device can update the display content of the second window management area according to the content operation data corresponding to the voice command.

[0091] Furthermore, a gesture feedback mechanism can be provided for the head-mounted device: In some embodiments, when the gesture operation is successful, a visual effect prompt is displayed at the edge of the first window management area according to a second preset effect parameter. The second preset effect parameter includes at least one of animation parameters, light effect change parameters, and color change parameters. For example, when the gesture operation is successful, the edge of the first window management area will emit a light effect. In this way, a gesture feedback mechanism can be provided for the head-mounted device, allowing the user to perceive feedback from the gesture operation.

[0092] In some embodiments, when the gesture operation is successful, the head-mounted device is controlled to provide tactile feedback; for example, when the gesture operation is successful, the head-mounted device vibrates; thus, the user can obtain tactile feedback through the vibration feedback device, allowing the user to perceive the feedback of the gesture operation.

[0093] Furthermore, in some embodiments, users can also customize gesture libraries to meet their needs. For example, users can bind frequently used operations to gestures they design themselves, thereby improving the personalization and efficiency of operations.

[0094] Furthermore, multiple target window contents can be added to the first window management area, and users can also switch the currently displayed content of the first window management area to different target window contents through gesture operations, improving the interactive convenience of the target window contents.

[0095] Furthermore, users can quickly perform multiple operations through continuous gestures, such as closing multiple target window content windows in sequence or continuously adjusting the size of multiple target window content windows, thus enhancing the functionality of gesture operations.

[0096] As can be seen from the above, firstly, after establishing a connection between the head-mounted device and the electronic device, a first window management area is created on the head-mounted device. This first window management area is used to synchronize the display content of the second window management area on the electronic device. In this way, the head-mounted device is essentially set up as an extended screen for the electronic device. The head-mounted device receives the target window content sent by the electronic device and displays it in the first window management area, realizing multi-screen display for the electronic device. This allows the electronic device to achieve multi-screen display without adding a separate display screen, reducing the cost of multi-screen display. Secondly, since content is only sent to the head-mounted device in the second window management area, it facilitates the management of content synchronization when the head-mounted device is used as an extended screen. Thus, all target window content that needs to be shared with the head-mounted device can be quickly shared from the electronic device to the head-mounted device by adding it to the second management area, improving the synchronization efficiency of the target window content.

[0097] The following describes an extended screen display control method applied to electronic devices. Please refer to Figure 6, which is a flowchart illustrating an extended screen display control method provided in an embodiment of this application. This extended screen display control method for electronic devices includes steps 601 to 603, wherein:

[0098] 601. After the head-mounted device and the electronic device establish a connection, a second window management area is established on the electronic device, and the display content of the second window management area is synchronized with the first window management area on the head-mounted device.

[0099] Before establishing a second window management area on the electronic device, it is necessary to establish a connection between the head-mounted device and the electronic device. The connection method between the head-mounted device and the electronic device can be referred to the relevant instructions in step 201, which will not be repeated here.

[0100] Once the connection between the head-mounted device and the electronic device is successful, a small, movable, and resizable window will be generated in a certain area of ​​the main display screen of the electronic device as a second window management area, and a virtual window will also be generated on the head-mounted device as a first window management area.

[0101] 602. In response to a drag operation on the target window content of the electronic device, move the target window content to the second window management area.

[0102] The target window content can be an application window, a file, or an image, etc. There are no restrictions on the specific form of the target window content.

[0103] On electronic devices, users can drag and drop target window content, such as application windows, files, or images, to the second window management area. Once the target window content is dragged to the second window management area, the electronic device's main display will show the target window content within that area; simultaneously, the head-mounted device will also display the target window content synchronously within the first window management area. To ensure that the head-mounted device can synchronously display the target window content from the electronic device, when the user moves the target window content to the second window management area via dragging, the electronic device sends the target window content to the head-mounted device, which can then receive this sent content.

[0104] Furthermore, when a user drags and drops target window content such as an application window, file, or image, the electronic device can preview the target window content that will be transmitted to the head-mounted device to ensure the accuracy of the transmitted content.

[0105] 603. When the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and the target window content is loaded into the second window management area for display.

[0106] On electronic devices, users can drag and drop target window content, such as application windows, files, or images, to the second window management area. Once the target window content is dragged to the second window management area, the electronic device's main display will show the target window content within that area; simultaneously, the head-mounted device will also display the target window content synchronously within the first window management area. To ensure that the head-mounted device can synchronously display the target window content from the electronic device, when the user moves the target window content to the second window management area via dragging, the electronic device sends the target window content to the head-mounted device, which can then receive this sent content.

[0107] The specific implementation of "displaying the content of the target window in the first window management area" can be found in the relevant description above (such as step 203), and will not be repeated here.

[0108] There are multiple ways to implement "loading the content of the target window into the second window management area for display" in step 603, including, for example:

[0109] (1) In some embodiments, the loading process of the target window content is displayed using a transition effect. In this case, step 603 may specifically include the following step 603A:

[0110] 603A. When the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and the target window content is displayed in the second window management area according to preset display transition parameters, wherein the preset display transition parameters include at least one of parameters that control the content to be gradually enlarged and parameters that control the color to be gradually darkened.

[0111] On electronic devices, users can drag and drop target window content, such as application windows, files, or images, into the second window management area. Once the target window content is dragged into the second window management area, the electronic device can display it according to preset display transition parameters. For example, the color of the target window content gradually changes from light to dark, returning to its original color; the target window content gradually enlarges until it fills the entire second window management area. Simultaneously, the electronic device sends the target window content to a head-mounted device, allowing the head-mounted device to display the target window content synchronously.

[0112] (2) In some embodiments, the loading process of the target window content is displayed using a transition effect. In this case, step 603 may specifically include the following step 603B:

[0113] 603B. When the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and the target window content is displayed in the second window management area according to the second window layout parameters corresponding to the content type of the target window content. The window layout parameters include at least one of window size, display ratio, and window position.

[0114] On electronic devices, users can drag and drop target window content, such as application windows, files, or images, into the second window management area. Once the target window content is dragged into the second window management area, the electronic device can display it according to the second window layout parameters corresponding to the content type of the target window content. For example, the target window content might be centered in the second window management area with a height ratio of 3:2 and a length ratio of 4:3. Simultaneously, the electronic device will send the target window content to a head-mounted device so that the head-mounted device can simultaneously display the target window content.

[0115] Furthermore, to improve the efficiency of window content management, the method also includes step 604 (not shown in the accompanying drawings):

[0116] 604. Based on the target operation data of the target window content, update the display content of the second window management area.

[0117] There are several ways to implement step 604, including, for example:

[0118] (1) In some embodiments, the target operation data is the first operation data, in which case step 604 may specifically include the following step 6041A:

[0119] 6041A. When second operation data on the target window content is detected on the electronic device, the display content of the second window management area is updated based on the second operation data.

[0120] When second operation data on the target window content is detected on the electronic device, the second operation data is also sent to the head-mounted device so that the head-mounted device can update the display content of the first window management area based on the second operation data.

[0121] On electronic devices, users can operate on the target window content in the second window management area, such as performing document editing, image zooming, etc. At this time, on one hand, the electronic device will update the displayed content of the second window management area based on the second operation data of the operation on the target window content on the electronic device. On the other hand, to synchronize the displayed content between the electronic device and the head-mounted device, when the second operation data of the target window content is detected on the electronic device, the electronic device will also synchronize the second operation data of the target window content to the head-mounted device. This allows the head-mounted device to update the displayed content of the first window management area based on the second operation data. Thus, by synchronizing the second operation data of the target window content to the head-mounted device, the electronic device can synchronize operations such as editing, zooming, and annotation on the target window content to the head-mounted device, ensuring real-time interaction of window data between the two devices.

[0122] (2) In some embodiments, the target operation data is the second operation data, in which case step 604 may specifically include the following step 6041B:

[0123] 6041B. When receiving first operation data of the target window content sent by the head-mounted device, the display content of the second window management area is updated based on the first operation data, wherein the first operation data is data of operation on the target window content on the head-mounted device.

[0124] Furthermore, after the target window content is loaded in the second window management area, visual effects are provided through the edge of the first window management area according to the first preset effect parameters. The first preset effect parameters include at least one of animation parameters, light effect change parameters, and color change parameters. In this way, the loading of the target window content can be made dynamic, and the target window content can be automatically aligned to the edge of the small window in the second window management area through the intelligent "snap" function.

[0125] Furthermore, a toolbar can also be set on the second window management area, and the toolbar on the second window management area can be displayed in a floating manner. In some embodiments, the method may further include: displaying the toolbar in the second window management area in response to an operation at the edge of the second window management area, wherein the toolbar of the second window management area may include at least one of window resizing controls, window closing controls, window minimizing controls, and editing controls, where the edge operation is performed at the edge of the second window management area (e.g., moving the cursor of the electronic device to the edge of the second window management area with the mouse). In this way, the toolbar of the second window management area can be hidden and displayed, and the user can wake up the toolbar of the second window management area through some operation, such as the toolbar automatically popping up when the user operates on the edge of the second window management area, thus providing a portable toolbar function entry point. Therefore, when the user needs to perform operations such as resizing, closing, minimizing, and editing content on the electronic device, he / she can move the cursor of the electronic device to the edge of the second window management area with the mouse, thereby triggering the waking up of the toolbar of the second window management area, realizing operations such as resizing, closing, minimizing, and editing content on the electronic device.

[0126] Furthermore, the method also includes: when second adjustment data for adjusting the first window management area is detected on the electronic device, sending the second adjustment data to the electronic device. On the electronic device, a user can perform adjustment operations on the first window management area, such as adjusting its size or position by dragging with a mouse. In this case, the electronic device can send the second adjustment data for adjusting the first window management area to the head-mounted device, so that the head-mounted device, upon receiving the second adjustment data, updates the size or position of the first window management area based on the second adjustment data; thus, the size or position of the first window management area can be adjusted via the electronic device. A cursor control device (such as a mouse) can be used to adjust the first window management area on the electronic device. During the adjustment process via the cursor control device, the cursor size of the cursor control device changes with the size of the first window management area to facilitate adjustment.

[0127] For the sake of simplicity, the description of each embodiment is emphasized. For the specific implementation process and beneficial effects of the extended screen display control method applied to electronic devices, please refer to the relevant description of the extended screen display control method applied to head-mounted devices for details not described in detail.

[0128] Furthermore, to better implement the extended screen display control method in this application embodiment, based on the extended screen display control method, this application embodiment also provides an extended screen display control device, as shown in FIG6, which is a schematic diagram of an embodiment of the extended screen display control device provided in this application embodiment. The extended screen display control device 700 includes:

[0129] The creation unit 701 is used to create a first window management area on the head-mounted device after a connection is established between the head-mounted device and the electronic device. The first window management area is used to synchronize the display content of the second window management area on the electronic device.

[0130] The receiving unit 702 is configured to receive target window content sent by the electronic device, wherein the target window content is window content located in the second window management area;

[0131] Control unit 703 is used to display the content of the target window in the first window management area.

[0132] In some embodiments, the control unit 703 is used for:

[0133] When first operation data on the target window content is detected on the head-mounted device, the display content of the first window management area is updated based on the first operation data;

[0134] And / or, when receiving second operation data of the target window content sent by the electronic device, the display content of the first window management area is updated based on the second operation data, wherein the second operation data is data of operation on the target window content on the electronic device.

[0135] In some embodiments, the control unit 703 is used for:

[0136] According to preset display transition parameters, the content of the target window is displayed in the first window management area. The preset display transition parameters include at least one of parameters that control the content to be gradually enlarged and parameters that control the color to be gradually darkened.

[0137] In some embodiments, the control unit 703 is used for:

[0138] After the target window content is loaded in the first window management area, visual effects are displayed at the edge of the first window management area according to the first preset effect parameters. The first preset effect parameters include at least one of animation parameters, light effect change parameters, and color change parameters.

[0139] In some embodiments, the control unit 703 is used for:

[0140] According to the first window layout parameters corresponding to the content type of the target window content, the target window content is displayed in the first window management area, wherein the window layout parameters include at least one of window size, display ratio, and window position.

[0141] In some embodiments, the control unit 703 is used for:

[0142] In response to an operation at the edge of the first window management area, a toolbar is displayed in the first window management area, wherein the toolbar includes at least one of a window resizing control, a window closing control, a window minimizing control, and an editing control.

[0143] In some embodiments, the control unit 703 is used for:

[0144] When first adjustment data for adjusting the first window management area is detected on the head-mounted device, the size or position of the first window management area is updated based on the first adjustment data;

[0145] And / or, when receiving second adjustment data of the first window management area sent by the electronic device, the size or position of the first window management area is updated based on the second adjustment data, wherein the second adjustment data is the data on the adjustment of the first window management area by the electronic device.

[0146] In some embodiments, the control unit 703 is used for:

[0147] In response to a gesture operation on the head-mounted device targeting the first window management area, the position or size of the first window management area is adjusted;

[0148] And / or, in response to a gesture operation on the head-mounted device targeting the display content of the first window management area, update the display content of the first window management area;

[0149] And / or, in response to a voice command triggered on the first window management area on the head-mounted device, adjust the position or size of the first window management area;

[0150] And / or, in response to a voice command triggered on the head-mounted device for the display content of the first window management area, update the display content of the first window management area.

[0151] In some embodiments, the control unit 703 is used for:

[0152] When the gesture operation is successful, visual effects are displayed at the edge of the first window management area according to the second preset effect parameters. The second preset effect parameters include at least one of animation parameters, light effect change parameters, and color change parameters.

[0153] Alternatively, once the gesture operation is successful, the head-mounted device can be controlled to provide tactile feedback.

[0154] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous embodiment of the extended screen display control method, which will not be repeated here.

[0155] Those skilled in the art will understand that all or part of the steps in the above extended screen display control method can be accomplished by instructions, or by controlling related hardware through instructions. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0156] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of computer programs that can be loaded by a processor to execute any of the extended screen display control methods provided in embodiments of this application.

[0157] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0158] In the above embodiments of the extended screen display control device, computer-readable storage medium, and head-mounted device, the descriptions of each embodiment have different focuses. Parts not described in detail in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes and beneficial effects of the extended screen display control device, computer-readable storage medium, head-mounted device, and their corresponding units described above can be referred to the description of the extended screen display control method in the above embodiments, and will not be repeated here.

[0159] The above provides a detailed description of an extended screen display control method, apparatus, head-mounted device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An extended screen display control method, the extended screen display control method being applied to a head-mounted device, the method comprising: After the head-mounted device and the electronic device establish a connection, a first window management area is established on the head-mounted device. The first window management area is used to synchronize the display content of the second window management area on the electronic device. Receive target window content sent by the electronic device, wherein the target window content is the window content located in the second window management area; as well as Display the content of the target window in the first window management area.

2. The extended screen display control method of claim 1, wherein, The method further includes: When first operation data on the target window content is detected on the head-mounted device, the display content of the first window management area is updated based on the first operation data; And / or, when receiving second operation data of the target window content sent by the electronic device, the display content of the first window management area is updated based on the second operation data, wherein the second operation data is data of operation on the target window content on the electronic device.

3. The extended screen display control method of claim 1, wherein, Displaying the target window content in the first window management area includes: According to preset display transition parameters, the content of the target window is displayed in the first window management area. The preset display transition parameters include at least one of parameters that control the content to be gradually enlarged and parameters that control the color to be gradually darkened.

4. The extended screen display control method of claim 3, wherein, The method further includes: After the target window content is loaded in the first window management area, visual effects are displayed at the edge of the first window management area according to the first preset effect parameters. The first preset effect parameters include at least one of animation parameters, light effect change parameters, and color change parameters.

5. The extended screen display control method of claim 1, wherein, Displaying the target window content in the first window management area includes: According to the first window layout parameters corresponding to the content type of the target window content, the target window content is displayed in the first window management area, wherein the window layout parameters include at least one of window size, display ratio, and window position.

6. The extended screen display control method of claim 1, wherein, The method further includes: In response to an operation at the edge of the first window management area, a toolbar is displayed in the first window management area, wherein the toolbar includes at least one of a window resizing control, a window closing control, a window minimizing control, and an editing control.

7. The extended screen display control method of claim 1, wherein, The method further includes: When first adjustment data for adjusting the first window management area is detected on the head-mounted device, the size or position of the first window management area is updated based on the first adjustment data; And / or, when receiving second adjustment data of the first window management area sent by the electronic device, the size or position of the first window management area is updated based on the second adjustment data, wherein the second adjustment data is the data on the adjustment of the first window management area by the electronic device.

8. The extended screen display control method of claim 1, wherein, The method further includes: In response to a gesture operation on the head-mounted device targeting the first window management area, the position or size of the first window management area is adjusted; And / or, in response to a gesture operation on the head-mounted device targeting the display content of the first window management area, update the display content of the first window management area; And / or, in response to a voice command triggered on the first window management area on the head-mounted device, adjust the position or size of the first window management area; And / or, in response to a voice command triggered on the head-mounted device for the display content of the first window management area, update the display content of the first window management area.

9. The extended screen display control method of claim 8, wherein, The method further includes: When the gesture operation is successful, visual effects are displayed at the edge of the first window management area according to the second preset effect parameters. The second preset effect parameters include at least one of animation parameters, light effect change parameters, and color change parameters. Alternatively, after the gesture operation is successful, the head-mounted device can be controlled to provide tactile feedback.

10. An extended screen display control method, the extended screen display control method being applied to an electronic device, the method comprising: After the head-mounted device and the electronic device establish a connection, a second window management area is created on the electronic device, and the display content of the second window management area is synchronized with the display content of the first window management area on the head-mounted device. In response to a drag operation targeting the content of a target window on the electronic device, the content of the target window is moved to the second window management area; as well as Once the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and the target window content is loaded into the second window management area for display.

11. The extended screen display control method of claim 10, wherein, The method further includes: When second operation data on the target window content is detected on the electronic device, the display content of the second window management area is updated based on the second operation data, and the second operation data is sent to the head-mounted device; or When second adjustment data for adjusting the first window management area is detected on the electronic device, the second adjustment data is sent to the electronic device.

12. The extended screen display control method of claim 10, wherein, The step of sending the target window content to the head-mounted device and loading the target window content into the second window management area for display after the target window content moves to the second window management area includes: When the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and displayed in the second window management area according to preset display transition parameters. The preset display transition parameters include at least one of parameters controlling the content to gradually enlarge and parameters controlling the color to gradually darken; or When the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and the target window content is displayed in the second window management area according to the second window layout parameters corresponding to the content type of the target window content. The window layout parameters include at least one of window size, display ratio, and window position.

13. The extended screen display control method of claim 10, wherein, The method further includes: After the target window content is loaded in the second window management area, visual effects are applied through the edges of the first window management area according to the first preset effect parameters. The first preset effect parameters include at least one of animation parameters, lighting effect parameters, and color change parameters; or In response to an operation at the edge of the second window management area, a toolbar is displayed in the second window management area, wherein the toolbar includes at least one of a window resizing control, a window closing control, a window minimizing control, and an editing control.

14. An extended screen display control device, the extended screen display control device comprising: A creation unit is used to create a first window management area on the head-mounted device after a connection is established between the head-mounted device and the electronic device. The first window management area is used to synchronize the display content of the second window management area on the electronic device. A receiving unit is configured to receive target window content sent by the electronic device, wherein the target window content is window content located in the second window management area; as well as A control unit is used to display the content of the target window in the first window management area.

15. The extended screen display control device according to claim 14, wherein, The control unit is used for: When first operation data on the target window content is detected on the head-mounted device, the display content of the first window management area is updated based on the first operation data; And / or, when receiving second operation data of the target window content sent by the electronic device, the display content of the first window management area is updated based on the second operation data, wherein the second operation data is data of operation on the target window content on the electronic device.

16. The extended screen display control device of claim 14, wherein, The control unit is used for: According to preset display transition parameters, the content of the target window is displayed in the first window management area. The preset display transition parameters include at least one of parameters that control the content to be gradually enlarged and parameters that control the color to be gradually darkened.

17. The extended screen display control device according to claim 16, wherein, The control unit is used for: After the target window content is loaded in the first window management area, visual effects are displayed at the edge of the first window management area according to the first preset effect parameters. The first preset effect parameters include at least one of animation parameters, light effect change parameters, and color change parameters.

18. The extended screen display control device according to claim 14, wherein, The control unit is used for: According to the first window layout parameters corresponding to the content type of the target window content, the target window content is displayed in the first window management area, wherein the window layout parameters include at least one of window size, display ratio, and window position.

19. A head-mounted device comprising a processor and a memory, the memory storing a computer program, wherein the processor, when invoking the computer program in the memory, executes the extended screen display control method as described in any one of claims 1 to 13.

20. A computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the extended screen display control method according to any one of claims 1 to 13.

Images