Expansion screen display control method and device, head-mounted equipment and readable storage medium

Abstract

The invention provides an expansion screen display control method and device, head-mounted equipment and a computer readable storage medium. The expansion screen display control method comprises the steps that after a head-mounted device is connected with an electronic device, a first window management area is established on the head-mounted device, and the first window management area is used for synchronizing display content of a second window management area on the electronic device; target window content sent by the electronic equipment is received, and the target window content is window content located in the second window management area; and displaying the target window content in the first window management area. The multi-screen display of the electronic equipment can be realized by taking the head-mounted equipment as the extension screen of the electronic equipment, and the multi-screen display cost of the electronic equipment is reduced.

Description

Technical Field

[0001] The present application relates to the field of multi-screen display technology, and in particular to an extended screen display control method, device, head-mounted device and computer-readable storage medium. Background Art

[0002] With the continuous development and progress of computer network technology, using electronic devices such as computers to display content (such as documents, videos or pictures, etc.) has become a normal part of users' daily life or work. In related technologies, in order to improve display and operation efficiency, a display screen is added to realize multi-screen display of electronic device content. However, the cost of adding a display screen to realize multi-screen display is high. Summary of the invention

[0003] The present application provides an extended screen display control method, device, head-mounted device and computer-readable storage medium, which can realize multi-screen display of electronic devices by using the head-mounted device as an extended screen of the electronic device, thereby reducing the multi-screen display cost of the electronic device.

[0004] In a first aspect, the present application provides an extended screen display control method, the extended screen display control method is applied to a head mounted device, the method comprising:

[0005] After the head mounted device is connected to the electronic device, a first window management area is established on the head mounted device, where the first window management area is used to synchronize display content of a second window management area on the electronic device;

[0006] Receiving target window content sent by the electronic device, wherein the target window content is window content in the second window management area;

[0007] The target window content is displayed in the first window management area.

[0008] In a second aspect, the present application provides an extended screen display control method, the extended screen display control method is applied to an electronic device, the method comprising:

[0009] After the head mounted device is connected to the electronic device, a second window management area is established on the electronic device, and the display content of the second window management area is synchronized with the display content of the first window management area on the head mounted device;

[0010] In response to a drag operation on a target window content of the electronic device, moving the target window content to the second window management area;

[0011] When the target window content is moved to the second window management area, the target window content is sent to the head mounted device, and the target window content is loaded into the second window management area for display.

[0012] In a third aspect, the present application provides an extended screen display control device, the extended screen display control device comprising:

[0013] A creating unit, configured to establish a first window management area on the head mounted device after the head mounted device is connected to the electronic device, wherein the first window management area is used to synchronize display content of a second window management area on the electronic device;

[0014] a receiving unit, configured to receive target window content sent by the electronic device, wherein the target window content is window content in the second window management area;

[0015] A control unit is used to display the target window content in the first window management area.

[0016] In a fourth aspect, the present application also provides a head-mounted device, which includes a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, it executes any extended screen display control method provided in the present application.

[0017] In a fifth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the extended screen display control method.

[0018] In the present application, on the first hand, after the head-mounted device is connected to the electronic device, a first window management area is established on the head-mounted device, and the first window management area is used to synchronize the display content of the second window management area on the electronic device, so that the head-mounted device is equivalent to being set as an extended screen of the electronic device; the head-mounted device receives the target window content sent by the electronic device, and displays the target window content in the first window management area, so as to realize multi-screen display of the electronic device, so that the electronic device can realize multi-screen display without increasing the display screen, thereby reducing the multi-screen display cost of the electronic device, and on the second hand, since it is sent to the head-mounted device only when it is in the second window management area, it is convenient to manage the content synchronization management when the head-mounted device is used as an extended screen, so that all target window contents that need to be shared to the head-mounted device for display on the electronic device can be quickly shared to the head-mounted device by adding the second management area, thereby improving the synchronization efficiency of the target window content. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a schematic structural block diagram of a head mounted device provided in an embodiment of the present application;

[0021] Figure 2 It is a flow chart of an extended screen display control method applied to a head mounted device provided in an embodiment of the present application;

[0022] Figure 3 This is a schematic diagram illustrating an embodiment of data synchronization between the first window management area and the second window management area in the embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of an embodiment of the extended screen display control process provided in the embodiments of the present application;

[0024] Figure 5 is a flowchart of another embodiment of the extended screen display control process provided in the embodiments of the present application;

[0025] Figure 6 This is a flow chart of an extended screen display control method for an electronic device provided in an embodiment of the present application;

[0026] Figure 7 It is a schematic diagram of the structure of an embodiment of the extended screen display control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0028] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0029] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0030] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0031] The embodiments of the present application provide an extended screen display control method, device, head-mounted device, and computer-readable storage medium. The extended screen display control device can be integrated in a head-mounted device. The head-mounted device can be smart glasses, smart helmets, etc. The smart glasses can be AR (augmented reality) glasses, VR (Virtual Reality) glasses, MR (Mixed Reality) glasses, XR (eXtended Reality) glasses, etc. The smart helmet can be an AR helmet, etc.

[0032] The executor of the extended screen display control method of the embodiment of the present application may be the extended screen display control device provided in the embodiment of the present application, or a head-mounted device integrating the extended screen display control device, wherein the extended screen display control device may be implemented in hardware or software.

[0033] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0034] Figure 1 It is a structural schematic block diagram of a head-mounted device provided in an embodiment of the present application.

[0035] like Figure 1As shown, the head mounted device 100 includes a processor 101 and a memory 102, and the processor 101 and the memory 102 are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0036] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire head-mounted device 100. The processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0037] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0038] Those skilled in the art will understand that Figure 1 The structure shown in the figure is merely a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the head-mounted device to which the embodiment of the present application is applied. The specific head-mounted device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0039] The processor 101 is used to run the computer program stored in the memory 102, and implement any one of the extended screen display control methods provided in the embodiments of the present application when executing the computer program. For example, the processor 101 is used to run the computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0040] After the head-mounted device establishes a connection with the electronic device, a first window management area is established on the head-mounted device, and the first window management area is used to synchronize the display content of the second window management area on the electronic device; the target window content sent by the electronic device is received, wherein the target window content is the window content in the second window management area; and the target window content is displayed in the first window management area.

[0041] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and brevity of description, the specific working process of the head-mounted device described above can refer to the corresponding process in the following extended screen display control method embodiment, and will not be repeated here.

[0042] The following describes the extended screen display control method for head-mounted devices. Figure 2 , Figure 2 201 to 203, wherein:

[0043] 201. After a head mounted device is connected to an electronic device, a first window management area is established on the head mounted device, where the first window management area is used to synchronize display content of a second window management area on the electronic device.

[0044] Among them, the head-mounted device can be smart glasses, smart helmets, etc., the smart glasses can be AR glasses, VR glasses, MR glasses, XR glasses, etc., and the smart helmet can be an AR helmet, etc.

[0045] Among them, the electronic device can be a desktop computer, a laptop computer, a mobile phone, a tablet computer, etc. The electronic device realizes content display (such as the second window management area on the electronic device, the target window content, etc.) through the main display screen (the main display screen refers to the main display screen of the electronic device by default), and the main display screen can be a display screen integrated in the electronic device, such as the display screen of a laptop computer, or an external display screen of the electronic device, such as the external display screen of a desktop computer. For the convenience of description, this article uses the tone of display on the electronic device. Unless otherwise specified, the display on the electronic device described below refers to the display on the main display screen of the electronic device.

[0046] Before the head-mounted device establishes the first window management area, it is necessary to establish a connection between the head-mounted device and the electronic device: in some embodiments, the head-mounted device and the electronic device can be connected by wire, for example, by a USB-C interface cable; in some embodiments, the head-mounted device and the electronic device can be connected by wireless, for example, by Wi-Fi Bluetooth and the like. Further, an automatic connection mechanism can be set, such as the electronic device can automatically detect the head-mounted device (such as AR glasses) nearby, and pop up a connection prompt to ask the user whether the head-mounted device is connected as an extended screen of the electronic device. This automatic connection mechanism can reduce the user's manual connection intervention; further, the icon in the status bar of the electronic device can also dynamically output a prompt message of successful connection, or output a prompt message of successful connection through a prompt tone (such as a brisk prompt tone of "connection successful"), so that the user's connection experience can be enhanced through visual and auditory feedback. Further, when connecting by wireless means, an automatic search function can be provided to achieve fast connection, such as the electronic device can automatically search for nearby Bluetooth devices, and the user can quickly complete the pairing by clicking the name of the head-mounted device on the electronic device, which simplifies the connection process between the head-mounted device and the electronic device to a certain extent. When the head-mounted device is successfully connected to the electronic device, the head-mounted device will be automatically set as the extended screen of the electronic device (the head-mounted device acts as an extended virtual display) to synchronize the display content of the electronic device.

[0047] The first window management area is a window management area created on the head mounted device, and the first window management area may occupy the entire display area of ​​the head mounted device, for example, Figure 3 As shown, the first window management area is presented in the entire visible area of ​​the head-mounted device.

[0048] The second window management area is a window management area created on the electronic device, and the second window management area occupies a part of the main display screen of the electronic device, for example, Figure 3 As shown, the second window management area is presented in the form of a small window.

[0049] When the connection between the head-mounted device and the electronic device is successful, a small movable and resizable window will be generated in a certain area on the main display screen of the electronic device as the second window management area, and a virtual window will also be generated on the head-mounted device as the first window management area.

[0050] 202. Receive target window content sent by the electronic device, wherein the target window content is window content in the second window management area.

[0051] The target window content may be an application window, a file, or a picture, etc., and the specific form of the target window content is not limited here.

[0052] On an electronic device, a user can drag target window content such as an application window, a file, or a picture to the second window management area. After the target window content is dragged to the second window management area, the main display screen of the electronic device can display the target window content in the second window management area; at the same time, the head-mounted device will also synchronously display the target window content in the first window management area. In order to enable the head-mounted device to synchronously display the target window content of the electronic device, when the user moves the target window content to the second window management area by dragging, the electronic device will send the target window content to the head-mounted device, and at this time, the head-mounted device can receive the target window content sent by the electronic device.

[0053] Furthermore, when the user drags target window content such as an application window, a file, or a picture, the target window content to be transmitted to the head-mounted device can be previewed on the electronic device to ensure the correctness of the transmitted content.

[0054] 203. Display the target window content in the first window management area.

[0055] In step 203, there are multiple ways to display the target window content in the first window management area, illustratively including the following ways ① and ②:

[0056] ① In some embodiments, the loading process of the target window content is displayed using a transition effect. Figure 4 As shown, step 203 may specifically include the following step 203A:

[0057] 203A. Display the target window content in the first window management area according to preset display transition parameters, wherein the preset display transition parameters include at least one of a parameter for controlling the content to be gradually enlarged and displayed, and a parameter for controlling the color to be gradually darkened and displayed.

[0058] On the electronic device, the user can drag the target window content such as application window, file, or picture to the second window management area. After dragging the target window content to the second window management area, on the one hand, the electronic device can display the target window content in the second window management area according to the preset display transition parameters, such as the color of the target window content gradually recovers from light to dark to the original color of the target window content, and the target window content gradually enlarges from small to flatten the entire second window management area; on the other hand, the electronic device will send the target window content to the head-mounted device so that the target window content can be synchronously displayed on the head-mounted device.

[0059] After the head-mounted device receives the target window content sent by the electronic device, it can display the target window content in the first window management area according to the preset display transition parameters. For example, the color of the target window content gradually changes from light to dark and then returns to the original color of the target window content. The target window content gradually increases from small to large until it covers the entire first window management area.

[0060] ② In some embodiments, the loading process of the target window content is displayed using the window adaptive layout effect. Figure 5 As shown, step 203 may specifically include the following step 203B:

[0061] 203B. Display the target window content in the first window management area according to a first window layout parameter corresponding to the content type of the target window content, wherein the window layout parameter includes at least one of a window size, a display ratio, and a window position.

[0062] The window layout parameters may include, but are not limited to, window size, display ratio, and window position. For example, a window layout parameter may be set for each content type. For example, if the window layout parameter is the window ratio, the application window may be set to a 16:9 window ratio, and the picture may be set to a 4:3 window ratio.

[0063] Among them, the first window layout parameter refers to the window layout parameter used on the head-mounted device, and the second window layout parameter refers to the window layout parameter used on the electronic device. The specific settings of the first window layout parameter and the second window layout parameter can be completely the same, partially the same, or completely different.

[0064] On the electronic device, the user can drag the target window content such as application window, file, or picture to the second window management area. After the target window content is dragged to the second window management area, on the one hand, the electronic device can display the target window content in the second window management area according to the second window layout parameters corresponding to the content type of the target window content, for example, according to the height ratio of 3:2 and the length ratio of 4:3 between the target window content and the second window management area, the target window content is displayed in the center of the second window management area; on the other hand, the electronic device will send the target window content to the head-mounted device so that the target window content can be synchronously displayed on the head-mounted device.

[0065] After the head-mounted device receives the target window content sent by the electronic device, it can display the target window content in the first window management area according to the first window layout parameters corresponding to the content type of the target window content. For example, the target window content is displayed on the left of the first window management area according to a height ratio of 3:2 and a length ratio of 4:3 between the target window content and the first window management area.

[0066] Furthermore, in some embodiments, after the target window content is loaded in the first window management area, the head-mounted device can also perform visual special effects prompts through the edge of the first window management area according to the first preset special effects parameters, wherein the first preset special effects parameters include at least one of animation parameters, light effect change parameters, and color change parameters. Similarly, the second window management area on the electronic device will also synchronize the visual special effects prompts. After the target window content is loaded in the second window management area, the electronic device can also perform visual special effects prompts through the edge of the second window management area according to the first preset special effects parameters; in this way, the dynamic loading of the target window content can be achieved, and the target window content can be automatically aligned to the small window edge of the second window management area through the intelligent "adsorption" function. In this way, the head-mounted device and the electronic device can perform visual special effects prompts synchronously after the target window content is loaded.

[0067] Furthermore, in order to improve the management efficiency of the window content displayed by the head mounted device and to improve the work efficiency of the user in the virtual environment, the extended screen display control method further includes the following step 204 (not shown in the drawings):

[0068] 204. Update display content of the first window management area based on the target operation data of the target window content.

[0069] The target operation data refers to data for operating the target window content, and may include at least one of the first operation data and the second operation data.

[0070] The first operation data is data of the operation on the target window content of the first window management area on the head mounted device. The operation on the target window content of the first window management area may be editing, enlarging, marking, and the like. In some embodiments, the input method keyboard may be popped up by clicking on the corresponding content of the first window management area on the head mounted device, or the input method keyboard may be popped up or hidden by sliding the touch panel of the head mounted device, thereby implementing the editing operation on the target window content.

[0071] The second operation data is data of the operation on the target window content of the second window management area on the electronic device. The operation on the target window content in the second window management area can be editing, enlarging, marking and the like.

[0072] There are many ways to implement step 204, illustratively including:

[0073] (1) In some embodiments, the target operation data is the first operation data. In this case, step 204 may specifically include the following step 2041A:

[0074] 2041A. When first operation data for operating the target window content on the head mounted device is detected, the display content of the first window management area is updated based on the first operation data.

[0075] On the head-mounted device, the user can operate the target window content of the first window management area, such as document editing, image magnification, document editing, etc. At this time, on the one hand, the head-mounted device will update the display content of the first window management area based on the first operation data of the operation on the target window content on the head-mounted device.

[0076] On the other hand, in order to synchronize the display content of the electronic device and the head-mounted device, when the first operation data of the target window content operation on the head-mounted device is detected, the head-mounted device will also synchronize the first operation data of the target window content to the electronic device, so that the electronic device can update the display content of the second window management area based on the first operation data. In this way, the head-mounted device can synchronize the first operation data of the target window content to the electronic device when the head-mounted device performs operations such as editing, enlarging, and annotating on the target window content, thereby ensuring the real-time interaction of window data between the head-mounted device and the electronic device.

[0077] Therefore, on the first aspect, after the head-mounted device is connected to the electronic device, a first window management area is established on the head-mounted device, and the first window management area is used to synchronize the display content of the second window management area on the electronic device. In this way, the head-mounted device is equivalent to being set as an extended screen of the electronic device; when the target window content of the electronic device is dragged to the second window management area created on the electronic device, the head-mounted device receives the target window content sent by the electronic device and displays the target window content in the first window management area. In this way, all target window content that needs to be shared to the head-mounted device for display can be quickly shared to the head-mounted device by dragging on the electronic device, thereby improving the synchronization efficiency of the target window content and realizing the connection between the electronic device and the head-mounted device. Multi-window interaction between wearable devices; secondly, the display content of the first window management area is updated by target operation data based on the target window content. Therefore, whether the target window content is modified and managed such as editing on the head-mounted device or on other electronic devices, the target window content displayed in the first window management area of ​​the head-mounted device can be updated synchronously in real time, without the need to synchronize the modified content to the head-mounted device for display after the modification is completed on the electronic device, so that the target window content can be managed synchronously on the electronic device or the head-mounted device, thereby improving the management efficiency of the target window content displayed based on the head-mounted device, and thereby improving the user's work efficiency in the virtual environment.

[0078] (2) In some embodiments, the target operation data is the second operation data. In this case, step 204 may specifically include the following step 2041B:

[0079] 2041B. When receiving second operation data of the target window content sent by the electronic device, updating the display content of the first window management area based on the second operation data, wherein the second operation data is data for operating the target window content on the electronic device.

[0080] On the electronic device, the user can operate the target window content in the second window management area, such as document editing, image magnification, document editing, etc. At this time, on the one hand, the electronic device will update the display content of the second window management area based on the second operation data of the operation on the target window content of the second window management area on the electronic device.

[0081] On the other hand, in order to synchronize the display content of the electronic device and the head-mounted device, when the second operation data of the target window content is detected on the electronic device, the electronic device will also synchronize the second operation data of the target window content to the head-mounted device, so that the head-mounted device can update the display content of the first window management area based on the second operation data. In this way, the electronic device can synchronize the second operation data of the target window content to the head-mounted device, and when the electronic device performs operations such as editing, enlarging, and annotating on the target window content, it can be synchronized to the head-mounted device, thereby ensuring the real-time interaction of window data between the head-mounted device and the electronic device.

[0082] Further, a toolbar may be set on the first window management area, and the toolbar on the first window management area may be displayed in a floating manner. In some embodiments, the method may further include: in response to an edge operation on the first window management area, displaying a toolbar on the first window management area, wherein the toolbar of the first window management area may include at least one of a window resize control, a window close control, a window minimize control, and an edit control, and the edge operation here refers to an operation on the edge of the first window management area (such as moving the cursor of the head-mounted device to the edge of the first window management area through gestures, interactive control devices, etc.). In this way, the toolbar of the first window management area can be hidden and displayed, and the user can wake up the toolbar of the first window management area through some operation, such as the toolbar can automatically pop up when the user operates the edge of the first window management area, so as to provide a portable toolbar function entrance. Therefore, when the user needs to perform operations such as adjusting the window size, closing the window, minimizing the window, editing the content, etc. through the head-mounted device, the cursor of the head-mounted device can be moved to the edge of the first window management area through gestures, interactive control devices, etc., thereby triggering the waking up of the toolbar of the first window management area, and realizing operations such as adjusting the window size, closing the window, minimizing the window, editing the content, etc. through the head-mounted device.

[0083] Similarly, a toolbar can also be set on the second window management area, and the toolbar on the second window management area can be displayed in a floating manner. In some embodiments, the method may further include: in response to an edge operation on the second window management area, displaying a toolbar on the second window management area, wherein the toolbar of the second window management area may include at least one of a window size adjustment control, a window closing control, a window minimization control, and an editing control, and the edge operation here is to operate on the edge of the second window management area (such as moving the cursor of the electronic device to the edge of the second window management area by a mouse). In this way, the toolbar of the second window management area can be hidden and displayed, and the user can wake up the toolbar of the second window management area through some operation, such as the toolbar can automatically pop up when the user operates the edge of the second window management area, so as to provide a portable toolbar function entrance. Therefore, when the user needs to perform operations such as adjusting the window size, closing the window, minimizing the window, editing the content, etc. through the electronic device, the cursor of the electronic device can be moved to the edge of the second window management area by a mouse, thereby triggering the toolbar of the second window management area to be awakened, and the operation of adjusting the window size, closing the window, minimizing the window, editing the content, etc. through the electronic device can be realized.

[0084] Furthermore, the size or position of the first window management area can also be adjusted. In some embodiments, the size or position of the first window management area can be adjusted through a head-mounted device. In this case, the method may further include: when first adjustment data for adjusting the first window management area on the head-mounted device is detected, the size or position of the first window management area is updated based on the first adjustment data. On the head-mounted device, the user can adjust the first window management area, such as adjusting the size or position of the window. In this case, the head-mounted device will update the size or position of the first window management area based on the first adjustment data for adjusting the first window management area on the head-mounted device; in this way, the size or position of the first window management area can be adjusted through the head-mounted device.

[0085] In some embodiments, the size or position of the first window management area can be adjusted by the electronic device, and the method may further include: when receiving the second adjustment data of the first window management area sent by the electronic device, updating the size or position of the first window management area based on the second adjustment data, wherein the second adjustment data is the data of the adjustment of the first window management area by the electronic device. On the electronic device, the user can adjust the first window management area, such as adjusting the size or position of the first window management area by dragging the mouse. At this time, the electronic device can send the second adjustment data of the first window management area adjusted on the electronic device to the head-mounted device, so that the head-mounted device updates the size or position of the first window management area based on the second adjustment data after receiving the second adjustment data of the first window management area sent by the electronic device; in this way, the size or position of the first window management area can be adjusted by the electronic device. The first window management area can be adjusted on the electronic device using a cursor control device (such as a mouse). During the process of adjusting the first window management area on the electronic device using the cursor control device, the cursor size of the cursor control device follows the size change of the first window management area to facilitate the adjustment of the first window management area.

[0086] Furthermore, on the head-mounted device, the first window management area or the display content of the first window management area can also be controlled based on gestures or voice. In some embodiments, in response to a gesture operation on the first window management area on the head-mounted device, the head-mounted device adjusts the position or size of the first window management area. In this way, the user can directly manipulate the first window management area through gestures or voice, thereby improving the management flexibility of the first window management area. In this way, by combining wireless connection, real-time synchronization of target window content and operation data, and gesture operation, seamless connection and multi-window interaction between the electronic device and the head-mounted device are achieved, allowing the user to flexibly manage multiple windows in a virtual environment.

[0087] In some embodiments, in response to a gesture operation on the head-mounted device on the display content of the first window management area, on the one hand, the head-mounted device updates the display content of the first window management area according to the content operation data corresponding to the gesture operation; on the other hand, the head-mounted device also synchronously sends the content operation data corresponding to the gesture operation to the electronic device, so that the electronic device can update the display content of the second window management area according to the content operation data corresponding to the gesture operation.

[0088] In some embodiments, in response to a voice command triggered on the head-mounted device to the first window management area, the position or size of the first window management area is adjusted. For example, using voice commands to "close the current window" or "enlarge the window to full screen" not only enhances the flexibility of operation, but also reduces the need to rely on physical input devices. In this way, by combining wireless connection, real-time synchronization of target window content and operation data, and voice commands, seamless connection and multi-window interaction between electronic devices and head-mounted devices are achieved, allowing users to flexibly manage multiple windows in a virtual environment.

[0089] In some embodiments, in response to a voice command triggered on the head-mounted device for the display content of the first window management area, on the one hand, the head-mounted device updates the display content of the first window management area according to the content operation data corresponding to the voice command; on the other hand, the head-mounted device also synchronously sends the content operation data corresponding to the voice command to the electronic device, so that the electronic device can update the display content of the second window management area according to the content operation data corresponding to the voice command.

[0090] Furthermore, a gesture feedback mechanism can be provided for the head mounted device: in some embodiments, when the gesture operation is successful, a visual special effect prompt is performed through the edge of the first window management area according to a second preset special effect parameter, wherein the second preset special effect parameter includes at least one of an animation parameter, a light effect change parameter, and a color change parameter. For example, when the gesture operation is successful, a light effect is emitted from the edge of the first window management area. In this way, a gesture feedback mechanism can be provided for the head mounted device, so that the user can perceive the feedback of the gesture operation.

[0091] In some embodiments, when the gesture operation is successful, the head-mounted device is controlled to provide a tactile prompt; for example, when the gesture operation is successful, the head-mounted device vibrates; in this way, the user can obtain a tactile prompt through the vibration feedback device, allowing the user to perceive the feedback of the gesture operation.

[0092] Furthermore, in some embodiments, users can also customize the gesture library to meet user needs. For example, users can bind commonly used operations to gestures designed by themselves, thereby improving the personalization and efficiency of operations.

[0093] Furthermore, multiple target window contents may be added to the first window management area, and the user may also switch the current display content of the first window management area to different target window contents through gesture operations, thereby improving the interactive convenience of the target window contents.

[0094] Furthermore, the user can also quickly perform multiple operations through continuous gesture operations, such as closing multiple target window content windows in sequence and continuously adjusting the sizes of multiple target window content windows, so that the gesture operation is enhanced in function.

[0095] From the above content, it can be seen that, on the first hand, after the head-mounted device is connected to the electronic device, a first window management area is established on the head-mounted device, and the first window management area is used to synchronize the display content of the second window management area on the electronic device, so that the head-mounted device is equivalent to being set as an extended screen of the electronic device; the head-mounted device receives the target window content sent by the electronic device, and displays the target window content in the first window management area, so as to realize the multi-screen display of the electronic device, so that the electronic device can realize multi-screen display without adding a display screen, thereby reducing the multi-screen display cost of the electronic device. On the second hand, since the content is sent to the head-mounted device only when it is in the second window management area, it is convenient to manage the content synchronization management when the head-mounted device is used as an extended screen, so that all the target window content that needs to be shared to the head-mounted device on the electronic device can be quickly shared to the head-mounted device by adding the second management area, thereby improving the synchronization efficiency of the target window content. .

[0096] The following describes the extended screen display control method applied to electronic devices. Figure 6 , Figure 6 601 to 603, wherein:

[0097] 601. After a head mounted device is connected to an electronic device, a second window management area is established on the electronic device, and display content of the second window management area is synchronized with display content of a first window management area on the head mounted device.

[0098] Before the electronic device establishes the second window management area, it is necessary to first establish a connection between the head mounted device and the electronic device. The connection method between the head mounted device and the electronic device can refer to the relevant description of step 201, which will not be repeated here.

[0099] When the connection between the head-mounted device and the electronic device is successful, a small movable and resizable window will be generated in a certain area on the main display screen of the electronic device as the second window management area, and a virtual window will also be generated on the head-mounted device as the first window management area.

[0100] 602. In response to a drag operation on a target window content of the electronic device, move the target window content to the second window management area.

[0101] The target window content may be an application window, a file, or a picture, etc., and the specific form of the target window content is not limited here.

[0102] On an electronic device, a user can drag target window content such as an application window, a file, or a picture to the second window management area. After the target window content is dragged to the second window management area, the main display screen of the electronic device can display the target window content in the second window management area; at the same time, the head-mounted device will also synchronously display the target window content in the first window management area. In order to enable the head-mounted device to synchronously display the target window content of the electronic device, when the user moves the target window content to the second window management area by dragging, the electronic device will send the target window content to the head-mounted device, and at this time, the head-mounted device can receive the target window content sent by the electronic device.

[0103] Furthermore, when the user drags target window content such as an application window, a file, or a picture, the target window content to be transmitted to the head-mounted device can be previewed on the electronic device to ensure the correctness of the transmitted content.

[0104] 603. After the target window content is moved to the second window management area, the target window content is sent to the head mounted device, and the target window content is loaded into the second window management area for display.

[0105] On an electronic device, a user can drag target window content such as an application window, a file, or a picture to the second window management area. After the target window content is dragged to the second window management area, the main display screen of the electronic device can display the target window content in the second window management area; at the same time, the head-mounted device will also synchronously display the target window content in the first window management area. In order to enable the head-mounted device to synchronously display the target window content of the electronic device, when the user moves the target window content to the second window management area by dragging, the electronic device will send the target window content to the head-mounted device, and at this time, the head-mounted device can receive the target window content sent by the electronic device.

[0106] For the specific implementation of "the head mounted device displays the target window content in the first window management area", reference may be made to the relevant description in the previous text (such as step 203), which will not be repeated here.

[0107] There are many implementation methods for "loading the target window content into the second window management area for display" in step 603, and exemplary methods include:

[0108] (1) In some embodiments, the loading process of the target window content is displayed using a transition effect. In this case, step 603 may specifically include the following step 603A:

[0109] 603A. After the target window content moves to the second window management area, the target window content is sent to the head mounted device, and the target window content is displayed in the second window management area according to preset display transition parameters, wherein the preset display transition parameters include at least one of parameters for controlling the gradual enlargement of the content and parameters for controlling the gradual darkening of the color.

[0110] On the electronic device, the user can drag the target window content such as application window, file, or picture to the second window management area. After dragging the target window content to the second window management area, on the one hand, the electronic device can display the target window content in the second window management area according to the preset display transition parameters, such as the color of the target window content gradually recovers from light to dark to the original color of the target window content, and the target window content gradually enlarges from small to flatten the entire second window management area; on the other hand, the electronic device will send the target window content to the head-mounted device so that the target window content can be synchronously displayed on the head-mounted device.

[0111] (2) In some embodiments, the loading process of the target window content is displayed using a transition effect. In this case, step 603 may specifically include the following step 603B:

[0112] 603B. After the target window content moves to the second window management area, the target window content is sent to the head mounted device, and the target window content is displayed in the second window management area according to the second window layout parameters corresponding to the content type of the target window content, wherein the window layout parameters include at least one of the window size, display ratio, and window position.

[0113] On the electronic device, the user can drag the target window content such as application window, file, or picture to the second window management area. After the target window content is dragged to the second window management area, on the one hand, the electronic device can display the target window content in the second window management area according to the second window layout parameters corresponding to the content type of the target window content, for example, according to the height ratio of 3:2 and the length ratio of 4:3 between the target window content and the second window management area, the target window content is displayed in the center of the second window management area; on the other hand, the electronic device will send the target window content to the head-mounted device so that the target window content can be synchronously displayed on the head-mounted device.

[0114] Furthermore, in order to improve the management efficiency of the window content, the method further includes step 604 (not shown in the drawings):

[0115] 604. Update the display content of the second window management area based on the target operation data of the target window content.

[0116] There are many ways to implement step 604, illustratively including:

[0117] (1) In some embodiments, the target operation data is the first operation data. In this case, step 604 may specifically include the following step 6041A:

[0118] 6041A. When second operation data for operating the target window content on the electronic device is detected, the display content of the second window management area is updated based on the second operation data.

[0119] When second operation data for operating the target window content on the electronic device is detected, the second operation data will also be sent to the head-mounted device so that the head-mounted device can update the display content of the first window management area based on the second operation data.

[0120] On the electronic device, the user can operate the target window content in the second window management area, such as performing document editing, image magnification, document editing, etc. At this time, on the one hand, the electronic device will update the display content of the second window management area based on the second operation data of the operation on the target window content of the second window management area on the electronic device. On the other hand, in order to synchronize the display content of the electronic device and the head-mounted device, after detecting the second operation data of the operation on the target window content on the electronic device, the electronic device will also synchronize the second operation data of the target window content to the head-mounted device, so that the head-mounted device can update the display content of the first window management area based on the second operation data. In this way, the electronic device can synchronize the second operation data of the target window content to the head-mounted device when the electronic device performs operations such as editing, magnification, and annotation on the target window content, thereby ensuring the real-time interaction of window data between the head-mounted device and the electronic device.

[0121] (2) In some embodiments, the target operation data is the second operation data. In this case, step 604 may specifically include the following step 6041B:

[0122] 6041B. When first operation data of the target window content sent by the head-mounted device is received, the display content of the second window management area is updated based on the first operation data, wherein the first operation data is data of operating the target window content on the head-mounted device.

[0123] Furthermore, after the loading of the target window content is completed in the second window management area, a visual special effect prompt is performed through the edge of the first window management area according to the first preset special effect parameter, wherein the first preset special effect parameter includes at least one of animation parameters, light effect change parameters, and color change parameters; in this way, the dynamic loading of the target window content can be achieved, and the target window content can be automatically aligned to the edge of the small window in the second window management area through the intelligent "adsorption" function.

[0124] Further, a toolbar may also be set on the second window management area, and the toolbar on the second window management area may be displayed in a floating manner. In some embodiments, the method may further include: in response to an edge operation on the second window management area, displaying a toolbar on the second window management area, wherein the toolbar of the second window management area may include at least one of a window resize control, a window close control, a window minimize control, and an edit control, and the edge operation here is an operation performed on the edge of the second window management area (such as moving the cursor of the electronic device to the edge of the second window management area by a mouse). In this way, the toolbar of the second window management area can be hidden and displayed, and the user can wake up the toolbar of the second window management area through some operation, such as the toolbar can automatically pop up when the user operates the edge of the second window management area, so as to provide a portable toolbar function entrance. Therefore, when the user needs to perform operations such as adjusting the window size, closing the window, minimizing the window, editing the content, etc. through the electronic device, the cursor of the electronic device can be moved to the edge of the second window management area by a mouse, thereby triggering the waking up of the toolbar of the second window management area, and realizing the operations such as adjusting the window size, closing the window, minimizing the window, editing the content, etc. through the electronic device.

[0125] Furthermore, the method further includes: when detecting the second adjustment data for adjusting the first window management area on the electronic device, sending the second adjustment data to the electronic device. On the electronic device, the user can adjust the first window management area, such as adjusting the size or position of the first window management area by dragging the mouse. At this time, the electronic device can send the second adjustment data for adjusting the first window management area on the electronic device to the head-mounted device, so that after receiving the second adjustment data of the first window management area sent by the electronic device, the head-mounted device updates the size or position of the first window management area based on the second adjustment data; in this way, the size or position of the first window management area can be adjusted by the electronic device. The first window management area can be adjusted on the electronic device using a cursor control device (such as a mouse). During the process of adjusting the first window management area on the electronic device using the cursor control device, the cursor size of the cursor control device follows the size change of the first window management area to facilitate the adjustment of the first window management area.

[0126] For the sake of simplicity, this text focuses on the description of each embodiment. For the specific implementation process and beneficial effects of the extended screen display control method applied to an electronic device, please refer to the relevant description of the extended screen display control method applied to a head-mounted device for the parts not described in detail.

[0127] In addition, in order to better implement the extended screen display control method in the embodiment of the present application, based on the extended screen display control method, an extended screen display control device is also provided in the embodiment of the present application, such as Figure 6 FIG. 1 is a schematic diagram of a structure of an embodiment of an extended screen display control device provided in an embodiment of the present application. The extended screen display control device 700 includes:

[0128] A creating unit 701 is used to establish a first window management area on the head mounted device after the head mounted device is connected to the electronic device, wherein the first window management area is used to synchronize display content of a second window management area on the electronic device;

[0129] The receiving unit 702 is configured to receive target window content sent by the electronic device, wherein the target window content is the window content in the second window management area;

[0130] The control unit 703 is configured to display the target window content in the first window management area.

[0131] In some embodiments, the control unit 703 is used to:

[0132] When first operation data for operating the target window content on the head mounted device is detected, based on the first operation data, the display content of the first window management area is updated;

[0133] And / or, when second operation data of the target window content sent by the electronic device is received, the display content of the first window management area is updated based on the second operation data, wherein the second operation data is data for operating the target window content on the electronic device.

[0134] In some embodiments, the control unit 703 is used to:

[0135] The target window content is displayed in the first window management area according to preset display transition parameters, wherein the preset display transition parameters include at least one of a parameter for controlling the content to be gradually enlarged and displayed, and a parameter for controlling the color to be gradually darkened and displayed.

[0136] In some embodiments, the control unit 703 is used to:

[0137] After the loading of the target window content is completed in the first window management area, a visual special effect prompt is performed through the edge of the first window management area according to the first preset special effect parameter, wherein the first preset special effect parameter includes at least one of animation parameters, light effect change parameters, and color change parameters.

[0138] In some embodiments, the control unit 703 is used to:

[0139] The target window content is displayed in the first window management area according to a first window layout parameter corresponding to the content type of the target window content, wherein the window layout parameter includes at least one of a window size, a display ratio, and a window position.

[0140] In some embodiments, the control unit 703 is used to:

[0141] In response to an operation on the edge of the first window management area, a toolbar is displayed in the first window management area, wherein the toolbar includes at least one of a window resize control, a window close control, a window minimize control, and an edit control.

[0142] In some embodiments, the control unit 703 is used to:

[0143] When first adjustment data for adjusting the first window management area on the head mounted device is detected, the size or position of the first window management area is updated based on the first adjustment data;

[0144] And / or, when second adjustment data of the first window management area sent by the electronic device is received, the size or position of the first window management area is updated based on the second adjustment data, wherein the second adjustment data is data for adjusting the first window management area on the electronic device.

[0145] In some embodiments, the control unit 703 is used to:

[0146] In response to a gesture operation on the head mounted device with respect to the first window management area, adjusting a position or a size of the first window management area;

[0147] and / or, in response to a gesture operation on the head mounted device with respect to the display content of the first window management area, updating the display content of the first window management area;

[0148] and / or, in response to a voice command triggered on the head mounted device to the first window management area, adjusting the position or size of the first window management area;

[0149] And / or, in response to a voice command triggered on the head mounted device for display content of the first window management area, updating the display content of the first window management area.

[0150] In some embodiments, the control unit 703 is used to:

[0151] When the gesture operation is successful, a visual special effect prompt is performed through the edge of the first window management area according to a second preset special effect parameter, wherein the second preset special effect parameter includes at least one of an animation parameter, a light effect change parameter, and a color change parameter;

[0152] Alternatively, when the gesture operation is successful, the head mounted device is controlled to provide a tactile prompt.

[0153] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous extended screen display control method embodiment, which will not be repeated here.

[0154] A person skilled in the art will appreciate that all or part of the steps in the above-mentioned extended screen display control method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0155] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any extended screen display control method provided in the embodiment of the present application.

[0156] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0157] In the above-mentioned embodiments of the extended screen display control device, computer-readable storage medium, and head-mounted device, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-described extended screen display control device, computer-readable storage medium, head-mounted device and its corresponding units can refer to the description of the extended screen display control method in the above embodiment, and will not be repeated here.

[0158] The above is a detailed introduction to an extended screen display control method, device, head-mounted device and computer-readable storage medium provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An extended screen display control method, characterized in that: The extended screen display control method is applied to a head mounted device, and the method comprises: After the head mounted device is connected to the electronic device, a first window management area is established on the head mounted device, where the first window management area is used to synchronize display content of a second window management area on the electronic device; Receiving target window content sent by the electronic device, wherein the target window content is window content in the second window management area; The target window content is displayed in the first window management area.

2. The extended screen display control method according to claim 1, characterized in that: The method further comprises: When first operation data for operating the target window content on the head mounted device is detected, based on the first operation data, the display content of the first window management area is updated; And / or, when second operation data of the target window content sent by the electronic device is received, the display content of the first window management area is updated based on the second operation data, wherein the second operation data is data for operating the target window content on the electronic device.

3. The extended screen display control method according to claim 1, characterized in that: The displaying the target window content in the first window management area includes: The target window content is displayed in the first window management area according to preset display transition parameters, wherein the preset display transition parameters include at least one of a parameter for controlling the content to be gradually enlarged and displayed, and a parameter for controlling the color to be gradually darkened and displayed.

4. The extended screen display control method according to claim 3, characterized in that: The method further comprises: After the loading of the target window content is completed in the first window management area, a visual special effect prompt is performed through the edge of the first window management area according to the first preset special effect parameter, wherein the first preset special effect parameter includes at least one of animation parameters, light effect change parameters, and color change parameters.

5. The extended screen display control method according to claim 1, characterized in that: The displaying the target window content in the first window management area includes: The target window content is displayed in the first window management area according to a first window layout parameter corresponding to the content type of the target window content, wherein the window layout parameter includes at least one of a window size, a display ratio, and a window position.

6. The extended screen display control method according to claim 1, characterized in that: The method further comprises: In response to an operation on the edge of the first window management area, a toolbar is displayed in the first window management area, wherein the toolbar includes at least one of a window resize control, a window close control, a window minimize control, and an edit control.

7. The extended screen display control method according to claim 1, characterized in that: The method further comprises: When first adjustment data for adjusting the first window management area on the head mounted device is detected, the size or position of the first window management area is updated based on the first adjustment data; And / or, when second adjustment data of the first window management area sent by the electronic device is received, the size or position of the first window management area is updated based on the second adjustment data, wherein the second adjustment data is data for adjusting the first window management area on the electronic device.

8. The extended screen display control method according to claim 1, characterized in that: The method further comprises: In response to a gesture operation on the head mounted device with respect to the first window management area, adjusting a position or a size of the first window management area; and / or, in response to a gesture operation on the head mounted device with respect to the display content of the first window management area, updating the display content of the first window management area; and / or, in response to a voice command triggered on the head mounted device to the first window management area, adjusting the position or size of the first window management area; And / or, in response to a voice command triggered on the head mounted device for display content of the first window management area, updating the display content of the first window management area.

9. The extended screen display control method according to claim 8, characterized in that: The method further comprises: When the gesture operation is successful, a visual special effect prompt is performed through the edge of the first window management area according to a second preset special effect parameter, wherein the second preset special effect parameter includes at least one of an animation parameter, a light effect change parameter, and a color change parameter; Alternatively, when the gesture operation is successful, the head mounted device is controlled to provide a tactile prompt.

10. An extended screen display control method, characterized in that: The extended screen display control method is applied to an electronic device, and the method comprises: After the head mounted device is connected to the electronic device, a second window management area is established on the electronic device, and the display content of the second window management area is synchronized with the display content of the first window management area on the head mounted device; In response to a drag operation on a target window content of the electronic device, moving the target window content to the second window management area; When the target window content is moved to the second window management area, the target window content is sent to the head mounted device, and the target window content is loaded into the second window management area for display.

11. The extended screen display control method according to claim 10, characterized in that: The method further comprises: When second operation data for operating the target window content on the electronic device is detected, based on the second operation data, the display content of the second window management area is updated, and the second operation data is sent to the head mounted device; or When second adjustment data for adjusting the first window management area on the electronic device is detected, the second adjustment data is sent to the electronic device.

12. The extended screen display control method according to claim 10, characterized in that: When the target window content is moved to the second window management area, sending the target window content to the head mounted device, and loading the target window content into the second window management area for display, comprises: When the target window content moves to the second window management area, the target window content is sent to the head mounted device, and the target window content is displayed in the second window management area according to preset display transition parameters, wherein the preset display transition parameters include at least one of a parameter for controlling the content to be gradually enlarged and displayed, and a parameter for controlling the color to be gradually darkened and displayed; or When the target window content moves to the second window management area, the target window content is sent to the head-mounted device, and the target window content is displayed in the second window management area according to the second window layout parameters corresponding to the content type of the target window content, wherein the window layout parameters include at least one of the window size, display ratio, and window position.

13. The extended screen display control method according to claim 10, characterized in that: The method further comprises: After the target window content is loaded in the second window management area, a visual special effect prompt is performed through the edge of the first window management area according to a first preset special effect parameter, wherein the first preset special effect parameter includes at least one of an animation parameter, a light effect change parameter, and a color change parameter; or In response to an operation on the edge of the second window management area, a toolbar is displayed in the second window management area, wherein the toolbar includes at least one of a window resize control, a window close control, a window minimize control, and an edit control.

14. An extended screen display control device, characterized in that: The extended screen display control device comprises: A creating unit, configured to establish a first window management area on the head mounted device after the head mounted device is connected to the electronic device, wherein the first window management area is used to synchronize display content of a second window management area on the electronic device; a receiving unit, configured to receive target window content sent by the electronic device, wherein the target window content is window content in the second window management area; A control unit is used to display the target window content in the first window management area.

15. A head mounted device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the extended screen display control method according to any one of claims 1 to 13 is executed.

16. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the extended screen display control method according to any one of claims 1 to 13.

Images