Mixed reality window adjustment method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a mixed reality window adjustment method and device, equipment and a storage medium. According to the technical scheme provided by the embodiment of the invention, when the window activation operation triggered by the touch unit is received, the target window in the virtual environment is determined, the window adjustment is performed on the target window according to the received window adjustment information, and when the window fixing operation triggered by the touch unit is received, the window adjustment is performed on the target window. The target window is fixed in the virtual environment, the adjustment of the window in the virtual environment is realized, the adjustment operation of the window does not need to depend on control equipment, and the flexibility of the adjustment operation of the window is effectively improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the technical field of virtual display, and in particular, to a method, device, equipment and storage medium for adjusting a mixed reality window. Background Art

[0002] With the development of computer technology and virtual display technologies such as virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR), virtual display devices are increasingly widely used.

[0003] A virtual display device can display a display screen corresponding to a virtual environment. For example, when displaying a window in a virtual reality environment, by rotating the virtual display device to a position corresponding to the window, the window can be observed in the display screen. When a user needs to adjust the window, it is generally necessary to use a control device such as a remote control. The window adjustment operation depends on the control device, and the flexibility of the window adjustment operation is relatively low. Summary of the Invention

[0004] Embodiments of the present application provide a method, device, equipment and storage medium for adjusting a mixed reality window, so as to solve the technical problem in the related art that the window adjustment operation depends on a control device and the flexibility of the window adjustment operation is relatively low, and effectively improve the flexibility of the window adjustment operation.

[0005] In a first aspect, embodiments of the present application provide a method for adjusting a mixed reality window, which is applied to a virtual display device. The virtual display device is provided with a touch unit. The method for adjusting a mixed reality window includes:

[0006] When receiving a window activation operation triggered by the touch unit, determining a target window in the virtual environment;

[0007] Receiving window adjustment information, and adjusting the target window according to the window adjustment information. The window adjustment includes one or a combination of more of window position adjustment, window size adjustment, and window distance adjustment;

[0008] When receiving a window fixing operation triggered by the touch unit, fixing the target window in the virtual environment.

[0009] In an embodiment of the present application, when a window activation operation triggered by a touch unit is received, a target window in a virtual environment is determined, the target window is adjusted according to the received window adjustment information, and when a window fixing operation triggered by the touch unit is received, the target window is fixed in the virtual environment, so as to realize the adjustment of the window in the virtual environment. The window adjustment operation does not need to rely on a control device, effectively improving the flexibility of the window adjustment operation.

[0010] Further, when receiving the window activation operation triggered by the touch unit, determining the target window in the virtual environment includes:

[0011] When receiving the window activation operation triggered by the touch unit, determining the orientation information of the virtual display device, where the orientation information is detected by an orientation detection unit on the virtual display device;

[0012] Determining the target window in the virtual environment according to the orientation information.

[0013] As described above, the target window in the virtual environment is quickly determined through the orientation information, improving the determination efficiency and accuracy of the target window and the window adjustment efficiency.

[0014] Further, after determining the orientation information of the virtual display device when receiving the window activation operation triggered by the touch unit, it further includes:

[0015] Displaying a positioning identifier in the display screen of the virtual display device.

[0016] As described above, the orientation direction corresponding to the orientation information is indicated through the positioning identifier, facilitating the user to quickly determine the target window, and the user can determine the corresponding target window according to the synchronously moving positioning identifier by moving the perspective, making the window adjustment operation more flexible.

[0017] Further, when receiving the window activation operation triggered by the touch unit, determining the target window in the virtual environment includes:

[0018] When receiving the window activation operation triggered by the touch unit, determining candidate windows in the virtual environment;

[0019] When receiving a window switching operation, switching the candidate windows according to the window switching operation;

[0020] When it is determined that the window selection condition is met, determining the current candidate window as the target window.

[0021] As described above, by first determining the candidate window during the receiving window switching operation, the user can switch the candidate window and, when it is determined that the window selection condition is met, determine the current candidate window as the target window, quickly determining the window that the user wishes to adjust, and making the window adjustment operation more flexible.

[0022] Further, the method adjusts the target window according to the received window adjustment information, including at least one of the following:

[0023] Receiving rotation information through the orientation detection unit on the virtual display device and adjusting the window position of the target window according to the rotation information;

[0024] Receiving first touch information in a first direction through the touch unit and adjusting the window size of the target window according to the first touch information;

[0025] Receiving second touch information in a second direction through the touch unit and adjusting the window distance of the target window according to the second touch information.

[0026] As described above, according to at least one of the rotation information, the first touch information, and the second touch information, at least one of the window position adjustment, window size adjustment, and window distance adjustment of the target window is realized, improving the accuracy and efficiency of window adjustment.

[0027] Further, after determining the target window in the virtual environment when receiving the window activation operation triggered by the touch unit, it further includes:

[0028] Moving the target window to the middle position of the display screen of the virtual display device.

[0029] As described above, by moving the target window to the middle of the screen, when the user moves the position of the target window, the target window is always located in the center of the screen, and the user's adjustment of the target window is more in line with the visual experience, and the window adjustment of the target window is more efficient, improving the user experience.

[0030] Further, after moving the target window to the middle position of the display screen of the virtual display device, it further includes:

[0031] As described above, by moving the target window to the middle of the screen and making the target window perpendicular to the line of sight direction of the virtual display device, when the user moves the position of the target window, the target window is always located in the center of the screen and faces the user directly, and the user's adjustment of the target window is more in line with the visual experience, and the window adjustment of the target window is more efficient, improving the user experience.

[0032] In a second aspect, an embodiment of the present application provides a mixed reality window adjustment device, which is applied to a virtual display device. The virtual display device is provided with a touch unit. The mixed reality window adjustment device includes an adjustment response module, a window adjustment module, and a window fixing module, where:

[0033] The adjustment response module is configured to determine a target window in the virtual environment when receiving a window activation operation triggered by the touch unit.

[0034] The window adjustment module is configured to receive window adjustment information and perform window adjustment on the target window according to the window adjustment information. The window adjustment includes one or a combination of more than one of window position adjustment, window size adjustment, and window distance adjustment.

[0035] The window fixing module is configured to fix the target window in the virtual environment when receiving a window fixing operation triggered by the touch unit.

[0036] When the embodiment of the present application receives a window activation operation triggered by the touch unit, it determines a target window in the virtual environment, performs window adjustment on the target window according to the received window adjustment information, and when receiving a window fixing operation triggered by the touch unit, fixes the target window in the virtual environment, so as to realize the adjustment of the window in the virtual environment. The adjustment operation of the window does not need to rely on a control device, effectively improving the flexibility of the window adjustment operation.

[0037] In a third aspect, an embodiment of the present application provides a mixed reality window adjustment device, including: a memory and one or more processors;

[0038] The memory is used to store one or more programs;

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement the mixed reality window adjustment method as described in the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the mixed reality window adjustment method as described in the first aspect when executed by a computer processor.

[0041] The above-provided mixed reality window adjustment device, device, and computer can be used to execute the mixed reality window adjustment method provided in any of the above embodiments, and have corresponding functions and beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1It is a flowchart of a method for adjusting a mixed reality window provided by an embodiment of the present application;

[0043] Figure 2 It is a schematic block diagram of a virtual display device provided by an embodiment of the present application;

[0044] Figure 3 It is a schematic diagram of the display of an application window in a virtual environment provided by an embodiment of the present application;

[0045] Figure 4 It is a flowchart of another method for adjusting a mixed reality window provided by an embodiment of the present application;

[0046] Figure 5 It is a schematic diagram of the adjusted display of the window position provided by an embodiment of the present application;

[0047] Figure 6 It is a schematic diagram of the adjusted display of the window size provided by an embodiment of the present application;

[0048] Figure 7 It is a schematic diagram of the adjusted display of the window distance provided by an embodiment of the present application;

[0049] Figure 8 It is a schematic structural diagram of a device for adjusting a mixed reality window provided by an embodiment of the present application;

[0050] Figure 9 It is a schematic structural diagram of a device for adjusting a mixed reality window provided by an embodiment of the present application. Detailed implementation manners

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the following further describes in detail specific embodiments of the present application with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, rather than limiting the present application. Additionally, it should be noted that for the sake of convenience of description, only parts related to the present application are shown in the accompanying drawings rather than all the content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the above-mentioned process can be terminated, but there may also be additional steps not included in the drawings. The above-mentioned process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0052] In existing mixed reality window adjustment solutions, window adjustment is generally performed through control devices such as mobile phones or dedicated controllers, or through gesture operations. Hardware-wise, the virtual display device needs to be configured with a gesture detection device or a control device to enable window adjustment. The window adjustment operation relies on the control device, resulting in relatively low flexibility of the window adjustment operation. Also, limited by the capture range of the gesture detection device (camera), when the user's hand is outside the camera's view, the user's hand cannot be located, leading to failure of the window adjustment operation. When using the camera to capture gestures, the camera needs to be constantly turned on, increasing the power consumption of the device. Moreover, limited by the computing power of the virtual display device, the configured gesture algorithm has low accuracy, and gesture interaction is prone to errors, resulting in poor window adjustment effects.

[0053] Based on this, an embodiment of a mixed reality window adjustment method of the present application is provided, aiming to trigger a window activation operation through a touch unit, and perform window adjustment on a target window according to the received window adjustment information. The window adjustment operation does not need to rely on a control device and a gesture detection device, effectively solving the technical problems in the prior art that the window adjustment operation relies on a control device, the flexibility of the window adjustment operation is relatively low, the power consumption of gesture operation for window adjustment is large, and the window adjustment effect is poor. It improves the flexibility of the window adjustment operation and the window adjustment effect, and reduces the power consumption of the virtual display device.

[0054] Figure 1 The flowchart of a mixed reality window adjustment method provided by an embodiment of the present application is given. The mixed reality window adjustment method provided by an embodiment of the present application can be executed by a mixed reality window adjustment device, and the mixed reality window adjustment device can be implemented in a hardware and / or software manner and integrated in a mixed reality window adjustment device.

[0055] The following describes the mixed reality window adjustment method executed by the mixed reality window adjustment device as an example.

[0056] Refer to Figure 1 , the mixed reality window adjustment method includes:

[0057] S110: When receiving a window activation operation triggered by a touch unit, determine a target window in the virtual environment.

[0058] As Figure 2 shown in the principle block diagram of a virtual display device provided, the virtual display device provided in this solution includes a touch unit, an orientation detection unit, a processing unit, and a display unit, where the touch unit, the orientation detection unit, and the display unit are all connected to the processing unit.

[0059] In one embodiment, the processing unit may execute the mixed reality window adjustment method provided by this solution. The processing unit may control the screen displayed by the display unit. The display unit is installed inside the virtual display device. After the user wears the virtual display device, the user can observe the corresponding screen through the display unit. The processing unit may determine the corresponding camera pose information based on the virtual environment, and render and display the corresponding screen on the display unit based on the camera pose information. Optionally, in the default state, the virtual display device is the origin of the virtual environment.

[0060] The touch unit can be used to receive touch operations (such as clicking and swiping on the touch unit). The user can issue relevant touch operations through the touch unit. The window activation operation provided by this solution can be triggered by the touch operation on the touch unit. In one embodiment, the touch unit provided by this solution can be a touchpad. The touchpad can detect click touch operations (including short presses and long presses), swipe operations in the first direction (such as the Y-axis of the touchpad), and swipe operations in the second direction (such as the X-axis of the touchpad). For example, if the virtual display device is virtual display glasses, the touch unit can be configured on the temple of the virtual display glasses. Among them, the length direction of the temple is the second direction (parallel to the forward gaze direction, that is, relative to the front-back direction of the virtual display glasses), and the direction perpendicular to the second direction is the first direction (that is, relative to the up-down direction of the virtual display glasses).

[0061] The orientation detection unit can be used to detect the orientation information (such as the orientation of the virtual display device in the virtual environment) and rotation information (such as the rotation direction, angular velocity, angle, etc.) of the virtual display device. The processing unit can adjust the camera pose information in the virtual environment according to the rotation information, and render and display the corresponding screen on the display unit in real time based on the adjusted pose information, so as to realize the synchronous rotation of the display screen and the virtual display device. Optionally, the orientation detection unit provided by this solution can be an inertial measurement unit (IMU unit). The virtual display device provided by this solution realizes the fast movement and scaling of the window in the virtual environment through the touch unit and the orientation detection unit, providing a stable multi-modal interaction experience for users.

[0062] As Figure 3 As shown in the schematic diagram of the application window display in a virtual environment provided, when opening relevant applications such as models, application panels, photos, videos, and documents in the virtual display device, corresponding application windows will be opened in the virtual environment of the virtual display device. Correspondingly, when opening multiple applications, corresponding multiple application windows will be opened in the virtual environment of the virtual display device.

[0063] When the user needs to adjust the application window, the window activation operation can be triggered through the touch unit. Among them, the window activation operation can be a click operation, a long - press operation, a double - click operation, or other set touch operations on the touch unit.

[0064] Exemplarily, when receiving the window activation operation triggered by the touch unit, determine the target window in the virtual environment in response to the window activation operation. Among them, the target window can be understood as the application window that needs to be adjusted in the virtual environment. Optionally, when there is only one application window or one application window is expanded in the virtual environment, the application window can be directly determined as the target window. When there are multiple application windows or multiple application windows are expanded in the virtual environment, the determination of the target window can be the latest opened application window, the application window located on the top layer, the application window closest to the origin of the virtual environment, or the application window closest to the center of the current display screen.

[0065] Optionally, when determining the target window, a set activation light effect can be rendered for the target window to remind the user that the target window is in an active state and the window adjustment operation can be performed on the target window. Optionally, the set activation light effect rendered for the target window can be highlighting the target window, stroke - displaying the target window, etc.

[0066] S120: Receive window adjustment information, and perform window adjustment on the target window according to the window adjustment information. The window adjustment includes one or a combination of more of window position adjustment, window size adjustment, and window distance adjustment.

[0067] Exemplarily, receive window adjustment information indicating the window adjustment operation on the target window, and perform window adjustment on the target window according to the window adjustment information. The window adjustment provided by this solution includes one or a combination of more of window position adjustment, window size adjustment, and window distance adjustment.

[0068] Among them, window position adjustment can be understood as adjusting the position of the window in the virtual environment (for example, adjusting the left - right and up - down movement of the window). When performing window position adjustment, the picture observed by the user is the left - right and up - down movement of the target window in the virtual environment. Window size adjustment can be understood as adjusting the size of the window in the virtual environment. When performing window size adjustment, the picture observed by the user is the enlargement and reduction of the target window in the virtual environment. Window distance adjustment can be understood as adjusting the distance of the window from the origin in the virtual environment. When performing window distance adjustment, the picture observed by the user is the approach and departure of the target window in the virtual environment (window distance adjustment will not change the size of the target window in the virtual environment).

[0069] S130: In the case of receiving the window fixation operation triggered by the touch unit, fix the target window in the virtual environment.

[0070] When the user finishes adjusting the target window, the window fixing operation can be triggered through the touch unit. Among them, the window fixing operation can be releasing the touch unit, clicking, long - pressing, double - clicking or other set touch operations on the touch unit.

[0071] Exemplarily, when receiving the window fixing operation triggered through the touch unit, the target window is fixed in the virtual environment in response to the window fixing operation, that is, the window position, window size and window distance of the target window in the virtual environment are fixed. Optionally, when fixing the target window in the virtual environment, the rendering setting activation light effect for the target window can be cancelled to remind the user that the target window has exited the activated state and the window adjustment operation has been completed.

[0072] As described above, when receiving the window activation operation triggered through the touch unit, the target window in the virtual environment is determined, the target window is adjusted according to the received window adjustment information, and when receiving the window fixing operation triggered through the touch unit, the target window is fixed in the virtual environment, so as to realize the adjustment of the window in the virtual environment. The window adjustment operation does not need to rely on a control device, effectively improving the flexibility of the window adjustment operation.

[0073] Based on the above - mentioned embodiment, Figure 4 a flowchart of another mixed - reality window adjustment method provided by the embodiment of the present application is given. This mixed - reality window adjustment method is a concretization of the above - mentioned mixed - reality window adjustment method. Refer to Figure 4 and this mixed - reality window adjustment method includes:

[0074] S210: When receiving the window activation operation triggered through the touch unit, determine the orientation information of the virtual display device.

[0075] S220: Determine the target window in the virtual environment according to the orientation information.

[0076] Among them, the orientation information provided by this solution can be detected by the orientation detection unit on the virtual display device. Exemplarily, when receiving the window activation operation triggered by the user through the touch unit, the orientation information output by the orientation detection unit on the virtual display device. Among them, the orientation information can be used to indicate the direction (front - facing direction) that the virtual display device faces in the virtual environment.

[0077] Further, according to the orientation information and the positions of each application window in the virtual environment, a target window is determined among the application windows in the virtual environment. For example, when the orientation direction corresponding to the orientation information points to a certain application window, that window can be used as the target window, or the application window closest to the orientation direction corresponding to the orientation information can be used as the target window.

[0078] In one embodiment, after the triggered window activation operation, the user can change the orientation direction pointed by the orientation information by turning the head orientation, so that the user's line of sight faces or is closer to the application window in the virtual environment where the user hopes to perform window adjustment, realizing the rapid determination of the target window. It can also be that before the triggered window activation operation, the user changes the orientation direction pointed by the orientation information by turning the head orientation, so that the user's line of sight faces or is closer to the application window in the virtual environment where the user hopes to perform window adjustment, and then triggers the window activation operation to realize the rapid determination of the target window. This solution rapidly determines the target window in the virtual environment through the orientation information, improving the efficiency and accuracy of target window determination and the window adjustment efficiency.

[0079] In a possible embodiment, after receiving the window activation operation triggered by the touch unit and determining the orientation information of the virtual display device, the mixed reality window adjustment method provided by this solution may further include: displaying a positioning identifier in the display screen of the virtual display device.

[0080] Exemplarily, after receiving the window activation operation and determining the orientation information of the virtual display device, the positioning identifier displayed in the display screen of the virtual display device, for example, displays the positioning identifier at the center of the display screen of the virtual display device. At this time, the display position of the positioning identifier is in the orientation direction corresponding to the orientation information, and the user can understand the orientation direction corresponding to the orientation information through the positioning identifier. Optionally, the positioning identifier provided by this solution can be a cursor of a set shape (such as a cross, a circle, a dot, etc.). This solution indicates the orientation direction corresponding to the orientation information through the positioning identifier, facilitating the user to quickly determine the target window, and the user can determine the corresponding target window according to the synchronously moving positioning identifier by moving the perspective, making the window adjustment operation more flexible.

[0081] In a possible embodiment, when determining the target window in the virtual environment in the mixed reality window adjustment method provided by this solution, it can also be: when receiving the window activation operation triggered by the touch unit, determining candidate windows in the virtual environment; when receiving a window switching operation, switching the candidate windows according to the window switching operation; and when determining that the window selection condition is met, determining the current candidate window as the target window.

[0082] Exemplarily, when receiving a window activation operation triggered by a user through a touch unit, the orientation information output by the orientation detection unit on the virtual display device is obtained. And based on the orientation information and the positions of each application window in the virtual environment, a candidate window is determined among the application windows in the virtual environment. For example, when the orientation direction corresponding to the orientation information points to a certain application window, that window can be used as the candidate window, or the application window closest to the orientation direction corresponding to the orientation information can be used as the candidate window.

[0083] Furthermore, the user can issue a window switching operation through the touch unit or the orientation detection unit to switch the candidate window. For example, a window switching operation can be triggered by a sliding operation on the touch unit, or by rotating the virtual display device so that the orientation detection unit outputs corresponding rotation information, thereby triggering the window switching operation. When receiving the window switching operation, the candidate window is switched among the multiple application windows displayed in the virtual environment according to the window switching operation.

[0084] Furthermore, when it is determined that the window selection condition is met, the current candidate window is determined as the target window. Among them, when the rotation information output by the orientation detection unit indicates that the virtual display device maintains the current orientation direction for a set time length, or when the touch unit receives a window selection operation (such as a click operation, a long press operation, a double click operation or other set touch operations) on the touch unit, it is considered that the window selection condition is met. In this solution, by first determining the switching candidate window when receiving the window switching operation, the user can switch the candidate window, and when it is determined that the window selection condition is met, the current candidate window is determined as the target window, quickly determining the window that the user hopes to adjust, and the window adjustment operation is more flexible.

[0085] In a possible embodiment, after determining the target window in the virtual environment when the mixed reality window adjustment method provided by this solution receives a window activation operation triggered by a touch unit, the target window can also be moved to the middle position of the display screen of the virtual display device.

[0086] Exemplarily, after determining the target window in the virtual environment, the position of the target window in the virtual environment is moved so that the window center of the target window is located on the orientation direction corresponding to the orientation information, thereby moving the target window to the middle position of the display screen of the virtual display device. In this solution, by moving the target window to the middle of the screen, when the user moves the position of the target window, the target window is always located in the center of the screen, the adjustment of the target window by the user is more in line with the visual experience, the window adjustment of the target window is more efficient, and the user experience is improved.

[0087] In one embodiment, after moving the target window to the middle position of the display screen of the virtual display device, the mixed reality window adjustment method provided by this solution can further adjust the angle of the target window to be perpendicular to the line-of-sight direction of the virtual display device.

[0088] Exemplarily, after determining the target window in the virtual environment and moving the target window to the middle position of the display screen of the virtual display device, adjust the angle of the target window so that the angle of the target window is adjusted to be perpendicular to the line-of-sight direction of the virtual display device. At this time, the orientation direction corresponding to the orientation information is perpendicular to the plane corresponding to the target window. By moving the target window to the center of the screen and making the target window perpendicular to the line-of-sight direction of the virtual display device, when the user moves the position of the target window, the target window is always located at the center of the screen and faces the user directly. The adjustment of the target window by the user is more in line with the visual experience, the window adjustment of the target window is more efficient, and the user experience is improved.

[0089] S230: Receive window adjustment information, and perform window adjustment on the target window according to the window adjustment information. The window adjustment includes one or a combination of more of window position adjustment, window size adjustment, and window distance adjustment.

[0090] In a possible embodiment, when the mixed reality window adjustment method provided by this solution receives window adjustment information and performs window adjustment on the target window according to the window adjustment information, it includes at least one of the following methods:

[0091] Method 1: Receive rotation information through the orientation detection unit on the virtual display device, and perform window position adjustment on the target window according to the rotation information.

[0092] Method 2: Receive first touch information in the first direction through the touch unit, and perform window size adjustment on the target window according to the first touch information.

[0093] Method 3: Receive second touch information in the second direction through the touch unit, and perform window distance adjustment on the target window according to the second touch information.

[0094] After determining the target window in the virtual environment, the user can perform window position adjustment by rotating the virtual display device, or can perform window size adjustment and / or window distance adjustment by operating the touch unit. Among them, Method 1, Method 2, and Method 3 can be performed simultaneously or separately, that is, multiple window adjustments among window position, window size, and window distance can be performed simultaneously, or window adjustments of window position, window size, and window distance can be performed separately.

[0095] For Method 1, when the user needs to adjust the window position of the target window in the virtual environment, the user can rotate the virtual display device. At this time, the orientation detection unit on the virtual display device outputs corresponding rotation information. According to the rotation information, the rotation direction, angle, etc. can be determined, and the position of the target window in the virtual environment can be adjusted according to the rotation direction, angle, etc. For example Figure 5 As shown in a schematic diagram of window position adjustment display provided, when the user wears the virtual display device and rotates it, the virtual display device also rotates synchronously. After the window position of the target window is adjusted according to the rotation information, the picture observed by the user is that the target window moves up and down and left and right synchronously in the virtual environment as the user's head turns left and right and up and down. Among them, the amplitude of the target window moving in the virtual environment can be determined according to the distance of the target window from the origin and the rotation angle. The greater the distance and the rotation angle, the greater the amplitude of the target window moving in the virtual environment. This solution accurately and efficiently realizes the window position adjustment of the target window according to the rotation information output by the orientation detection unit on the virtual display device, improving the accuracy and efficiency of window adjustment.

[0096] For Method 2, when the user needs to adjust the window size of the target window in the virtual environment, the user can trigger a sliding touch operation in the first direction on the touch unit. At this time, the touch unit on the virtual display device outputs corresponding first touch information indicating the first direction. The window size of the target window in the virtual environment is adjusted according to the first touch information. For example Figure 6 As shown in a schematic diagram of window size adjustment display provided, when the user wears the virtual display device and triggers a sliding touch operation in the first direction on the touch unit (for example, sliding up and down the touch unit on the temple of the glasses with a finger), the user receives the first touch information output by the touch unit indicating the first direction. After the window size of the target window is adjusted according to the first touch information, the picture observed by the user is that the window size of the target window increases and decreases as the user slides up and down on the touch unit. Among them, the amplitude of the window size change of the target window in the virtual environment can be determined according to the first sliding distance and the first sliding speed reflected by the first touch information. The greater the first sliding distance and the first sliding speed, the greater the amplitude of the window size change of the target window in the virtual environment. This solution accurately and efficiently realizes the window size adjustment of the target window according to the first touch information output by the touch unit on the virtual display device, improving the accuracy and efficiency of window adjustment.

[0097] For Method 3, when the user needs to adjust the window distance of the target window in the virtual environment, the user can trigger a sliding touch operation in the second direction on the touch unit. At this time, the touch unit on the virtual display device outputs corresponding second touch information indicating the second direction. The window distance of the target window in the virtual environment is adjusted according to the second touch information. For exampleFigure 7 As shown in the schematic diagram of window distance adjustment provided, when the user wears a virtual display device and triggers a sliding touch operation in the second direction on the touch unit (for example, slides the finger back and forth on the touch unit of the temple), the touch unit outputs corresponding second touch information indicating the second direction. After adjusting the window distance of the target window according to the second touch information, the user observes that the target window moves back and forth on the touch unit, and the window distance of the target window decreases and increases accordingly. Among them, the amplitude of the distance change of the target window in the virtual environment can be determined according to the second sliding distance and the second sliding speed reflected by the second touch information. The greater the second sliding distance and the second sliding speed, the greater the amplitude of the distance change of the target window in the virtual environment. This solution accurately and efficiently realizes the adjustment of the window distance of the target window according to the second touch information indicating the second direction output by the touch unit on the virtual display device, improving the accuracy and efficiency of window adjustment.

[0098] S240: When receiving the window fixing operation triggered by the touch unit, fix the target window in the virtual environment.

[0099] As described above, when receiving the window activation operation triggered by the touch unit, the target window in the virtual environment is determined, the window of the target window is adjusted according to the received window adjustment information, and when receiving the window fixing operation triggered by the touch unit, the target window is fixed in the virtual environment, realizing the adjustment of the window in the virtual environment. The adjustment operation of the window does not need to rely on a control device, effectively improving the flexibility of the window adjustment operation. At the same time, the target window in the virtual environment is quickly determined through the orientation information, improving the determination efficiency and accuracy of the target window and the window adjustment efficiency. The target window can also be moved to the middle of the screen. When the user moves the position of the target window, the target window is always located in the center of the screen. The adjustment of the target window by the user is more in line with the visual experience, and the window adjustment of the target window is more efficient, improving the user experience.

[0100] Figure 8 The structure diagram of a mixed reality window adjustment device provided by an embodiment of the present application is given. Refer to Figure 8 , this mixed reality window adjustment device can be applied to a virtual display device. The virtual display device is provided with a touch unit. The mixed reality window adjustment device includes an adjustment response module 31, a window adjustment module 32, and a window fixing module 33.

[0101] Among them, the adjustment response module 31 is configured to determine a target window in the virtual environment when receiving a window activation operation triggered by the touch unit; the window adjustment module 32 is configured to receive window adjustment information and perform window adjustment on the target window according to the window adjustment information, and the window adjustment includes one or a combination of more of window position adjustment, window size adjustment, and window distance adjustment; the window fixing module 33 is configured to fix the target window in the virtual environment when receiving a window fixing operation triggered by the touch unit.

[0102] As described above, when receiving a window activation operation triggered by the touch unit, a target window in the virtual environment is determined, the window is adjusted according to the received window adjustment information, and when receiving a window fixing operation triggered by the touch unit, the target window is fixed in the virtual environment, so as to realize the adjustment of the window in the virtual environment. The window adjustment operation does not need to rely on a control device, effectively improving the flexibility of the window adjustment operation.

[0103] In a possible embodiment, when the adjustment response module 31 determines a target window in the virtual environment when receiving a window activation operation triggered by the touch unit, it includes:

[0104] When receiving a window activation operation triggered by the touch unit, determine the orientation information of the virtual display device, and the orientation information is detected by an orientation detection unit on the virtual display device;

[0105] Determine the target window in the virtual environment according to the orientation information.

[0106] In a possible embodiment, the adjustment response module 31 is further configured to display a positioning identifier in the display screen of the virtual display device after determining the orientation information of the virtual display device when receiving a window activation operation triggered by the touch unit.

[0107] In a possible embodiment, when the adjustment response module 31 determines a target window in the virtual environment when receiving a window activation operation triggered by the touch unit, it includes:

[0108] When receiving a window activation operation triggered by the touch unit, determine candidate windows in the virtual environment;

[0109] When receiving a window switching operation, switch the candidate windows according to the window switching operation;

[0110] When it is determined that the window selection condition is satisfied, the current candidate window is determined as the target window.

[0111] In a possible embodiment, when the window adjustment module 32 receives window adjustment information and adjusts the target window according to the window adjustment information, it includes at least one of the following:

[0112] Receiving rotation information through an orientation detection unit on the virtual display device, and adjusting the window position of the target window according to the rotation information;

[0113] Receiving first touch information in a first direction through the touch unit, and adjusting the window size of the target window according to the first touch information;

[0114] Receiving second touch information in a second direction through the touch unit, and adjusting the window distance of the target window according to the second touch information.

[0115] In a possible embodiment, the mixed reality window adjustment device further includes a window alignment module for moving the target window to the middle position of the display screen of the virtual display device.

[0116] In a possible embodiment, the window alignment module is further configured to adjust the angle of the target window to be perpendicular to the line-of-sight direction of the virtual display device after moving the target window to the middle position of the display screen of the virtual display device.

[0117] It should be noted that in the embodiments of the above-mentioned mixed reality window adjustment device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present application.

[0118] The embodiments of the present application further provide a mixed reality window adjustment device, and the mixed reality window adjustment device can integrate the mixed reality window adjustment device provided by the embodiments of the present application. Figure 9 is a schematic structural diagram of a mixed reality window adjustment device provided by the embodiments of the present application. Refer to Figure 9 , the mixed reality window adjustment device includes: an input device 43, an output device 44, a memory 42, and one or more processors 41; the memory 42 is used to store one or more programs; when the one or more programs are executed by the one or more processors 41, the one or more processors 41 implement the mixed reality window adjustment method provided by the above embodiments. Among them, the input device 43, the output device 44, the memory 42, and the processor 41 can be connected through a bus or other means, Figure 9 taking connection through a bus as an example.

[0119] The memory 42 serves as a computer-readable storage medium that can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the mixed reality window adjustment method provided in any embodiment of the present application (for example, the adjustment response module 31, the window adjustment module 32, and the window fixing module 33 in the mixed reality window adjustment device). The memory 42 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 42 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 42 may further include a memory remotely provided with respect to the processor 41, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0120] The input device 43 can be used to receive input digital or character information and generate key signal inputs related to the user settings and function controls of the device. The output device 44 may include a display device such as a display screen.

[0121] The processor 41 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 42, that is, implements the above-mentioned mixed reality window adjustment method.

[0122] The above-provided mixed reality window adjustment device, device, and computer can be used to execute the mixed reality window adjustment method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0123] The embodiment of the present application also provides a storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are used to execute the mixed reality window adjustment method provided in the above embodiment. The mixed reality window adjustment method includes: determining a target window in the virtual environment when receiving a window activation operation triggered by the touch unit; receiving window adjustment information and performing window adjustment on the target window according to the window adjustment information, where the window adjustment includes one or a combination of more of window position adjustment, window size adjustment, and window distance adjustment; and fixing the target window in the virtual environment when receiving a window fixing operation triggered by the touch unit.

[0124] Storage Medium - Any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media such as CD-ROMs, floppy disks, or magnetic tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions executable by one or more processors (e.g., embodied as a computer program).

[0125] Of course, for a storage medium storing computer-executable instructions provided in an embodiment of the present application, the computer-executable instructions are not limited to the mixed reality window adjustment method provided above, and may also execute related operations in the mixed reality window adjustment method provided in any embodiment of the present application.

[0126] The mixed reality window adjustment device, equipment, and storage medium provided in the above embodiments can execute the mixed reality window adjustment method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, reference may be made to the mixed reality window adjustment method provided in any embodiment of the present application.

[0127] The above is only a preferred embodiment of the present application and the technical principles applied. The present application is not limited to the specific embodiments provided here. Various obvious changes, re-adjustments, and substitutions that can be made by those skilled in the art will not depart from the protection scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments. Without departing from the concept of the present application, it may also include more other equivalent embodiments, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for adjusting a mixed reality window, applied to a virtual display device, wherein the virtual display device is provided with a touch unit, characterized in that, The method for adjusting a mixed reality window includes: When receiving a window activation operation triggered by the touch unit, determining a target window in the virtual environment; Receiving window adjustment information, and performing window adjustment on the target window according to the window adjustment information, where the window adjustment includes one or a combination of more of window position adjustment, window size adjustment, and window distance adjustment; When receiving a window fixing operation triggered by the touch unit, fixing the target window in the virtual environment.

2. The mixed reality window adjustment method according to claim 1, wherein The step of determining a target window in the virtual environment when receiving a window activation operation triggered by the touch unit includes: When receiving a window activation operation triggered by the touch unit, determining the orientation information of the virtual display device, where the orientation information is detected by an orientation detection unit on the virtual display device; Determining a target window in the virtual environment according to the orientation information.

3. The method for adjusting a mixed reality window according to claim 2, wherein After determining the orientation information of the virtual display device when receiving a window activation operation triggered by the touch unit, it further includes: Displaying a positioning identifier in the display screen of the virtual display device.

4. The method for adjusting a mixed reality window according to claim 1, wherein The step of determining a target window in the virtual environment when receiving a window activation operation triggered by the touch unit includes: When receiving a window activation operation triggered by the touch unit, determining candidate windows in the virtual environment; When receiving a window switching operation, switching candidate windows according to the window switching operation; When determining that the window selection condition is satisfied, determining the current candidate window as the target window.

5. The mixed reality window adjustment method according to claim 1, wherein The step of receiving window adjustment information and performing window adjustment on the target window according to the window adjustment information includes at least one of the following: Receiving rotation information through the orientation detection unit on the virtual display device, and performing window position adjustment on the target window according to the rotation information; Receiving first touch information in a first direction through the touch unit, and performing window size adjustment on the target window according to the first touch information; Receiving second touch information in a second direction through the touch unit, and performing window distance adjustment on the target window according to the second touch information.

6. The mixed reality window adjustment method according to claim 1, wherein After determining a target window in the virtual environment when receiving a window activation operation triggered by the touch unit, it further includes: Moving the target window to the middle position of the display screen of the virtual display device.

7. The mixed reality window adjustment method according to claim 6, wherein After moving the target window to the middle position of the display screen of the virtual display device, it further includes: Adjusting the angle of the target window to be perpendicular to the line of sight direction of the virtual display device.

8. A mixed reality window adjustment device is applied to a virtual display device, and the virtual display device is provided with a touch unit, characterized in that, The mixed reality window adjustment device includes an adjustment response module, a window adjustment module, and a window fixing module, where: The adjustment response module is configured to determine a target window in the virtual environment when receiving a window activation operation triggered by the touch unit; The window adjustment module is configured to receive window adjustment information and perform window adjustment on the target window according to the window adjustment information, where the window adjustment includes one or a combination of more than one of window position adjustment, window size adjustment, and window distance adjustment; The window fixation module is configured to fix the target window in the virtual environment when receiving a window fixation operation triggered by the touch unit.

9. A mixed reality window adjustment device, characterized in that, Comprising: A memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the mixed reality window adjustment method according to any one of claims 1-7.

10. A storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the mixed reality window adjustment method according to any one of claims 1-7 when executed by a computer processor.

Images