Method for keyboard input in three-dimensional space, and head-mounted display device, medium and product
By responding to physical keyboard inputs in a head-mounted display device to generate three-dimensional or two-dimensional candidate word positions, the problem of the word selection box not being able to focus in 3D spatial input is solved, improving input speed and fluency.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- HANGZHOU LINGBAN TECH CO LTD
- Filing Date
- 2026-02-09
- Publication Date
- 2026-07-02
Abstract
Description
Three-dimensional keyboard input methods, head-mounted display devices, media and products
[0001] Cross-reference of related applications
[0002] This application claims priority to Chinese Patent Application No. 202411909583.X, filed on December 23, 2024, the entire contents of which are incorporated herein by reference. Technical Field
[0003] Embodiments of this disclosure relate to the field of computer technology, and more specifically to keyboard input methods in three-dimensional space, head-mounted display devices, media, and products. Background Technology
[0004] With the development of interactive technology for head-mounted displays, information can be input within a spatial system using a physical keyboard. Currently, the common methods for inputting information within a spatial system using a physical keyboard are: in spatial systems based on AOSP (Android Open Source Project), the common physical keyboard input method is a 2D virtual keyboard; or in the spatial system of a head-mounted display device, an extended input method service is used to implement a 3D virtual keyboard input method.
[0005] However, when using the above methods, the following technical problems often exist: 3D spatial input cannot be supported when using 2D virtual keyboard input methods; when using a solution that fully utilizes extended input method services to implement 3D virtual keyboard input methods, the response speed is slow when using physical keyboard input, and the word selection box cannot dynamically focus near the input box.
[0006] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0007] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.
[0008] Some embodiments of this disclosure provide a three-dimensional keyboard input method, a head-mounted display device, a computer-readable medium, and a computer program product to solve one or more of the technical problems mentioned in the background section above.
[0009] In a first aspect, some embodiments of this disclosure provide a three-dimensional keyboard input method, the method comprising: in response to detecting a typing operation performed on an associated physical keyboard, determining a candidate word list based on at least one key corresponding to the typing operation, wherein an input cursor is displayed in a window corresponding to the typing operation, the window being displayed in the three-dimensional space of the head-mounted display device; determining the spatial dimension of the window corresponding to the typing operation; in response to determining that the spatial dimension represents a two-dimensional window, generating candidate word column position information based on the two-dimensional position information of the input cursor in the window; and displaying a candidate word column in the window based on the candidate word column position information, wherein at least one candidate word from the candidate word list is displayed in the candidate word column.
[0010] In one possible implementation, displaying the candidate word field in the window based on the candidate word field position information includes: displaying the candidate word field in the window based on the candidate word field position information and the native input method service.
[0011] In one embodiment, the method further includes: in response to determining the spatial dimension characterizing the three-dimensional window, displaying a spatially independent candidate word column in the three-dimensional space, wherein the displayed candidate word column corresponds to the three-dimensional application displayed in the window, and the displayed candidate word column displays at least one candidate word from the candidate word list.
[0012] In one embodiment, displaying a spatially independent candidate word column in the aforementioned three-dimensional space includes: generating candidate word column position information based on the position information of the input box corresponding to the input cursor; and displaying a spatially independent candidate word column in the aforementioned three-dimensional space based on the generated candidate word column position information and the extended input method service.
[0013] In one embodiment, the method further includes: in response to detecting a selection operation performed on a candidate word displayed in the displayed candidate word bar, determining an input channel corresponding to the selection operation; in response to determining that the input channel corresponds to a two-dimensional window, displaying the selected candidate word in a two-dimensional input box corresponding to the input cursor; and in response to determining that the input channel corresponds to a three-dimensional window, displaying the selected candidate word in a three-dimensional input box corresponding to the input cursor.
[0014] In one possible implementation, the method further includes: in response to detecting a click operation on an input box in a window, determining whether a physical keyboard is connected; in response to determining that no physical keyboard is connected, displaying a corresponding virtual keyboard in the three-dimensional space corresponding to the window, wherein the spatial hierarchy of the virtual keyboard is independent.
[0015] In one possible implementation, before determining a candidate word list based on at least one key corresponding to the typing operation in response to detecting a typing operation on an associated physical keyboard, the method further includes: establishing a message channel between the head-mounted display device and the extended input method service; and activating the native input method service in response to detecting a click operation on an input box in a window.
[0016] Secondly, some embodiments of this disclosure provide a head-mounted display device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.
[0017] Thirdly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.
[0018] Fourthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the implementations of the first aspect above.
[0019] The various embodiments of this disclosure have the following beneficial effects: The three-dimensional spatial keyboard input method of some embodiments of this disclosure can support 3D spatial input, allowing the word selection box to dynamically focus near the input box. Specifically, the reason why 3D spatial input cannot be supported and the word selection box cannot dynamically focus near the input box is that: when using a 2D virtual keyboard input method, 3D spatial input cannot be supported; when using a solution that fully utilizes an extended input method service to implement a 3D virtual keyboard input method, the response speed is slow when using a physical keyboard, and the word selection box cannot dynamically focus near the input box. Based on this, the three-dimensional spatial keyboard input method of some embodiments of this disclosure firstly, in response to detecting a typing operation acting on an associated physical keyboard, determines a candidate word list based on at least one key corresponding to the typing operation. The window corresponding to the typing operation displays an input cursor, and the window is displayed in the three-dimensional space of the head-mounted display device. Therefore, after a typing operation is triggered at the input cursor position in the user's three-dimensional spatial window, a list of candidate words for the user to select is determined based on at least one key entered. Then, the spatial dimension of the window corresponding to the typing operation is determined. Therefore, the spatial dimension of the window where the user needs to input information can be determined to determine the display method of the candidate word bar. Then, in response to determining the spatial dimension representing the two-dimensional window, the candidate word bar position information is generated based on the two-dimensional position information of the input cursor in the window. Thus, when the window is a two-dimensional window, the position information of the candidate word bar to be displayed can be dynamically determined based on the two-dimensional position information of the input cursor. Finally, the candidate word bar is displayed in the window based on the candidate word bar position information, wherein at least one candidate word from the candidate word list is displayed in the candidate word bar. Thus, the candidate word bar can be dynamically displayed in a two-dimensional window based on the dynamically determined position information of the candidate word bar. Also, because the candidate word bar can be displayed in a two-dimensional window in three-dimensional space, 3D spatial input can be supported. Furthermore, because the position of the candidate word bar is dynamically positioned using two-dimensional position information when displaying the candidate word bar in a two-dimensional window, it is not necessary to position the cursor and the candidate word bar in three-dimensional space, and therefore it is not necessary to fully use the input method service extended for 3D spatial systems, which improves the response speed of physical keyboard input and realizes the dynamic display of the candidate word box based on the position of the input cursor, so that the word selection box is dynamically focused near the input box. Attached Figure Description
[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.
[0021] Figure 1 is a flowchart of some embodiments of the keyboard input method in three-dimensional space according to the present disclosure;
[0022] Figure 2 is a flowchart of some other embodiments of the keyboard input method in three-dimensional space according to the present disclosure;
[0023] Figure 3 is a schematic diagram of a head-mounted display device suitable for implementing some embodiments of the present disclosure. Detailed Implementation
[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0025] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.
[0026] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0027] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0028] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0029] Before performing any of the operations involving the collection, storage, and use of user personal information (such as keystrokes, selected candidate words, and content entered into input boxes) as disclosed in this disclosure, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, informing personal information subjects, and obtaining prior authorization and consent from personal information subjects.
[0030] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.
[0031] Figure 1 illustrates a flow chart 100 of some embodiments of a three-dimensional keyboard input method according to the present disclosure. The three-dimensional keyboard input method includes the following steps:
[0032] Step 101: In response to detecting a typing operation on an associated physical keyboard, determine a list of candidate words based on at least one key corresponding to the typing operation.
[0033] In some embodiments, the execution subject of the three-dimensional keyboard input method (e.g., a head-mounted display device) can determine a candidate word list based on at least one key corresponding to the input operation in response to detecting a typing operation on an associated physical keyboard. The physical keyboard can be a physical keyboard connected to the head-mounted display device. The connection method between the physical keyboard and the head-mounted display device can include, but is not limited to, at least one of the following: wired connection, Bluetooth, 3G / 4G connection, WiFi connection, WiMAX connection, Zigbee connection, UWB (ultra-wideband) connection, and other currently known or future wireless connection methods. The head-mounted display device can be worn by a user to view displayed content. The head-mounted display device can include, but is not limited to, AR (Augmented Reality) glasses, MR (Mixed Reality) glasses, and VR (Virtual Reality) glasses. The input operation can be pressing a key on a physical keyboard. The input operation can correspond to at least one key, and the at least one key has a sequence. The aforementioned key position includes the currently typed key position and the key position previously typed when inputting information. For example, the information the user wants to input this time could be "today". The input method used can be Pinyin input method. The currently typed key position can be "t". The previously typed key positions can include: j, i, n. Therefore, when "t" is typed, at least one key position corresponding to the typing operation can include: j, i, n, t. An input cursor is displayed in the window corresponding to the aforementioned typing operation. The input cursor can be displayed in the input box. After the user clicks in the input box of a window, the input cursor can be displayed in the input box. The input cursor can indicate the position to be entered. The input cursor can be displayed in the input box at a preset frequency. The aforementioned window is displayed in the three-dimensional space of the aforementioned head-mounted display device. The window can be a virtual screen in the three-dimensional space of the head-mounted display device used to display content. The window in the three-dimensional space can be a two-dimensional window or a three-dimensional window. A two-dimensional window can be used to display two-dimensional display content. A three-dimensional window can be used to display three-dimensional display content. In practice, the aforementioned executing entity can utilize the candidate word determination function of the input method to determine a list of candidate words based on at least one key corresponding to the typing operation. For example, the candidate word list could be: today, gold bar, allowance, motivation.
[0034] In one implementation, prior to step 101, the executing entity can establish a message channel between the head-mounted display device and the extended input method service. The extended input method service can be deployed locally on the executing entity, or it can be deployed both locally and on a server. In practice, a link to the extended input method service can be declared in the runtime code of the head-mounted display device to establish the message channel between the head-mounted display device and the extended input method service. Secondly, in response to detecting a click operation on an input box in the window, the native input method service can be started. Thus, it can be ensured that both the extended input method service and the native input method service are available before the user types.
[0035] In one implementation, the native input method service can be used to handle, but is not limited to, the following tasks: displaying a candidate word bar in a two-dimensional window, detecting typing operations, and detecting selection operations. The extended input method service can be used to handle, but is not limited to, the following tasks: displaying a candidate word bar corresponding to a three-dimensional window, determining spatial dimensions, determining the candidate word list, and determining the selected candidate word.
[0036] Step 102: Determine the spatial dimension of the window corresponding to the typing operation.
[0037] In some embodiments, the executing entity may determine the spatial dimension of the window corresponding to the typing operation. The spatial dimension of the window may be the display dimension of the graphical user interface of the application displayed in the window. For example, when a 3D application is displayed in the window, the spatial dimension of the window may be three-dimensional, and the window may be a three-dimensional window. When a 2D application is displayed in the window, the spatial dimension of the window may be two-dimensional, and the window may be a two-dimensional window.
[0038] Step 103: In response to determining the spatial dimension representation of the two-dimensional window, generate candidate word column position information based on the two-dimensional position information of the input cursor in the window.
[0039] In some embodiments, in response to determining the spatial dimension representing a two-dimensional window, the executing entity can generate candidate word field position information based on the two-dimensional position information of the input cursor in the window. The two-dimensional position information can be the two-dimensional coordinates of the input cursor in the upper window. For example, the two-dimensional position information can be the two-dimensional coordinates of the center point of the input cursor. In practice, the executing entity can shift the vertical coordinate included in the two-dimensional position information down by a preset distance to obtain the candidate word field position information. As an example, the preset distance can be 100, the two-dimensional position information can be (300, 100), and the candidate word field position information can be (300, 200). The candidate word field position information can be used to locate the center position of the candidate word field or to locate the upper left corner position of the candidate word field.
[0040] In some optional implementations of certain embodiments, the executing entity can generate candidate word column position information based on the two-dimensional position information of the input cursor in the window through the following steps:
[0041] The first step is to shift the ordinate of the two-dimensional position information downwards to obtain the offset two-dimensional position information. In practice, the executing entity can shift the ordinate of the two-dimensional position information downwards by a preset distance to obtain the offset two-dimensional position information.
[0042] The second step is to determine the offset two-dimensional position information as the initial top left corner point of the candidate word column.
[0043] The third step is to determine the initial top-right, bottom-left, and bottom-right corners of the corresponding candidate word column based on the initial top-left corner, the preset length, and the preset width. In practice, the executing entity can use the initial top-left corner, the preset length, and the preset width to mark the position of the candidate word column, thus obtaining the initial top-right, bottom-left, and bottom-right corners.
[0044] The fourth step is to determine whether any of the initial top-left, top-right, bottom-left, and bottom-right corners exceed the window's border.
[0045] The fifth step, in response to determining the initial top-right corner and the initial bottom-right corner as they extend beyond the window's border, determines the distance beyond the right border of the window corresponding to the initial top-right corner or the initial bottom-right corner.
[0046] Step 6: Generate a first adjustment distance based on the excess distance. In practice, the executing entity can determine the first adjustment distance as the sum of the excess distance and the first preset adjustment distance. The first preset adjustment distance can be greater than or equal to 0, and less than or equal to the difference between the window length and the preset length.
[0047] Step 7: Shift the initial top-left, top-right, bottom-left, and bottom-right corners to the left by the first adjustment distance to obtain the top-left, top-right, bottom-left, and bottom-right corners as the candidate word column position information. This allows for adaptive leftward adjustment of the first adjustment distance when the candidate word column exceeds the right edge of the window, preventing it from extending beyond the right edge.
[0048] Step 8: In response to determining that the initial lower left corner and the initial lower right corner exceed the border of the window, the distance between the initial upper left corner and the top position of the input cursor is determined as the first distance.
[0049] Step 9: Determine the second distance by summing the preset width and the first distance.
[0050] Step 10: Generate a second adjustment distance based on the second distance. In practice, the executing entity can determine the second adjustment distance as the sum of the second distance and the second preset adjustment distance. The second preset adjustment distance can be greater than or equal to the length of the input cursor, and less than or equal to the vertical distance between the top border of the window and the initial top-left corner.
[0051] Step 11: Shift the initial top-left, top-right, bottom-left, and bottom-right corners upwards by the second adjustment distance to obtain the top-left, top-right, bottom-left, and bottom-right corners as the candidate word column position information. This allows for adaptive upward adjustment of the second adjustment distance when the candidate word column exceeds the bottom border of the window, preventing it from extending beyond the window's bottom edge.
[0052] Step 104: Display the candidate word column in the window based on the candidate word column position information.
[0053] In some embodiments, the executing entity can display a candidate word column in a window based on the candidate word column position information. The candidate word column displays at least one candidate word from the candidate word list. The candidate word column can display a preset number of candidate words or a preset length of characters; any extra candidate words can be displayed on the next page. In practice, the executing entity can display the candidate word column at the position corresponding to the candidate word column position information in the window. Thus, the candidate word column can be displayed in a two-dimensional window.
[0054] In one implementation, the executing entity can display the candidate word bar in a window based on the candidate word bar location information and the native input method service. The native input method service can be the input method service corresponding to a two-dimensional application. For example, the native input method service can be the Android input method service. The native input method service can be deployed locally or simultaneously locally and on the server. Specifically, the candidate word bar location information can be used as an interface parameter to call the candidate word bar display interface of the native input method service to display the candidate word bar in the window. Therefore, the native input method service can be directly used to display the candidate word bar in a two-dimensional window, eliminating the time spent on the message channel between the head-mounted display device and the extended input method service, and the time spent drawing the three-dimensional candidate word bar. This enables rapid candidate word bar drawing and improves the smoothness of physical keyboard input.
[0055] In one embodiment, the executing entity may further determine the input channel corresponding to the selection operation in response to detecting a selection operation performed on a candidate word displayed in the displayed candidate word bar. The input channel may represent the window type corresponding to the input box. The interaction method corresponding to the selection operation may include, but is not limited to, at least one of the following: touch interaction, button interaction, gesture interaction, voice interaction, head control interaction, and eye-tracking interaction. Next, in response to determining that the input channel corresponds to a two-dimensional window, the selected candidate word may be displayed in the two-dimensional input box corresponding to the input cursor. Then, in response to determining that the input channel corresponds to a three-dimensional window, the selected candidate word may be displayed in the three-dimensional input box corresponding to the input cursor. Thus, the selected candidate word can be displayed for both the two-dimensional and three-dimensional windows.
[0056] In one implementation, the executing entity can determine whether a physical keyboard is connected in response to detecting a click operation on an input box in a window. The interaction method corresponding to the click operation may include, but is not limited to, at least one of the following: touch interaction, button interaction, gesture interaction, voice interaction, head control interaction, and eye-tracking interaction. Then, in response to determining that no physical keyboard is connected, a virtual keyboard for the corresponding window can be displayed in three-dimensional space. In practice, the executing entity can display the virtual keyboard on the layer above the window. The spatial hierarchy of the virtual keyboard is independent. Thus, when typing using the virtual keyboard, a three-dimensional display can be used to present a stereoscopic visual experience.
[0057] The various embodiments of this disclosure have the following beneficial effects: The three-dimensional spatial keyboard input method of some embodiments of this disclosure can support 3D spatial input, allowing the word selection box to dynamically focus near the input box. Specifically, the reason why 3D spatial input cannot be supported and the word selection box cannot dynamically focus near the input box is that: when using a 2D virtual keyboard input method, 3D spatial input cannot be supported; when using a solution that fully utilizes an extended input method service to implement a 3D virtual keyboard input method, the response speed is slow when using a physical keyboard, and the word selection box cannot dynamically focus near the input box. Based on this, the three-dimensional spatial keyboard input method of some embodiments of this disclosure firstly, in response to detecting a typing operation acting on an associated physical keyboard, determines a candidate word list based on at least one key corresponding to the typing operation. The window corresponding to the typing operation displays an input cursor, and the window is displayed in the three-dimensional space of the head-mounted display device. Thus, after a typing operation is triggered at the input cursor position in the user's three-dimensional space window, a list of candidate words for the user to select is determined based on at least one key entered. Then, the spatial dimension of the window corresponding to the typing operation is determined. Therefore, the spatial dimension of the window where the user needs to input information can be determined to determine the display method of the candidate word bar. Then, in response to determining the spatial dimension representing the two-dimensional window, the candidate word bar position information is generated based on the two-dimensional position information of the input cursor in the window. Thus, when the window is a two-dimensional window, the position information of the candidate word bar to be displayed can be dynamically determined based on the two-dimensional position information of the input cursor. Finally, the candidate word bar is displayed in the window based on the candidate word bar position information, wherein at least one candidate word from the candidate word list is displayed in the candidate word bar. Thus, the candidate word bar can be dynamically displayed in a two-dimensional window based on the dynamically determined position information of the candidate word bar. Also, because the candidate word bar can be displayed in a two-dimensional window in three-dimensional space, 3D spatial input can be supported. Furthermore, because the position of the candidate word bar is dynamically positioned using two-dimensional position information when displaying the candidate word bar in a two-dimensional window, it is not necessary to position the cursor and candidate word bar in three-dimensional space, and therefore it is not necessary to fully use the input method service extended for 3D spatial systems, which improves the response speed of physical keyboard input and realizes the dynamic display of the candidate word box based on the position of the input cursor, so that the word selection box is dynamically focused near the input box.
[0058] Referring further to Figure 2, a flow 200 of another embodiment of the keyboard input method in three-dimensional space is illustrated. This flow 200 of the three-dimensional keyboard input method includes the following steps:
[0059] Step 201: In response to detecting a typing operation on an associated physical keyboard, determine a list of candidate words based on at least one key corresponding to the typing operation.
[0060] Step 202: Determine the spatial dimension of the window corresponding to the typing operation.
[0061] Step 203: In response to determining the spatial dimension representation of the two-dimensional window, generate candidate word column position information based on the two-dimensional position information of the input cursor in the window.
[0062] Step 204: Display the candidate word column in the window based on the candidate word column position information.
[0063] In some embodiments, the specific implementation of steps 201-204 and the resulting technical effects can be referred to steps 101-104 in the embodiments corresponding to Figure 1, and will not be repeated here.
[0064] Step 205: In response to determining the spatial dimension representation of the three-dimensional window, display a spatially independent candidate word column in the three-dimensional space.
[0065] In some embodiments, the execution subject of the three-dimensional keyboard input method (e.g., a head-mounted display device) can display a spatially hierarchical candidate word bar in three-dimensional space in response to determining a spatial dimension representing a three-dimensional window. The displayed candidate word bar corresponds to the three-dimensional application displayed in the window. The displayed candidate word bar shows at least one candidate word from the candidate word list. The spatial hierarchy of the displayed candidate word bar is independent. It can be understood that the candidate word bar is displayed on top of the window. In practice, the execution subject can display the candidate word bar below the window in three-dimensional space. The execution subject can also display the candidate word bar at the center of the window in three-dimensional space. Thus, a three-dimensional display method can be used to display the candidate word bar.
[0066] In some optional implementations of certain embodiments, the executing entity may perform the following steps:
[0067] The first step is to generate candidate word field position information based on the position information of the input box corresponding to the input cursor. For example, the position information of the input box can be represented by the coordinates of the center of the bottom edge of the input box. Here, there is no limitation on the position type of the input box represented by the position information. In practice, the executing entity can shift the vertical coordinate of the position information of the input box down by a first preset distance and increase the depth coordinate of the position information by a second preset distance to obtain the candidate word field position information. Here, the candidate word field position information is in three-dimensional coordinates.
[0068] The second step involves displaying a spatially independent candidate word bar in three-dimensional space based on the generated candidate word bar location information and the extended input method service. The extended input method service can be an extension service that improves upon the native input method service by providing coordinate positioning and UI rendering in three-dimensional space. For example, the extended input method service could be an extension service that uses Unity to implement coordinate positioning and UI rendering in three-dimensional space on top of the Android input method service. In practice, the executing entity can use the candidate word bar location information as an interface parameter to call the candidate word bar display interface of the extended input method service to display a spatially independent candidate word bar in three-dimensional space.
[0069] As can be seen from Figure 2, compared with the description of some embodiments corresponding to Figure 1, the flowchart 200 of the three-dimensional keyboard input method in some embodiments corresponding to Figure 2 embodies the extended steps of displaying the candidate word bar corresponding to the three-dimensional window. Therefore, the schemes described in these embodiments can be matched to the candidate word bar display of three-dimensional applications to present a hierarchical three-dimensional display effect.
[0070] Referring now to FIG3, a schematic diagram of a head-mounted display device 300 (e.g., a head-mounted display device) suitable for implementing some embodiments of the present disclosure is shown. The head-mounted display device shown in FIG3 is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present disclosure.
[0071] As shown in Figure 3, the head-mounted display device 300 may include a processing unit 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the head-mounted display device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0072] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, microdisplays, speakers, vibrators, etc.; and communication devices 309. Communication device 309 allows head-mounted display device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 illustrates a head-mounted display device 300 with various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Each box shown in Figure 3 may represent one device, or multiple devices may be represented as needed.
[0073] Head-mounted display devices may include display modules that can be used to project images in front of a user's eyes. The display module may include microdisplays and optical components.
[0074] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.
[0075] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0076] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.
[0077] The aforementioned computer-readable medium may be included in the aforementioned head-mounted display device; or it may exist independently and not assembled into the head-mounted display device. The aforementioned computer-readable medium carries one or more programs that, when executed by the head-mounted display device, cause the head-mounted display device to: in response to detecting a keystroke operation on an associated physical keyboard, determine a candidate word list based on at least one key corresponding to the keystroke operation, wherein an input cursor is displayed in a window corresponding to the keystroke operation, and the window is displayed in the three-dimensional space of the head-mounted display device; determine the spatial dimension of the window corresponding to the keystroke operation; in response to determining that the spatial dimension represents a two-dimensional window, generate candidate word column position information based on the two-dimensional position information of the input cursor in the window; and display a candidate word column in the window based on the candidate word column position information, wherein at least one candidate word from the candidate word list is displayed in the candidate word column.
[0078] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, and conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0080] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0081] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements any of the three-dimensional keyboard input methods described above.
[0082] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.
Claims
1. A three-dimensional keyboard input method, applied to a head-mounted display device, comprising: In response to detecting a keystroke operation on an associated physical keyboard, a candidate word list is determined based on at least one key corresponding to the keystroke operation, wherein an input cursor is displayed in a window corresponding to the keystroke operation, and the window is displayed in the three-dimensional space of the head-mounted display device; Determine the spatial dimensions of the window corresponding to the typing operation; In response to determining the spatial dimension representation of the two-dimensional window, candidate word column position information is generated based on the two-dimensional position information of the input cursor in the window; Based on the candidate word position information, a candidate word column is displayed in the window, wherein at least one candidate word from the candidate word list is displayed in the candidate word column.
2. The method according to claim 1, wherein, The step of displaying the candidate word field in the window according to the candidate word field position information includes: Based on the candidate word position information and the native input method service, the candidate word bar is displayed in the window.
3. The method according to claim 1, wherein, The method further includes: In response to determining the spatial dimension characterizing the three-dimensional window, a spatially independent candidate word column is displayed in the three-dimensional space, wherein the displayed candidate word column corresponds to the three-dimensional application displayed in the window, and at least one candidate word from the candidate word list is displayed in the displayed candidate word column.
4. The method according to claim 3, wherein, The method of displaying a candidate word column with spatially independent hierarchical structure in the three-dimensional space includes: Based on the position information of the input box corresponding to the input cursor, generate candidate word column position information; Based on the generated candidate word column location information and extended input method services, a spatially independent candidate word column is displayed in the three-dimensional space.
5. The method according to any one of claims 1-4, wherein, The method further includes: In response to detecting a selection operation performed on a candidate word displayed in the displayed candidate word bar, the input channel corresponding to the selection operation is determined; In response to determining the two-dimensional window corresponding to the input channel, the selected candidate word is displayed in the two-dimensional input box corresponding to the input cursor; In response to determining the three-dimensional window corresponding to the input channel, the selected candidate word is displayed in the three-dimensional input box corresponding to the input cursor.
6. The method according to claim 1, wherein, The method further includes: In response to the detection of a click operation on an input box in the window, determine whether a physical keyboard is connected; In response to determining that no physical keyboard is connected, a corresponding virtual keyboard for the window is displayed in the three-dimensional space, wherein the spatial hierarchy of the virtual keyboard is independent.
7. The method according to claim 1, wherein, Before determining a candidate word list based on at least one key corresponding to the typing operation in response to detecting a typing operation on an associated physical keyboard, the method further includes: Establish a message channel between the head-mounted display device and the extended input method service; In response to the detection of a click operation on the input box in the window, the native input method service is started.
8. A head-mounted display device, comprising: Display module, used to project an image in front of the user's eyes; One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.
9. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.
10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.