Keyboard input method for three-dimensional space, head-mounted display device, medium and product

Abstract

The embodiment of the invention discloses a keyboard input method for a three-dimensional space, head-mounted display equipment, a medium and a product. According to one specific embodiment, the method comprises the steps that in response to detection of typing operation acting on an associated entity keyboard, a candidate word list is determined according to at least one key position corresponding to the typing operation, an input cursor is displayed in a window corresponding to the typing operation, and the window is displayed in a three-dimensional space of the head-mounted display device; determining a spatial dimension of a window corresponding to the typing operation; in response to determining that the spatial dimension represents a two-dimensional window, generating candidate word column position information according to two-dimensional position information of the input cursor in the window; according to the candidate word column position information, a candidate word column is displayed in the window, and at least one candidate word in the candidate word list is displayed in the candidate word column. According to the embodiment, 3D space input can be supported, and the word selection box can be dynamically focused near the input box.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a keyboard input method, a head-mounted display device, a medium, and a product in a three-dimensional space. Background Art

[0002] With the development of head-mounted display device interaction technology, information can be input in the space system through a physical keyboard. At present, when inputting information in the space system through a physical keyboard, the commonly used methods are: in the AOSP-based space system, the general physical keyboard input uses the 2D virtual keyboard input method; or in the space system of the head-mounted display device, the extended input method service is fully used to implement the 3D virtual keyboard input method.

[0003] However, when the above method is adopted, the following technical problems often occur: when using the 2D virtual keyboard input method, 3D space input cannot be supported; when the 3D virtual keyboard input method is fully implemented using the extended input method service, the response speed is slow when inputting with the physical keyboard, and the word selection box cannot be dynamically focused near the input box.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a keyboard input method in three-dimensional space, a head-mounted display device, a computer-readable medium, and a computer program product to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a keyboard input method in a three-dimensional space, the method comprising: in response to detecting a typing operation acting on an associated physical keyboard, determining a candidate word list according to at least one key position corresponding to the typing operation, wherein an input cursor is displayed in a window corresponding to the typing operation, and the window is displayed in the three-dimensional space of the head-mounted display device; determining the spatial dimension of the window corresponding to the typing operation; in response to determining that the spatial dimension represents a two-dimensional window, generating candidate word column position information according to the two-dimensional position information of the input cursor in the window; displaying a candidate word column in the window according to the candidate word column position information, wherein at least one candidate word in the candidate word list is displayed in the candidate word column.

[0008] Optionally, the displaying of the candidate word bar in the window according to the candidate word bar position information includes: displaying the candidate word bar in the window according to the candidate word bar position information and a native input method service.

[0009] Optionally, the method also includes: in response to determining that the above-mentioned spatial dimension represents a three-dimensional window, displaying a spatially independent candidate word column in the above-mentioned three-dimensional space, wherein the displayed candidate word column corresponds to the three-dimensional application displayed in the above-mentioned window, and the displayed candidate word column displays at least one candidate word in the above-mentioned candidate word list.

[0010] Optionally, spatially independent candidate word columns are displayed in the three-dimensional space, including: generating candidate word column position information based on the position information of the input box corresponding to the input cursor; and displaying spatially independent candidate word columns in the three-dimensional space based on the generated candidate word column position information and the extended input method service.

[0011] Optionally, the method also includes: in response to detecting a selection operation on a candidate word displayed in the displayed candidate word column, determining an input channel corresponding to the selection operation; in response to determining that the input channel corresponds to a two-dimensional window, displaying the selected candidate word in a two-dimensional input box corresponding to the input cursor; in response to determining that the input channel corresponds to a three-dimensional window, displaying the selected candidate word in a three-dimensional input box corresponding to the input cursor.

[0012] Optionally, the method also includes: in response to detecting a click operation acting on an input box in the window, determining whether a physical keyboard is connected; in response to determining that no physical keyboard is connected, displaying a corresponding virtual keyboard corresponding to the above window in the above three-dimensional space, wherein the spatial hierarchy of the above virtual keyboard is independent.

[0013] Optionally, before determining a candidate word list in response to detecting a typing operation on an associated physical keyboard and based on at least one key position corresponding to the typing operation, the method further includes: establishing a message channel between the head-mounted display device and the extended input method service; and starting a native input method service in response to detecting a click operation on an input box in the window.

[0014] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0015] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0016] In a fourth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner of the above-mentioned first aspect when executed by a processor.

[0017] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the keyboard input method of the three-dimensional space of some embodiments of the present disclosure, 3D space input can be supported, and the word selection box can be dynamically focused near the input box. Specifically, the reason why 3D space input cannot be supported and the word selection box cannot be dynamically focused near the input box is that: when using the 2D virtual keyboard input method, 3D space input cannot be supported; when the scheme of the 3D virtual keyboard input method is fully implemented using the extended input method service, the response speed is slow when the physical keyboard is input, and the word selection box cannot be dynamically focused near the input box. Based on this, the keyboard input method of the three-dimensional space of some embodiments of the present disclosure, first, in response to detecting a typing operation acting on the associated physical keyboard, determines a list of candidate words according to at least one key position corresponding to the above typing operation. Among them, an input cursor is displayed in the window corresponding to the above typing operation, and the above window is displayed in the three-dimensional space of the above head-mounted display device. Thus, after the input cursor position in the window of the user's three-dimensional space triggers the typing operation, a list of candidate words for the user to select can be determined according to at least one key position typed. Then, the spatial dimension of the window corresponding to the above typing operation is determined. Thus, the spatial dimension of the window in which the user currently needs to input information can be determined to determine the display mode of the candidate word column. Then, in response to determining that the above spatial dimension represents a two-dimensional window, the candidate word column position information is generated according to the two-dimensional position information of the above input cursor in the above window. Thus, when the window is a two-dimensional window, the position information of the candidate word column to be displayed can be dynamically determined according to the two-dimensional position information of the input cursor. Finally, according to the above candidate word column position information, the candidate word column is displayed in the above window, wherein at least one candidate word in the above candidate word list is displayed in the above candidate word column. Thus, the candidate word column can be dynamically displayed in the two-dimensional window according to the dynamically determined position information of the candidate word column. Also, because the candidate word column can be displayed in a two-dimensional window in a three-dimensional space, 3D space input can be supported. Also, because when the candidate word column is displayed in the two-dimensional window, the position of the candidate word column dynamically positioned by the two-dimensional position information is adopted, so that there is no need to position the indicating cursor and the candidate word column in the three-dimensional space, and further, there is no need to fully use the input method service extended for the 3D space system, which improves the response speed of the physical keyboard input, realizes the dynamic display of the candidate word box according to the position of the input cursor, and makes the word selection box dynamically focused near the input box. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0019] Figure 1is a flow chart of some embodiments of a keyboard input method in a three-dimensional space according to the present disclosure;

[0020] Figure 2 are flow charts of other embodiments of the keyboard input method in three-dimensional space according to the present disclosure;

[0021] Figure 3 It is a schematic diagram of the structure of a head-mounted display device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0023] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0024] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0025] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0026] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0027] With regard to the collection, storage, and use of user personal information (such as key positions typed, candidate words selected, and content entered into an input box) involved in this disclosure, before performing the corresponding operations, the relevant organizations or individuals shall fulfill their obligations, including conducting personal information security impact assessments, fulfilling the obligation to inform the personal information subject, and obtaining the authorization and consent of the personal information subject in advance.

[0028] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0029] Figure 1The process 100 of some embodiments of the three-dimensional space keyboard input method according to the present disclosure is shown. The three-dimensional space keyboard input method includes the following steps:

[0030] Step 101, in response to detecting a typing operation acting on an associated physical keyboard, determining a candidate word list according to at least one key position corresponding to the typing operation.

[0031] In some embodiments, the execution subject of the keyboard input method in three-dimensional space (such as a head-mounted display device) can respond to the detection of a typing operation acting on an associated physical keyboard, and determine a candidate word list according to at least one key position corresponding to the typing operation. Among them, the physical keyboard can be a physical keyboard connected to the head-mounted display device. The connection method between the physical keyboard and the head-mounted display device can include but is not limited to at least one of the following: wired connection, Bluetooth, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultrawideband) connection, and other wireless connection methods currently known or developed in the future. The head-mounted display device can be worn on the head by the user to view the displayed content. The head-mounted display device can include but is not limited to: AR glasses, MR glasses, VR glasses. The typing operation can be an operation of pressing a key on the physical keyboard. The typing operation can correspond to at least one key position, and at least one key position has a sequence. The at least one key position includes the currently typed key position and the key position previously typed when inputting information this time. For example, the information to be input by the user this time can be "today". The input method used can be a pinyin input method. The key currently typed may be "t". The key previously typed may include: j, i, n. Then when "t" is typed, at least one key corresponding to the typing operation may include: j, i, n, t. An input cursor is displayed in the window corresponding to the above typing operation. The above input cursor may be displayed in an input box. After a user clicks in an input box of a window, an input cursor may be displayed in the input box. The input cursor may represent the position to be input. The input cursor may be displayed in the input box with a preset frequency. The above window is displayed in the three-dimensional space of the above head-mounted display device. The window may be a virtual screen for displaying content in the three-dimensional space of the head-mounted display device. The window in the three-dimensional space may be a two-dimensional window or a three-dimensional window. The two-dimensional window may be used to display two-dimensional display content. The three-dimensional window may be used to display three-dimensional display content. In practice, the above execution subject may use the candidate word determination function of the input method to determine a candidate word list according to at least one key corresponding to the above typing operation. For example, the candidate word list may be: today, gold bars, allowance, and momentum.

[0032] Optionally, before step 101, the execution subject may establish a message channel between the head-mounted display device and the extended input method service. The extended input method service may be deployed locally on the execution subject, or deployed locally and on the server. In practice, a link to the extended input method service may be declared in the running code of the head-mounted display device to establish a message channel between the head-mounted display device and the extended input method service. Secondly, the native input method service may be started in response to detecting a click operation acting on the input box in the window. Thus, it is possible to ensure that the extended input method service and the native input method service are available before the user types.

[0033] Optionally, the native input method service can be used to process but not limited to the following tasks: display of candidate word bars in a two-dimensional window, detection of typing operations, and detection of selection operations. The extended input method service can be used to process but not limited to the following tasks: display of candidate word bars corresponding to a three-dimensional window, spatial dimension judgment, determination of a candidate word list, and determination of a selected candidate word.

[0034] Step 102: determine the spatial dimension of the window corresponding to the input operation.

[0035] In some embodiments, the execution entity may determine the spatial dimension of the window corresponding to the input operation. The spatial dimension of the window may be the display dimension of the graphical user interface of the application displayed in the window. For example, when a 3D application is displayed in the window, the spatial dimension of the window may be three-dimensional, and the window may be a three-dimensional window. When a 2D application is displayed in the window, the spatial dimension of the window may be two-dimensional, and the window may be a two-dimensional window.

[0036] Step 103, in response to determining that the spatial dimension represents the two-dimensional window, generating candidate word column position information according to the two-dimensional position information of the input cursor in the window.

[0037] In some embodiments, in response to determining that the above-mentioned spatial dimension represents a two-dimensional window, the above-mentioned execution entity may generate candidate word column position information based on the two-dimensional position information of the above-mentioned input cursor in the above-mentioned window. The above-mentioned two-dimensional position information may be the two-dimensional coordinates of the above-mentioned input cursor in the above-mentioned window. For example, the above-mentioned two-dimensional position information may be the two-dimensional coordinates of the center point of the above-mentioned input cursor. In practice, the above-mentioned execution entity may move the vertical coordinate included in the above-mentioned two-dimensional position information downward by a preset distance to obtain the candidate word column position information. As an example, the preset distance may be 100, the above-mentioned two-dimensional position information may be (300, 100), and the candidate word column position information may be (300, 200). The candidate word column position information may be used to locate the center position of the candidate word column, or may be used to locate the upper left corner position of the candidate word column.

[0038] In some optional implementations of some embodiments, the execution subject may generate candidate word column position information according to the two-dimensional position information of the input cursor in the window through the following steps:

[0039] The first step is to perform downward offset processing on the vertical coordinate included in the two-dimensional position information to obtain offset two-dimensional position information. In practice, the execution subject may move the vertical coordinate included in the two-dimensional position information downward by a preset distance to obtain offset two-dimensional position information.

[0040] The second step is to determine the above offset two-dimensional position information as the initial upper left corner point of the above candidate word column.

[0041] The third step is to determine the initial upper right corner point, initial lower left corner point, and initial lower right corner point of the corresponding candidate word column according to the initial upper left corner point, the preset length, and the preset width corresponding to the candidate word column. In practice, the execution subject can calibrate the position of the candidate word column with the initial upper left corner point, the preset length, and the preset width to obtain the initial upper right corner point, the initial lower left corner point, and the initial lower right corner point.

[0042] The fourth step is to determine whether there is a corner point beyond the border of the window among the initial upper left corner point, the initial upper right corner point, the initial lower left corner point and the initial lower right corner point.

[0043] In step 5, in response to determining that the initial upper right corner point and the initial lower right corner point exceed the border of the window, determine the excess distance of the initial upper right corner point or the initial lower right corner point relative to the right border of the window.

[0044] Step 6: Generate a first adjustment distance based on the excess distance. In practice, the execution entity may determine the sum of the excess distance and the first preset adjustment distance as the first adjustment distance. The first preset adjustment distance may be greater than or equal to 0 and less than or equal to the difference between the length of the window and the preset length.

[0045] Step 7: Shift the initial upper left corner point, the initial upper right corner point, the initial lower left corner point and the initial lower right corner point to the left by the first adjustment distance, and obtain the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point as the candidate word column position information. Thus, when the candidate word column exceeds the right border of the window, the first adjustment distance can be adaptively adjusted to the left to avoid the candidate word column exceeding the right border of the window.

[0046] In step eight, in response to determining that the initial lower left corner point and the initial lower right corner point are beyond the border of the window, the distance between the initial upper left corner point and the top position of the input cursor is determined as a first distance.

[0047] In the ninth step, the sum of the preset width and the first distance is determined as the second distance.

[0048] Step 10: Generate a second adjustment distance based on the second distance. In practice, the execution entity may determine the sum of the second distance and the second preset adjustment distance as the second adjustment distance. The second preset adjustment distance may be greater than or equal to the length of the input cursor and less than or equal to the vertical distance between the upper border of the window and the initial upper left corner point.

[0049] In the eleventh step, the initial upper left corner point, the initial upper right corner point, the initial lower left corner point and the initial lower right corner point are translated upward by the second adjustment distance to obtain the upper left corner point, the upper right corner point, the lower left corner point and the lower right corner point as the candidate word column position information. Thus, when exceeding the lower border of the window, the second adjustment distance can be adaptively adjusted upward to prevent the candidate word column from exceeding the lower border of the window.

[0050] Step 104: display the candidate word column in the window according to the candidate word column position information.

[0051] In some embodiments, the execution entity may display a candidate word column in the window according to the candidate word column position information. The candidate word column displays at least one candidate word in the candidate word list. The candidate word column may display a preset number of candidate words or characters of a preset length, and the redundant candidate words may be displayed on the next page of the candidate word column. In practice, the execution entity may display the candidate word column at the position corresponding to the candidate word column position information in the window. Thus, the candidate word column may be displayed in a two-dimensional window.

[0052] Optionally, the execution subject can display the candidate word bar in the window according to the candidate word bar position information and the native input method service. The native input method service can be an input method service corresponding to a two-dimensional application. For example, the native input method service can be an Android input method service. The native input method service can be deployed locally, or deployed locally and on the server at the same time. Specifically, the candidate word bar position information can be used as an interface input parameter to call the candidate word bar display interface of the native input method service to display the candidate word bar in the window. Thus, the native input method service can be directly used to realize the display of the candidate word bar in the two-dimensional window, omitting the message channel between the head-mounted display device and the extended input method service and the time for drawing the three-dimensional candidate word bar, so that fast candidate word bar drawing can be realized, which improves the fluency of physical keyboard input.

[0053] Optionally, the execution subject may also determine the input channel corresponding to the selection operation in response to detecting a selection operation on the candidate words displayed in the displayed candidate word column. The input channel may characterize the window type corresponding to the input box. The interaction method corresponding to the selection operation may include but is not limited to at least one of the following: touch interaction, button interaction, gesture interaction, voice interaction, head control interaction, and eye movement interaction. Then, in response to determining that the input channel corresponds to a two-dimensional window, the selected candidate word may be displayed in the two-dimensional input box corresponding to the input cursor. Secondly, in response to determining that the input channel corresponds to a three-dimensional window, the selected candidate word may be displayed in the three-dimensional input box corresponding to the input cursor. Thus, the selected candidate words may be displayed for the two-dimensional window and the three-dimensional window, respectively.

[0054] Optionally, the above-mentioned execution entity may determine whether there is a physical keyboard connected in response to detecting a click operation acting on the input box in the window. The interaction method corresponding to the above-mentioned click operation may include but is not limited to at least one of the following: touch interaction, key interaction, gesture interaction, voice interaction, head control interaction, and eye movement interaction. Then, in response to determining that there is no physical keyboard connection, a corresponding virtual keyboard corresponding to the above-mentioned window may be displayed in the above-mentioned three-dimensional space. In practice, the above-mentioned execution entity may display a virtual keyboard in the upper layer of the above-mentioned window. Among them, the spatial hierarchy of the above-mentioned virtual keyboard is independent. Thus, when typing using the virtual keyboard, the virtual keyboard can be displayed in a three-dimensional display form to present a three-dimensional visual experience.

[0055] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the keyboard input method of the three-dimensional space of some embodiments of the present disclosure, 3D space input can be supported, and the word selection box can be dynamically focused near the input box. Specifically, the reason why 3D space input cannot be supported and the word selection box cannot be dynamically focused near the input box is that: when using the 2D virtual keyboard input method, 3D space input cannot be supported; when the scheme of the 3D virtual keyboard input method is fully implemented using the extended input method service, the response speed is slow when the physical keyboard is input, and the word selection box cannot be dynamically focused near the input box. Based on this, the keyboard input method of the three-dimensional space of some embodiments of the present disclosure, first, in response to detecting a typing operation acting on the associated physical keyboard, determines a list of candidate words according to at least one key position corresponding to the above typing operation. Among them, an input cursor is displayed in the window corresponding to the above typing operation, and the above window is displayed in the three-dimensional space of the above head-mounted display device. Thus, after the input cursor position in the window of the user's three-dimensional space triggers the typing operation, a list of candidate words for the user to select can be determined according to at least one key position typed. Then, the spatial dimension of the window corresponding to the above typing operation is determined. Thus, the spatial dimension of the window in which the user currently needs to input information can be determined to determine the display mode of the candidate word column. Then, in response to determining that the above spatial dimension represents a two-dimensional window, the candidate word column position information is generated according to the two-dimensional position information of the above input cursor in the above window. Thus, when the window is a two-dimensional window, the position information of the candidate word column to be displayed can be dynamically determined according to the two-dimensional position information of the input cursor. Finally, according to the above candidate word column position information, the candidate word column is displayed in the above window, wherein at least one candidate word in the above candidate word list is displayed in the above candidate word column. Thus, the candidate word column can be dynamically displayed in the two-dimensional window according to the dynamically determined position information of the candidate word column. Also, because the candidate word column can be displayed in a two-dimensional window in a three-dimensional space, 3D space input can be supported. Also, because when the candidate word column is displayed in the two-dimensional window, the position of the candidate word column dynamically positioned by the two-dimensional position information is adopted, so that there is no need to position the indicating cursor and the candidate word column in the three-dimensional space, and further, there is no need to fully use the input method service extended for the 3D space system, which improves the response speed of the physical keyboard input, realizes the dynamic display of the candidate word box according to the position of the input cursor, and makes the word selection box dynamically focused near the input box.

[0056] Further references Figure 2 , which shows a process 200 of another embodiment of a three-dimensional space keyboard input method. The process 200 of the three-dimensional space keyboard input method includes the following steps:

[0057] Step 201, in response to detecting a typing operation acting on an associated physical keyboard, determining a candidate word list according to at least one key position corresponding to the typing operation.

[0058] Step 202: determine the spatial dimension of the window corresponding to the input operation.

[0059] Step 203, in response to determining that the spatial dimension represents the two-dimensional window, generating candidate word column position information according to the two-dimensional position information of the input cursor in the window.

[0060] Step 204: display the candidate word column in the window according to the candidate word column position information.

[0061] In some embodiments, the specific implementation of steps 201-204 and the technical effects brought about can be referred to Figure 1 The corresponding steps 101-104 in the embodiments are not described in detail here.

[0062] Step 205 , in response to determining that the spatial dimension represents the three-dimensional window, displaying spatially hierarchically independent candidate word columns in the three-dimensional space.

[0063] In some embodiments, the execution body of the keyboard input method in three-dimensional space (such as a head-mounted display device) can display a candidate word column with independent spatial hierarchy in the above three-dimensional space in response to determining that the above spatial dimension represents a three-dimensional window. Among them, the displayed candidate word column corresponds to the three-dimensional application displayed in the above window. The displayed candidate word column displays at least one candidate word in the above candidate word list. The displayed candidate word column has independent spatial hierarchy. It can be understood that the candidate word column is displayed in the upper layer of the above window. In practice, the above execution body can display the candidate word column below the above window in the above three-dimensional space. The above execution body can also display the candidate word column at the center of the above window in the above three-dimensional space. In this way, the candidate word column can be displayed using a three-dimensional display method.

[0064] In some optional implementations of some embodiments, the above execution subject may perform the following steps:

[0065] The first step is to generate candidate word column position information based on the position information of the input box corresponding to the input cursor. For example, the position information of the input box can be represented by the coordinates of the center of the bottom edge of the input box. Here, there is no limitation on the position type of the input box represented by the position information of the input box. In practice, the execution entity can move the vertical coordinate included in the position information of the input box downward by a first preset distance, and increase the depth coordinate included in the position information by a second preset distance to obtain the candidate word column position information. Here, the candidate word column position information is a three-dimensional coordinate.

[0066] The second step is to display the candidate word column with independent spatial hierarchy in the above three-dimensional space according to the generated candidate word column position information and the extended input method service. Among them, the above extended input method service can be an extended service that extends the coordinate positioning and UI drawing in the three-dimensional space on the basis of the native input method service. For example, the above extended input method service can be an extended service that uses Unity to implement coordinate positioning and UI drawing in the three-dimensional space based on the Android input method service. In practice, the above execution subject can use the candidate word column position information as an interface input parameter to call the candidate word column display interface of the extended input method service to display the candidate word column with independent spatial hierarchy in the above three-dimensional space.

[0067] from Figure 2 It can be seen that Figure 1 Compared with the description of some corresponding embodiments, Figure 2 The process 200 of the keyboard input method in three-dimensional space in some corresponding embodiments reflects the extended steps of displaying the candidate word column corresponding to the three-dimensional window. Therefore, the schemes described in these embodiments can match the candidate word column display of the three-dimensional application to present a three-dimensional display effect with a sense of hierarchy.

[0068] Reference below Figure 3 , which shows a schematic structural diagram of a head-mounted display device 300 (eg, a head-mounted display device) suitable for implementing some embodiments of the present disclosure. Figure 3 The head mounted display device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0069] like Figure 3 As shown, the head mounted display device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the head mounted display device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0070] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a micro display, a speaker, a vibrator, etc.; and communication devices 309. The communication devices 309 may allow the head mounted display device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3The head mounted display device 300 having various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0071] A head mounted display device may include a display module that can be used to image in front of the user's eyes. The display module may include a micro display and an optical element.

[0072] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0073] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0074] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0075] The computer-readable medium may be included in the head-mounted display device; or it may exist independently without being assembled into the head-mounted display device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the head-mounted display device, the head-mounted display device: in response to detecting a typing operation acting on an associated physical keyboard, according to at least one key position corresponding to the typing operation, determines a candidate word list, wherein an input cursor is displayed in a window corresponding to the typing operation, and the window is displayed in the three-dimensional space of the head-mounted display device; determines the spatial dimension of the window corresponding to the typing operation; in response to determining that the spatial dimension represents a two-dimensional window, generates candidate word column position information according to the two-dimensional position information of the input cursor in the window; according to the candidate word column position information, displays a candidate word column in the window, wherein at least one candidate word in the candidate word list is displayed in the candidate word column.

[0076] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0077] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0078] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0079] Some embodiments of the present disclosure further provide a computer program product, including a computer program, which implements any of the above-mentioned three-dimensional space keyboard input methods when executed by a processor.

[0080] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A three-dimensional keyboard input method, applied to a head-mounted display device, comprising: In response to detecting a typing operation on an associated physical keyboard, determining a candidate word list according to at least one key position corresponding to the typing operation, wherein an input cursor is displayed in a window corresponding to the typing operation, and the window is displayed in a three-dimensional space of the head mounted display device; Determining the spatial dimension of the window corresponding to the input operation; In response to determining that the spatial dimension represents a two-dimensional window, generating candidate word column position information according to the two-dimensional position information of the input cursor in the window; The candidate word column is displayed in the window according to the candidate word column position information, wherein at least one candidate word in the candidate word list is displayed in the candidate word column.

2. The method according to claim 1, wherein: The step of displaying the candidate word column in the window according to the candidate word column position information includes: The candidate word column is displayed in the window according to the candidate word column position information and the native input method service.

3. The method according to claim 1, wherein: The method further comprises: In response to determining that the spatial dimension represents a three-dimensional window, a spatially independent candidate word column is displayed in the three-dimensional space, wherein the displayed candidate word column corresponds to the three-dimensional application displayed in the window, and the displayed candidate word column displays at least one candidate word in the candidate word list.

4. The method according to claim 3, wherein: The displaying of spatially hierarchically independent candidate word columns in the three-dimensional space includes: Generate candidate word column position information according to the position information of the input box corresponding to the input cursor; According to the generated candidate word column position information and the extended input method service, spatially independent candidate word columns are displayed in the three-dimensional space.

5. The method according to any one of claims 1 to 4, wherein: The method further comprises: In response to detecting a selection operation on a candidate word displayed in the displayed candidate word column, determining an input channel corresponding to the selection operation; In response to determining that the input channel corresponds to the two-dimensional window, displaying the selected candidate word in the two-dimensional input box corresponding to the input cursor; In response to determining that the input channel corresponds to a three-dimensional window, the selected candidate word is displayed in the three-dimensional input box corresponding to the input cursor.

6. The method according to claim 1, wherein: The method further comprises: In response to detecting a click operation acting on an input box in the window, determining whether a physical keyboard is connected; In response to determining that no physical keyboard is connected, a corresponding virtual keyboard corresponding to the window is displayed in the three-dimensional space, wherein the spatial hierarchy of the virtual keyboard is independent.

7. The method according to claim 1, wherein: Before determining the candidate word list in response to detecting a typing operation on the associated physical keyboard and according to at least one key position corresponding to the typing operation, the method further includes: Establishing a message channel between the head mounted display device and the extended input method service; In response to detecting a click operation acting on an input box in the window, a native input method service is started.

8. A head mounted display device, comprising: A display module, used to form an image in front of the user's eyes; one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images