Information input method and device, wearable equipment and computer readable storage medium

Abstract

The invention relates to the technical field of human-computer interaction, in particular to an information input method and device, wearable equipment and a computer readable storage medium. The information input method comprises the following steps: detecting a bearing medium in a shooting view field range of the wearable equipment; position information of the bearing medium is obtained, an input interface of the wearable device is displayed according to the position information of the bearing medium, and the input interface is located on the bearing medium; and in response to the received input operation in the input interface, determining input information according to the input operation. According to the information input method, the position displayed by the input interface is displayed on the bearing medium, in the information input process, the input information is determined through man-machine interaction, other entity input interfaces are not needed, and the information input method is more convenient for a user to use.

Description

Technical Field

[0001] The present application relates to the technical field of human-computer interaction, and particularly to an information input method, apparatus, wearable device, and computer-readable storage medium. Background Art

[0002] In existing wearable devices, such as AR / VR glasses, the text input function is a particularly important part of human-computer interaction. One of the existing implementation methods is to use cross-screen input on a mobile phone. Although such a text input method can achieve human-computer interaction through text input, during the use process, the user needs to use the input interface on the mobile phone as the text input terminal to input text to complete the text input on the AR / VR glasses side. That is to say, when completing the text input on the AR / VR glasses side, the user needs to wear AR / VR glasses and carry a mobile phone with text input function at the same time. Once the mobile phone is not with the user, the AR / VR glasses will not be able to achieve text input, making it inconvenient for the user to use and reducing the user experience. Therefore, in the process of realizing human-computer interaction through text input in wearable devices, how to make it more convenient for users to use wearable devices and realize the information input of wearable devices during the information interaction process has become an urgent problem to be solved. Summary of the Invention

[0003] The present application provides an information input method, apparatus, wearable device, and computer-readable storage medium, enabling users to more conveniently complete the information input on the wearable device side during the human-computer interaction process with the wearable device.

[0004] In a first aspect, the present application provides an information input method, the method comprising:

[0005] Detect a carrier medium within the shooting field of view of the wearable device;

[0006] Obtain the position information of the carrier medium,

[0007] According to the position information of the carrier medium, display an input interface of the wearable device on the carrier medium; in response to receiving an input operation in the input interface, determine input information according to the operation.

[0008] In a second aspect, the present application further provides an information input apparatus, the apparatus comprising:

[0009] A carrier medium detection module, which detects a carrier medium within the shooting field of view of the wearable device;

[0010] A position information acquisition module,

[0011] An input interface display module displays an input interface of the wearable device according to the position information of the carrying medium, and the input interface is located on the carrying medium.

[0012] An input information determination module responds to receiving an input operation in the input interface and determines input information according to the operation.

[0013] In a third aspect, the present application further provides a wearable device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the information input method as described above when executing the computer program.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the processor is caused to implement the information input method as described above.

[0015] The present application discloses an information input method, device, wearable device and computer-readable storage medium. By detecting a carrying medium within the shooting field of view of a shooting device of the wearable device; obtaining position information of the carrying medium, and then according to the position information of the carrying medium, displaying an input interface of the wearable device, wherein the input interface is located on the carrying medium; responding to receiving an input operation in the input interface and determining input information according to the input operation. The above information input method obtains the position information of the carrying medium, and according to the position information of the carrying medium, displays the input interface of the wearable device, where the position where the input interface is displayed is on the carrying medium. After positioning the input interface, the user can perform relevant input operations on the input interface and determine input information according to the operation. During the process of human-computer interaction between the user and the wearable device, by displaying a virtual input interface on the carrying medium and performing information input on the virtual input interface, without the need to rely on other physical input interfaces, it is more convenient for the user to use. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings below are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a schematic block diagram of the structure of a wearable device provided by an embodiment of the present application;

[0018] Figure 2 It is a schematic flowchart of an information input method provided by an embodiment of the present application;

[0019] Figure 3 is a schematic flowchart of another information input method provided by an embodiment of the present application;

[0020] Figure 4 is a schematic block diagram of another information input method provided by an embodiment of the present application;

[0021] Figure 5 is an effect diagram of the input interface display provided by an embodiment of the present application;

[0022] Figure 6 is a schematic block diagram of an information input device provided by an embodiment of the present application. Detailed implementation manners

[0023] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0024] The flowchart shown in the accompanying drawings is only an example illustration, and does not necessarily include all contents and operations / steps, nor does it necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged. Therefore, the actual execution order may be changed according to the actual situation.

[0025] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0026] It should also be understood that the term " / and / " used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0027] The embodiments of the present application provide an information input method, device, wearable device, and computer-readable storage medium. Among them, the information input method can be applied to a wearable device. By obtaining the position information of the carrier medium, according to the position information of the carrier medium, the input interface of the wearable device is displayed, where the input interface is located on the carrier medium, and the user inputs and displays information on the input interface. The input interface of the wearable device is a virtual input interface presented by the wearable device and does not require the aid of other physical input interfaces, making it more convenient for the user to use.

[0028] Among them, the wearable device can be a wearable device such as AR glasses, smart bracelet or smart watch.

[0029] Please refer to Figure 1 , Figure 1 , which is a schematic block diagram of the structure of a wearable device 100 provided by an embodiment of the present application. In Figure 1 , the wearable device 100 includes a processor 1001 and a memory 1002. Among them, the processor 1001 and the memory 1002 are connected by a bus, and the bus can be any applicable bus such as an Inter-integrated Circuit (I2C) bus.

[0030] Among them, the memory 1002 can include a storage medium and an internal memory. The storage medium can be a volatile storage medium or a non-volatile storage medium. The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor 1001 can be made to execute any one of the information input methods in the embodiments of the present application.

[0031] The processor 1001 is used to provide computing and control capabilities to support the operation of the entire wearable device 100.

[0032] Among them, the processor 1001 can be a Central Processing Unit (CPU), and the processor can also be a general-purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components and other types of processors. The general-purpose processor can be a microprocessor or, the general-purpose processor can also be any conventional processor, etc.

[0033] In addition to including a processor and a memory, the wearable device 100 also includes a camera, a depth sensor, a display device, etc. Among them, the camera can also be called a shooting device, and the shooting device is used to shoot the surrounding environment and the user of the wearable device, etc., such as for shooting within its shooting field of view to determine whether there is a preset carrying medium within its shooting field of view. The carrying medium can be, for example, the user's palm, a book or a writing board, etc. The depth sensor is, for example, a depth camera or a binocular camera, and is used to measure the distance from the target object (such as the carrying medium) to the wearable device. The display device is used to display information, such as for displaying an input interface and input information.

[0034] In some embodiments, the wearable device may be a smart watch, a smart bracelet, AR glasses, etc. Taking AR glasses as an example, the AR glasses include a camera, a depth sensor, a display device, etc. The display device is specifically the waveguide display module of the AR glasses, and is used to present virtual information to the user.

[0035] Wherein, the processor 1001 is configured to run a computer program stored in the memory 1002, and when executing the computer program, implement the following steps:

[0036] Detect a bearing medium within the shooting field of view of the shooting device of the wearable device; obtain the position information of the bearing medium, and according to the position information of the bearing medium, display an input interface of the wearable device, where the input interface is located on the bearing medium; in response to receiving an input operation in the input interface, determine input information according to the operation.

[0037] In one embodiment, the processor 1001 is further configured to implement:

[0038] Obtain first indication information of the indication end of the user using the input indicator, where the first indication information of the indication end is the distance from the indication end to the wearable device; if the difference between the first indication information of the indication end and the first position information of the bearing medium is within a preset range, determine that the indication end of the input indicator touches the input interface located on the bearing medium; obtain second indication information of the indication end on the input interface, and determine input information according to the second indication information of the indication end on the input interface.

[0039] In one embodiment, before the processor 1001 implements obtaining the position information of the bearing medium, it is further configured to implement:

[0040] Obtain the residence time of the bearing medium within the shooting field of view; if the residence time is greater than or equal to the preset residence time of the bearing medium, execute the step of obtaining the position information of the bearing medium.

[0041] In one embodiment, when the processor 1001 implements displaying the input interface of the wearable device according to the position information of the bearing medium, it is further configured to implement:

[0042] If the first position information of the bearing medium is within a preset distance range and the second position information of the bearing medium is within a preset position range, display the input interface of the wearable device, where the position information of the bearing medium includes first position information and second position information, the preset distance range is determined according to the distance from the bearing medium to the wearable device, and the preset position range is the corresponding position range within the shooting field of view of the wearable device.

[0043] In one embodiment, after the processor 1001 implements determining the input information, it is further configured to implement:

[0044] Obtain input information and display the input information in the display box on the input interface.

[0045] In one embodiment, after the processor 1001 implements obtaining the user's operation on the input interface and determining the input information according to the operation, it is further used to implement:

[0046] If it is detected that the carrier medium moves out of the shooting field of view, or if the state of the carrier medium changes, hide the input interface.

[0047] In one embodiment, the processor 1001 is further used to implement:

[0048] Obtain the user's gesture actions; if the user's gesture actions match the preset input gesture actions, control the wearable device to enter the input interface mode and detect the carrier medium within the shooting field of view of the wearable device; if the user's gesture actions match the preset exit gesture actions, control the wearable device to exit the input interface mode.

[0049] The following will describe in detail some embodiments of the present application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0050] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of an information input method provided by an embodiment of the present application. This information input method can be applied to a wearable device.

[0051] As Figure 2 shown, this information input method specifically includes steps S201 to S204.

[0052] S201. Detect the carrier medium within the shooting field of view of the wearable device.

[0053] Specifically, the user can trigger the wearable device to start shooting through methods such as buttons, voice, gesture actions, etc., and detect the carrier medium within the field of view of the wearable device. For example, the user can gently touch the start shooting button of the wearable device to make the wearable device enter the shooting state. The wearable device captures an image within its shooting field of view, and according to the result of the captured image, the wearable device detects whether there is a carrier medium in the captured image. If the wearable device detects a carrier medium in the captured image, it continues to execute the subsequent steps of the information input method. If the wearable device detects no carrier medium in the captured image, it terminates the execution of the subsequent steps of the information input method.

[0054] It should be noted that the carrier medium here is the carrier medium preset by the wearable device. When the carrier medium in the captured image is the same as the preset carrier medium, the wearable device determines that there is a carrier medium within the captured field of view. Conversely, if there is no carrier medium in the captured image that matches the preset carrier medium, the wearable device will determine that there is no carrier medium within the captured field of view. In addition, the user can also use voice. For example, the user issues a voice command: "I want to take a picture". After receiving the above voice command issued by the user, the wearable device automatically takes a picture within the captured field of view. According to the result of the captured image after shooting, the wearable device detects whether there is a carrier medium in the captured image. The specific process of the wearable device detecting whether there is a carrier medium in the captured image is the same as the above button method and will not be elaborated here.

[0055] Exemplarily, the carrier medium can be the user's palm, cardboard, book, etc.; the wearable device can be any wearable device such as augmented reality (AR) glasses, virtual reality (VR) glasses, smart watch, smart bracelet, etc.

[0056] In some embodiments, before detecting the carrier medium within the captured field of view of the wearable device, the gesture action of the user is obtained; if the gesture action of the user matches the preset input gesture action, the wearable device is controlled to enter the input interface mode, and the carrier medium within the captured field of view of the wearable device is detected. Among them, the preset input gesture action can be two palm pinches or other reasonable preset input gesture actions. For example, if the gesture action of the user is two palm pinches, then the gesture action of the user matches the preset input gesture action. At this time, the wearable device is controlled to enter the input interface mode, and the carrier medium within the captured field of view of the wearable device is detected; conversely, if the gesture action of the user is two fist clenches or any other gesture action that is not two palm pinches, then the gesture action of the user does not match the preset input gesture action. At this time, the wearable device does not perform the operation of entering the input interface mode, and the information input method ends.

[0057] Exemplarily, the captured field of view of the wearable device can be the captured field of view detected by the camera within the wearable device. The captured field of view can be within a range where the captured field of view angle is 60° or 80° and the distance from the wearable device is 70 cm or 80 cm. Here, the specific value of the field of view angle within the captured field of view and the detectable distance of the wearable device are not limited and can be reasonably set according to the needs of the actual application scenario.

[0058] Specifically, the wearable device detects the bearing medium within its own shooting field of view. If the wearable device detects the bearing medium within its own shooting range, it continues to execute the subsequent steps of the information input method. If the wearable device does not detect the bearing medium within its own shooting range, the execution process of the information input method is terminated.

[0059] S202. Obtain the position information of the bearing medium.

[0060] Exemplarily, the position information of the bearing medium can be obtained based on a depth sensor through triangulation or the like. Among them, the depth sensor can be a depth camera or a binocular camera.

[0061] Exemplarily, the position information of the bearing medium can be obtained by the wearable device shooting the bearing medium. By shooting the bearing medium, the wearable device obtains the position in the captured image of the bearing medium.

[0062] In some embodiments, if there is a bearing medium within the shooting field of view, before obtaining the position information of the bearing medium, obtain the residence time of the bearing medium within the shooting field of view; if the residence time is greater than or equal to the preset residence time of the bearing medium, execute the step of obtaining the position information of the bearing medium. If the residence time is less than the preset residence time of the bearing medium, stop executing the step of obtaining the position information of the bearing medium. Among them, the preset residence time can be any appropriate time set according to actual scene needs, such as 2s or 3s. For example, the preset residence time of the bearing medium is 3s. If the wearable device obtains that the residence time of the bearing medium within the shooting field of view is 5s, then the residence time of 5s of the bearing medium within the shooting field of view is greater than the preset residence time of 3s of the bearing medium, and the step of obtaining the position information of the bearing medium is executed; if the wearable device obtains that the residence time of the bearing medium within the shooting field of view is 2s, then the residence time of 2s of the bearing medium within the shooting field of view is less than the preset residence time of 3s of the bearing medium, and the step of obtaining the position information of the bearing medium is stopped.

[0063] Exemplarily, the position information of the bearing medium includes first position information and second position information. Among them, the first position information is the distance from the bearing medium to the wearable device; the second position information is the position of the bearing medium in the image captured by the wearable device.

[0064] Exemplarily, the first position information of the bearing medium is the distance from the bearing medium to the wearable device. For example, the first position information of the bearing medium can be 40 cm or 45 cm or any other appropriate distance. Further, assuming that the bearing medium here is the user's palm and the wearable device is an AR glasses, according to the appropriate viewing distance from the eyes to the palm of the general user, the first position information between the palm and the wearable device can be determined as 40 cm. Of course, it can be understood that if the user is a teenager or a user with a smaller or taller body, etc., the corresponding first position information can also be reasonably set, which is not limited here.

[0065] Exemplarily, the second position information is the position of the bearing medium in the image captured by the wearable device. For example, the position in the image captured by the wearable device can be any appropriate position in the image, such as the lower left corner, upper left corner, upper right corner, lower right corner, etc. captured by the imaging device. When the bearing medium is the palm and the palm exists within the shooting field of view of the wearable device, the wearable device captures the palm to obtain the position of the captured palm image, and the wearable device acquires the second position information of the palm image.

[0066] S203. Display the input interface of the wearable device according to the position information of the bearing medium, and the input interface is located on the bearing medium.

[0067] Taking the wearable device as an AR glasses as an example, according to the position information of the bearing medium, the input interface is displayed through the waveguide display module of the AR glasses, and the input interface is located on the bearing medium. Taking the bearing medium as the palm as an example, specifically, the position information of the palm is used to make the displayed input interface match the size of the palm and be exactly displayed on the palm.

[0068] In some embodiments, if the first position information of the bearing medium is within a preset distance range and the second position information is within a preset position range, the input interface of the wearable device is displayed, where the preset distance range is determined according to the distance from the bearing medium to the wearable device, and the preset position range is the corresponding position range within the shooting field of view of the wearable device.

[0069] Exemplarily, when the bearing medium is the palm and the wearable device is AR glasses, the preset distance range is the distance range determined according to the palm. Specifically, it can be the distance range determined according to factors such as the user's arm length and the comfort level when the user wears AR glasses to observe the palm. Considering that when the user's palm is too close to the AR glasses, it is uncomfortable for the user to wear AR glasses to view the palm and it is not convenient for the user to perform the corresponding input operation. When the user's palm is too far from the user, when the user wears AR glasses, the palm cannot be detected within the shooting field of view of the AR glasses and it is not convenient for input operations when inputting information later. The preset distance range can be set within 30 cm to 60 cm or 35 cm to 65 cm or any appropriate distance range. Further, when the bearing medium is the palm and the wearable device is AR glasses, when the preset distance range is 30 cm to 60 cm, the first position information of the palm detected by the AR glasses is 40 cm. At this time, the first position information of the palm is within the preset distance range. Then the AR glasses determine the position information of the palm. If the position information is within the preset position range, the input interface of the AR glasses is displayed. On the contrary, when the preset distance range is 30 cm to 60 cm, the first position information of the palm detected by the AR glasses is 20 cm. At this time, the first position information of the palm is not within the preset distance range. At this time, the information input ends, or the distance between the palm and the AR glasses is adjusted, and the AR glasses re-obtain the first position information of the palm and re-determine whether the first position information of the palm is within the preset distance range. It should be noted that the arm lengths of different users may be different, and the preset distance range here can be reasonably set according to factors such as different user arm lengths.

[0070] Exemplarily, when the bearing medium is the palm and the wearable device is AR glasses, the preset position range is the corresponding position range within the shooting field of view of the AR glasses. Specifically, it can be the preset position range corresponding to a shooting field of view angle of 70° or 80° of the AR glasses. If the palm is within the preset position range, the input interface of the AR glasses is displayed. According to the position of the displayed input interface, if the palm is at the position where the input interface is located, the input interface is directly on the palm. If the palm is not at the position where the input interface is located, the palm can move in the direction of the position of the input interface, or the AR glasses can rotate in the direction of the palm, so that the input interface moves in the direction of the position of the palm until the input interface is on the palm. If the position information of the palm is not within the preset position range, that is, the palm is not detected within the shooting field of view of the AR glasses, this step ends, or the palm can be moved, or the AR glasses can be rotated until the position information of the palm is within the preset position range.

[0071] It should be noted that the position of the palm and the preset position range here are both variable. The purpose is to detect the palm within the preset distance range, and then the palm bears the input interface.

[0072] Exemplarily, if the first position information of the bearing medium is within a preset distance range and the second position information of the bearing medium is within a preset position range, the input interface of the wearable device is displayed. Taking the palm as the bearing medium and the AR glasses as the wearable device as an example, for instance, the known preset distance range is 30 cm to 60 cm, and the preset position range is the corresponding preset position range when the shooting field of view angle of the AR glasses is 75°. If the first position information of the palm is 42 cm, then the first position information with the palm as the bearing medium is within the preset distance range. Next, it is determined whether the second position information of the palm is within the preset position range. If the second position information of the palm is within the preset position range corresponding to the shooting field of view angle of 75°, the input interface of the AR glasses is displayed. Specifically, if the position of the palm can exactly display the input interface at this time, the input interface of the AR glasses is directly displayed on the palm. If the position of the palm is not at the display position of the input interface, the user can adjust the specific position of the palm or rotate the shooting field of view range of the AR glasses to display the input interface of the AR glasses on the palm.

[0073] For another example, the known preset distance range is 35 cm to 65 cm, and the first position information of the palm is 80 cm. At this time, the first position information of the palm is not within the preset distance range, so the input interface of the AR glasses is not displayed.

[0074] It should be noted that if the first position information of the palm is within the preset distance range, the AR glasses can automatically adjust the size of the input interface according to the value of the first position information, adjust the input interface to fit the size of the palm center, and display the input interface at the palm center through the imaging principle.

[0075] S204. In response to receiving the input operation in the input interface, determine the input information according to the input operation.

[0076] Among them, the input operation of the user on the input interface can be in ways such as clicking, swiping, touching, etc. For example, if the user performs a clicking operation on the input interface, the wearable device obtains the first indication information and the second indication information of the user clicking on the input interface, and determines the input information according to the first indication information and the second indication information.

[0077] In some embodiments, to obtain the first indication information of the indication end of the input indicator used by the user, the first indication information can be obtained by a depth camera or a binocular RGB camera on the wearable device through methods such as triangulation. If the difference between the first indication information of the indication end and the first position information of the bearing medium is within a preset range, where the preset range can be (-1 mm ∽ 1 mm) or (-1.2 mm ∽ 0.9 mm) or any other suitable range; then it is determined that the indication end of the input indicator touches the input interface located on the bearing medium; the wearable device captures an image of the specific position where the indication end touches the input interface, and according to the position of the indication end in the image captured by the wearable device, obtains the second indication information of the indication end on the input interface. According to the second indication information of the indication end on the input interface, the input information is determined, where the first indication information of the indication end is the distance from the indication end to the wearable device, and the second indication information of the indication end is the specific position where the indication end touches the input interface.

[0078] It should be noted that the input indicator can be any suitable input indicator such as a finger or a pen, and the indication end can be any suitable indication end such as a fingertip or a pen tip, and there is no limitation on this.

[0079] Exemplarily, when the input indicator is the user's finger, the first indication information of the fingertip of the user's finger is obtained, where the first indication information of the fingertip is the distance from the finger to the wearable device. If the difference between the first indication information of the fingertip and the first position information of the bearing medium is within the preset range, it is determined that the fingertip of the finger touches the input interface located on the bearing medium. For example, when the bearing medium is the palm, the preset range is (-1 mm ∽ 1 mm). When the first indication information of the fingertip is 20.1 cm and the first position information of the palm is 20 cm, the difference between the first position information of the fingertip and the first position information of the palm is 0.1 cm, that is, 1 mm. At this time, the difference between the first indication information of the fingertip and the first position information of the palm is within the preset range, and it is determined that the fingertip of the finger touches the input interface located on the palm. The wearable device can obtain the second indication information of the indication end on the input interface. On the contrary, when the bearing medium is the palm, when the first indication information of the fingertip is 20.5 cm and the first position information of the palm is 20 cm, the difference between the first indication information of the fingertip and the first position information of the palm is 0.5 cm, that is, 5 mm. At this time, the difference between the first indication information of the fingertip and the first position information of the palm is not within the preset range, and it is determined that the fingertip of the finger does not touch the input interface located on the palm, and the wearable device cannot obtain the second indication information of the indication end on the input interface.

[0080] Exemplarily, obtain second indication information of an indicating end on an input interface, and determine input information according to the second indication information of the indicating end on the input interface. For example, when the indicating end is a fingertip, if the fingertip touches MNO, GHI, GHI, ABC, MNO, MNO, ABC on the nine-key pinyin in sequence, the specific positions of the fingertip on the input interface captured by the wearable device in sequence are the positions where MNO, GHI, GHI, ABC, MNO, MNO, ABC are located, that is, the second indication information of the fingertip on the input interface is the positions where MNO, GHI, GHI, ABC, MNO, MNO, ABC are located in sequence. According to the second indication information of the fingertip on the input interface, the wearable device determines that the input information is "How are you?".

[0081] An information input method provided in the above embodiment detects a carrier medium within the shooting field of view of the wearable device, obtains position information of the carrier medium, and according to the position information of the carrier medium, wherein the position information of the carrier medium includes first position information and second position information, displays an input interface of the wearable device. In response to receiving an input operation in the input interface, determine input information according to the input operation. By displaying a virtual input interface on the carrier medium, it is more convenient for users to use.

[0082] Please refer to Figure 3 , Figure 3 is a schematic flowchart of another information input method provided in an embodiment of the present application. This information input method specifically includes steps S301 to S304.

[0083] S301. In response to receiving an input operation in the input interface, determine input information according to the input operation.

[0084] Exemplarily, the operations performed by the user on the input interface can be in the forms of clicking, swiping, touching, etc.

[0085] Exemplarily, if the input indicating member is a pen held by the user's hand and the carrier medium is the user's palm, the preset range is (-1.2 mm ∽ 0.9 mm). When the first indication information of the pen tip is 20.1 cm, the first position information of the palm is 20 cm, and the difference between the first indication information of the pen tip and the first position information of the palm is 0.1 cm, that is, 1 mm. At this time, the difference between the first indication information of the pen tip and the first position information of the palm is within the preset range, then it is determined that the pen tip touches the input interface located on the palm, and the wearable device can obtain the second indication information of the indicating end on the input interface. Conversely, if the difference between the first indication information of the pen tip and the first position information of the palm is not within the preset range, the wearable device cannot obtain the second indication information of the indicating end on the input interface.

[0086] S302. Obtain the input information and display the input information in the display box of the input interface.

[0087] Exemplarily, after the wearable device determines the input information, it displays the input information in the display box of the input interface. For example, when the wearable device determines that the input information is "How are you", the input information "you", "are", and "you" are sequentially displayed in the display box of the input interface.

[0088] S303. If it is detected that the carrier medium moves out of the shooting field of view, or the state of the carrier medium changes, hide the input interface.

[0089] Exemplarily, after obtaining the user's operation on the input interface and determining the input information according to the operation, if it is detected that the carrier medium moves out of the shooting field of view, or the state of the carrier medium changes, hide the input interface. Among them, the change in the state of the carrier medium can be that the carrier medium changes from an open state to a closed state or any other state that cannot display the input interface. Hiding the input interface can be achieved by the wearable device performing operations such as minimizing the input interface or exiting the input interface.

[0090] S304. Exit the input interface mode.

[0091] Exemplarily, the input interface mode can be exited by means such as buttons, voice, gestures, etc.

[0092] In some embodiments, the user's gesture action can be obtained again. If the user's gesture action matches the preset exit gesture action, control the wearable device to exit the input interface mode. Among them, the preset exit gesture action can be two fist clenches or other reasonable preset exit gesture actions. For example, if the user's gesture action is two fist clenches, then the user's gesture action matches the preset exit gesture action, and at this time, control the wearable device to exit the input interface mode. Conversely, if the user's gesture action is any other gesture action other than two fist clenches, then the user's gesture action does not match the preset exit gesture action, and at this time, do not exit the input interface mode.

[0093] Another information input method provided by the above embodiments obtains the input information by obtaining the user's input operation on the input interface, and displays the input information in the display box of the input interface. If it is detected that the carrier medium moves out of the shooting field of view, or the state of the carrier medium changes, hide the input interface, and finally, perform the operation of exiting the input interface mode. In this method, when the user completes information input, the input interface mode of the wearable device can be exited in a timely manner through the human-computer interaction based on the user's gesture action, which is convenient for operation and facilitates the user to perform operations.

[0094] Specifically, here, taking the user using the AR glasses device and the user's palm as an example, the implementation process of the information input method is specifically analyzed. Please refer to Figure 4 ,Figure 4 It is a schematic block diagram of another information input method provided by an embodiment of the present application, which may specifically include the following steps S401 to S406.

[0095] S401. Detect a carrier medium within the shooting field of view of the wearable device.

[0096] Exemplarily, the user can touch the AR glasses leg lightly or click the control button of the AR glasses to turn on the camera on the AR glasses. Obtain the user's gesture actions within the camera on the AR glasses; if the user's gesture actions match the preset input gesture actions, control the AR glasses to enter the input interface mode, and detect the palm within the shooting field of view of the AR glasses. If the shooting field of view of the AR glasses is within a range with a shooting field angle of 50° and a distance of 70 cm from the camera, if a palm is detected within the shooting field of view with a shooting field angle of 50° and a distance of 70 cm from the camera, then continue to execute the subsequent steps of the information input method. If no palm is detected within the shooting field of view with a shooting field angle of 50° and a distance of 70 cm from the shooting device, then terminate the execution process of the information input method.

[0097] S402. Obtain the residence time of the carrier medium within the shooting field of view; if the residence time is greater than or equal to the preset residence time of the carrier medium, then execute the step of obtaining the position information of the carrier medium.

[0098] Exemplarily, obtain the residence time of the palm within the shooting field of view. If it is detected that the residence time of the palm within the shooting field of view is 4 s and the preset residence time is 3 s, then the residence time of the palm within the shooting field of view is greater than the preset residence time, and then execute the step of obtaining the position information of the carrier medium. On the contrary, if it is detected that the residence time of the palm within the shooting field of view is 2 s and the preset residence time is 3 s, then the residence time of the palm within the shooting field of view is less than the preset residence time, and then do not execute the step of obtaining the position information of the carrier medium, and end the information input method.

[0099] S403. Obtain the position information of the carrier medium.

[0100] Exemplarily, the first position information of the palm obtained can be 40 cm. Here, the first position information of the palm is also the depth information, indicating the distance from the palm to the AR glasses. The second position information of the palm obtained can be the position where the palm is located in the palm image captured by the AR glasses.

[0101] S404. Display the input interface of the wearable device according to the position information of the carrier medium.

[0102] Exemplarily, according to the first position information of the palm at 40 cm and the second position information of the palm, where the preset distance range is 30 cm to 60 cm, the first position information is within the preset distance range. Then, it is determined whether the second position information of the palm is within the preset position range. Assume that the preset position range is the corresponding preset position range when the shooting field of view angle of the AR glasses is 80°. If the second position information of the palm is within the preset position range when the shooting field of view angle is 80°, and if the position of the palm can exactly display the input interface at this time, the input interface of the AR glasses is directly displayed on the palm. If the position of the palm is different from the position of the input interface at this time, the input interface of the AR glasses can be moved by moving the palm or the user turning the head. After the palm or the input interface is moved, the palm can exactly display the input interface. If the position information of the palm is not within the above preset position range, the user can adjust the specific position of the palm or rotate the AR glasses to change the shooting field of view of the AR glasses camera, and display the input interface of the AR glasses on the palm. It should be noted that the position information of both the palm and the input interface of the AR glasses is not fixed. The palm can change with the change of the position information of the input interface of the AR glasses, or the position information of the input interface of the AR glasses can be changed to adapt to the position of the palm.

[0103] S405. In response to receiving an input operation in the input interface, determine input information according to the input operation.

[0104] Exemplarily, obtain the first indication information of the fingertip of the finger of the user's other hand. Assume that the first indication information of the fingertip is 20.1 cm, and the preset range is (-1 mm to 1 mm). Then, the difference between the first indication information of the fingertip of the other hand and the first position information of the palm is 0.1 cm, that is, 1 mm. At this time, the difference between the first indication information of the fingertip of the other hand and the first position information of the palm is within the preset range, so it is determined that the fingertip of the finger touches the input interface located on the palm.

[0105] Exemplarily, obtain the second indication information of the fingertip on the input interface, and determine the input information according to the second indication information of the fingertip on the input interface. For example, if the second indication information of the fingertip on the input interface is successively at the positions of MNO, GHI, GHI, ABC, MNO, MNO, ABC in the pinyin nine-key, then according to the second indication information of the fingertip on the input interface, the input information can be determined as "How are you".

[0106] Exemplarily, after determining the input information, the input information "you", "hao", and "ma" are successively displayed in the display box of the input interface. For the specific display effect of the text input of the AR glasses and the palm, please refer to Figure 5 .

[0107] S406. If it is detected that the carrying medium has moved out of the shooting field of view, or if the state of the carrying medium has changed, hide the input interface.

[0108] Exemplarily, after obtaining the user's operation on the input interface and determining the input information according to the operation, if it is detected that the palm has moved out of the shooting field of view, or if the state of the palm has changed from an open state to a clenched fist state or other changes occur and the input interface cannot be displayed, hide the input interface. At this time, the AR glasses obtain the user's gesture actions. If the user's gesture actions are two clenched fists and match the preset exit gesture actions, control the AR glasses to close the input interface mode.

[0109] The information input method for the interaction between an AR glasses device and a palm provided in the above embodiment describes the specific process of the interaction between the AR glasses and the palm. From the AR glasses entering the input interface mode by obtaining the user's gesture operation, then realizing information input through the interaction between the AR glasses and the palm and fingers until the AR glasses close the input interface mode. During the entire information input process of the interaction between the AR glasses device and the palm, no other physical input interface is involved, and the virtual input interface is presented on the carrying medium such as the palm, and the user can conveniently perform the operation of information input.

[0110] Please refer to Figure 6 , Figure 6 FIG. is a schematic block diagram of an information input device provided by an embodiment of the present application. The information input device is used to execute the foregoing information input method. Among them, the information input device can be configured in a wearable device.

[0111] As Figure 6 shown, the information input device 600 includes: a carrying medium detection module 601, a position information acquisition module 602, an input interface display module 603, and an input information determination module 604.

[0112] The carrying medium detection module 601 is used to detect the carrying medium within the shooting field of view of the wearable device.

[0113] The position information acquisition module 602 is used to acquire the position information of the carrying medium.

[0114] The input interface display module 603 is used to display the input interface of the wearable device according to the position information of the carrying medium, and the input interface is located on the carrying medium.

[0115] The input information determination module 604 is used to respond to the input operation received in the input interface and determine the input information according to the input operation.

[0116] In some embodiments, the carrying medium detection module 601 is specifically used for:

[0117] Obtain the user's gesture action; if the user's gesture action matches the preset input gesture action, control the wearable device to enter the input interface mode, and detect the carrier medium within the shooting field of view of the wearable device.

[0118] In some embodiments, the position information acquisition module 602 is specifically configured to: obtain the residence time of the carrier medium within the shooting field of view; if the residence time is greater than or equal to the preset residence time of the carrier medium, perform the step of obtaining the position information of the carrier medium.

[0119] In some embodiments, the input interface display module 603 is specifically configured to:

[0120] According to the position information of the carrier medium, display the input interface of the wearable device, and the input interface is located on the carrier medium. If it is detected that the carrier medium moves out of the shooting field of view, or the state of the carrier medium is detected to change, hide the input interface.

[0121] In some embodiments, the input interface display module 603 is further specifically configured to:

[0122] If the first position information of the carrier medium is within the preset distance range and the second position information of the carrier medium is within the preset position range, display the input interface of the wearable device, where the position information of the carrier medium includes the first position information and the second position information, the preset distance range is determined according to the distance from the carrier medium to the wearable device, and the preset position range is the position range corresponding to the shooting field of view of the wearable device.

[0123] In some embodiments, the input information determination module 604 is specifically configured to:

[0124] Obtain the user's operation on the input interface, and determine the input information according to the operation.

[0125] In some embodiments, the input information determination module 604 is further specifically configured to:

[0126] Obtain the input information and display the input information in the display box of the input interface. Obtain the user's gesture action, and if the user's gesture action matches the preset exit gesture action, control the wearable device to exit the input interface mode.

[0127] In some embodiments, the input information determination module 604 is further specifically configured to:

[0128] Obtain first indication information of the indication end of the input indicator used by the user, where the first indication information of the indication end is the distance from the indication end to the wearable device; if the difference between the first indication information of the indication end and the first position information of the bearing medium is within a preset range, it is determined that the indication end of the input indicator touches the input interface located on the bearing medium; obtain second indication information of the indication end on the input interface, and determine input information according to the second indication information of the indication end on the input interface.

[0129] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described device and each module can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0130] The above-mentioned device can be implemented in the form of a computer program, and the computer program can run on a wearable device as shown in Figure 1 shown.

[0131] In an embodiment of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program, and the computer program includes program instructions. The processor executes the program instructions to implement any one of the information input methods provided in the embodiments of the present application.

[0132] Among them, the computer-readable storage medium can be an internal storage unit of the computer device in the foregoing embodiment, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0133] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and these modifications or substitutions should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An information input method, characterized in that, applied to a wearable device, comprising: detecting a carrier medium within the shooting field of view of the wearable device; acquiring position information of the carrier medium; displaying an input interface of the wearable device according to the position information of the carrier medium, the input interface being located on the carrier medium; responding to an input operation received in the input interface, and determining input information according to the input operation.

2. The information input method according to claim 1, characterized in that, the step of responding to an input operation received in the input interface and determining input information according to the input operation comprises: acquiring first indication information of an indication end of a user using an input indicator, wherein the first indication information of the indication end is the distance from the indication end to the wearable device; if the difference between the first indication information of the indication end and the first position information of the carrier medium is within a preset range, determining that the indication end of the input indicator touches the input interface located on the carrier medium; acquiring second indication information of the indication end on the input interface, and determining input information according to the second indication information of the indication end on the input interface.

3. The information input method according to claim 1, characterized in that, before the step of acquiring the position information of the carrier medium, it comprises: acquiring the residence time of the carrier medium within the shooting field of view; if the residence time is greater than or equal to a preset residence time of the carrier medium, performing the step of acquiring the position information of the carrier medium.

4. The information input method according to claim 1, characterized in that, the step of displaying an input interface of the wearable device according to the position information of the carrier medium comprises: if the first position information of the carrier medium is within a preset distance range and the second position information of the carrier medium is within a preset position range, displaying an input interface of the wearable device, wherein the position information of the carrier medium includes first position information and second position information, the preset distance range is determined according to the distance from the carrier medium to the wearable device, and the preset position range is a corresponding position range within the shooting field of view of the wearable device.

5. The information input method according to claim 1, characterized in that, after the input information is determined, the method further comprises: acquiring the input information and displaying the input information in a display box of the input interface.

6. The information input method according to claim 1, characterized in that, after the step of responding to an input operation received in the input interface and determining input information according to the input operation, the method comprises: if it is detected that the carrier medium moves out of the shooting field of view, or the state of the carrier medium is detected to change, hiding the input interface.

7. The information input method according to claim 1, characterized in that, the method further comprises: acquiring a gesture action of the user; If the gesture action of the user matches the preset input gesture action, control the wearable device to enter the input interface mode, and detect a carrier medium within the shooting field of view of the wearable device; If the gesture action of the user matches the preset exit gesture action, control the wearable device to exit the input interface mode.

8. An information input device, Characterized in that, Comprising: A carrier medium detection module, which detects a carrier medium within the shooting field of view of the wearable device; A position information acquisition module, which acquires the position information of the carrier medium; An input interface display module, which displays the input interface of the wearable device according to the position information of the carrier medium, and the input interface is located on the carrier medium; An input information determination module, which responds to receiving an input operation in the input interface and determines input information according to the input operation.

9. A wearable device, Characterized in that, The wearable device includes a memory and a processor; The memory is used for storing a computer program; The processor is used for executing the computer program and implementing the information input method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, Characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to implement the information input method according to any one of claims 1 to 7.

Images