Implementation method and device of input method based on AR technology

Abstract

The present invention discloses an implementation method and device of an input method based on AR technology, relates to the field of AR technology, and provides an AR input method, which solves the problem that data input cannot be realized by using an input method in a virtual scene. The method comprises: using a smart terminal with a camera function to shoot a real scene picture, and converting the real scene picture into a virtual picture displayed by the terminal after processing by an AR application; turning on the virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual picture; obtaining the key value instruction input by the user through the virtual keyboard, and outputting the corresponding characters after processing. The device applies the method proposed in the above scheme.

Description

Technical Field

[0001] The present invention relates to the field of AR technology, and in particular to an implementation method and device of an input method based on AR technology. Background Art

[0002] With the development of augmented reality technology (AR), more and more AR products appear in our daily lives. Various products have appeared in the fields of AR education, AR marketing, AR tourism, etc., but most interactive operations are achieved through laser beams, clicks, touches and other buttons to achieve interaction between virtual and reality. With the development of AR technology and the continuous expansion of demand in fields such as collaborative office, AR input method will also become an indispensable core function in the future. Summary of the invention

[0003] The purpose of the present invention is to provide a method and device for implementing an input method based on AR technology, and to provide an AR input method that solves the problem that data input cannot be implemented using an input method in a virtual scene.

[0004] In order to achieve the above object, a first aspect of the present invention provides an implementation method of an input method based on AR technology, comprising:

[0005] Using a smart terminal with a camera function to shoot a real scene picture, and converting the real scene picture into a virtual picture displayed by the terminal after processing by an AR application;

[0006] Turning on a virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual screen;

[0007] The key value instruction input by the user through the virtual keyboard is obtained, and the corresponding characters are output after processing.

[0008] Preferably, the method for presenting the virtual keyboard in the virtual screen includes:

[0009] The virtual screen is divided into a plurality of partitions, and any partition is set as a display partition of the virtual keyboard.

[0010] Preferably, the method of enabling the virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual screen further includes:

[0011] Any one or more of the display angle, display position, keyboard size, keyboard color, and display transparency of the virtual keyboard in the display partition has a manual adjustment function.

[0012] Preferably, before the step of obtaining the key value instruction input by the user through the virtual keyboard and outputting the corresponding character after processing, the method further includes:

[0013] Constructing a virtual coordinate system in the virtual picture, and obtaining the virtual coordinates of the center point of each key in the virtual keyboard;

[0014] Based on the virtual display size of each key and the corresponding virtual coordinates, the virtual coordinate range corresponding to each key is identified.

[0015] Furthermore, the method of obtaining the key value instruction input by the user through the virtual keyboard and outputting the corresponding characters after processing includes:

[0016] The intelligent terminal presents the virtual screen converted from the real scene screen to the user in real time, and also presents the virtual keyboard in the virtual screen to the user at the same time;

[0017] Based on the virtual keyboard, the user simulates a key input action at a position corresponding to the virtual keyboard in the real scene;

[0018] The AR application obtains the key value input action and the coordinates of the area where the key value input action falls, and identifies the key value instruction of the user tapping the corresponding key based on the virtual coordinate range;

[0019] The AR application searches for the corresponding character from the built-in input method based on the key value instruction, and outputs it in a pop-up window after the user selects and confirms it.

[0020] Preferably, it also includes:

[0021] The smart terminal is connected to other smart device signals to achieve synchronous input of the output characters to the smart devices.

[0022] Compared with the prior art, the implementation method of the AR technology input method provided by the present invention has the following beneficial effects:

[0023] In the implementation method of the input method based on AR technology provided by the present invention, a real-life picture is captured by a smart terminal, and after being processed by an AR application installed in the smart terminal, the real-life picture can be converted into a virtual picture displayed by the terminal. At this time, the user can view the virtual picture corresponding to the real-life picture through the smart terminal screen, and then after turning on the virtual keyboard function of the AR application, a virtual keyboard will be displayed in the virtual picture. The user can place his fingers at the corresponding positions of the real-life picture according to the position of the virtual keyboard in the virtual picture to simulate tapping the keys on the virtual keyboard. Finally, the key value instructions that the user wants to input on the virtual keyboard are identified by capturing the tapping position of the user on the virtual keyboard, and the characters input by the user are obtained after processing.

[0024] It can be seen that the present invention can realize character input based on AR technology, solving the problem that characters cannot be input in AR scenes.

[0025] A second aspect of the present invention provides an implementation device of an input method based on AR technology, to which the implementation method of the input method based on AR technology described in the above technical solution is applied, and the device comprises:

[0026] A conversion display unit is used to shoot a real scene picture using an intelligent terminal with a camera function, and convert the real scene picture into a virtual picture displayed by the terminal after processing by an AR application;

[0027] A virtual keyboard switch unit, used to turn on the virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual screen;

[0028] The recognition output unit is used to obtain the key value instructions input by the user through the virtual keyboard, and output corresponding characters after processing.

[0029] Preferably, the virtual keyboard switch unit comprises:

[0030] The partition module is used to divide the virtual screen into a plurality of partitions and set any partition as a display partition of the virtual keyboard.

[0031] Preferably, it also includes:

[0032] A virtual coordinate unit, used to construct a virtual coordinate system in the virtual screen and obtain the virtual coordinates of the center point of each key in the virtual keyboard;

[0033] The input recognition unit recognizes the virtual coordinate range corresponding to each key based on the virtual display size of each key and the corresponding virtual coordinate.

[0034] Compared with the prior art, the beneficial effects of the device for implementing the input method based on AR technology provided by the present invention are the same as the beneficial effects of the method for implementing the input method based on AR technology provided by the above technical solution, which will not be elaborated here.

[0035] A third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the implementation method of the above-mentioned input method based on AR technology are executed.

[0036] Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present invention are the same as the beneficial effects of the implementation method based on the AR technology input method provided by the above technical solution, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0038] Figure 1 It is a flowchart of a method for implementing an input method based on AR technology in an embodiment of the present invention;

[0039] Figure 2 It is a timing diagram of the method for implementing the input method based on AR technology in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0041] Embodiment 1

[0042] See also Figure 1 and Figure 2 This embodiment provides an implementation method of an input method based on AR technology, including:

[0043] Use a smart terminal with a camera function to shoot real-life images, and after processing by an AR application, convert the real-life images into virtual images displayed on the terminal; turn on the virtual keyboard function of the AR application to present the virtual keyboard in the virtual image; obtain the key value instructions entered by the user through the virtual keyboard, and output the corresponding characters after processing.

[0044] In the implementation method of the input method based on AR technology provided in this embodiment, the real scene picture is captured by the smart terminal, and the real scene picture can be converted into a virtual picture displayed by the terminal after being processed by the AR application installed in the smart terminal. At this time, the user can view the virtual picture corresponding to the real scene picture through the smart terminal screen, and then after turning on the virtual keyboard function of the AR application, a virtual keyboard will be displayed in the virtual picture. The user can place his fingers at the corresponding position of the real scene picture according to the position of the virtual keyboard in the virtual picture to simulate tapping the keys on the virtual keyboard. Finally, the key value instructions that the user wants to enter on the virtual keyboard are identified by capturing the tapping position of the user on the virtual keyboard, and the characters entered by the user are obtained after processing.

[0045] It can be seen that this embodiment can implement character input based on AR technology, solving the problem of being unable to input characters in AR scenes.

[0046] In the above embodiment, the method for presenting the virtual keyboard in the virtual screen includes: dividing the virtual screen into a plurality of partitions, and setting any partition as a display partition of the virtual keyboard.

[0047] In specific implementation, the current virtual screen is divided into multiple partitions based on a preset partition rule, and a specified partition is selected as the display partition of the virtual keyboard. The purpose of this setting is that no matter how the terminal device moves or how the real scene changes, the display position of the virtual keyboard in the virtual screen is always fixed, which can facilitate users to quickly find the corresponding position of the virtual keyboard in the real scene, and then use the virtual keyboard to quickly input characters.

[0048] In the above embodiment, the method of enabling the virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual screen further includes:

[0049] The display angle, display position, keyboard size, keyboard color, and display transparency of the virtual keyboard in the display partition can be manually adjusted. In other words, the display angle, display position, keyboard size, keyboard color, and display transparency of the virtual keyboard in the virtual screen can be adjusted through the setting button on the AR application to suit the user's personal preferences.

[0050] In the above embodiment, before the step of obtaining the key value instruction input by the user through the virtual keyboard and outputting the corresponding characters after processing, the method further includes:

[0051] A virtual coordinate system is constructed in the virtual screen, and the virtual coordinates of the center point of each key in the virtual keyboard are obtained; based on the virtual display size of each key and the corresponding virtual coordinates, the virtual coordinate range corresponding to each key is identified.

[0052] In the specific implementation, any reference object in the virtual screen is selected to construct a virtual coordinate system, and the virtual coordinate system is simultaneously extended to the real scene. Then, the virtual coordinates of the center point of each key in the virtual keyboard are calculated and saved based on the virtual coordinate system in the virtual screen. According to the virtual display size of each key and the corresponding virtual coordinate, the virtual coordinate range corresponding to each key is identified, and the result is cached in the AR application.

[0053] In the above embodiment, the method of obtaining the key value instruction input by the user through the virtual keyboard and outputting the corresponding characters after processing includes:

[0054] The smart terminal presents a virtual screen converted from a real-life screen to the user in real time, and also presents a virtual keyboard in the virtual screen to the user; based on the virtual keyboard, the user simulates key input actions at positions corresponding to the virtual keyboard in the real-life screen; the AR application obtains the key input actions and the coordinates of the area where the key input actions fall, and identifies the key value instructions of the user tapping the corresponding keys based on the virtual coordinate range; the AR application searches for the corresponding characters from the built-in input method based on the key value instructions, and outputs them in the form of a pop-up window after the user selects and confirms.

[0055] During the specific implementation, the real-life scene is captured in real time by the camera of the smart terminal, and the user can see the virtual scene converted from the real-life scene through the display screen of the smart terminal. Since the real-life scene and the virtual scene are synchronized and corresponding, the user can predict the corresponding position of the virtual keyboard in the real-life scene according to the position of the virtual keyboard in the virtual scene, and then place the finger at the corresponding position of the real-life scene to simulate the keyboard input action. At this time, the typing action of the finger on the virtual keyboard will also be synchronously displayed in the virtual scene, and then the coordinates of the finger's tapping point are captured in the virtual scene and matched with the virtual coordinate range corresponding to each key in the virtual keyboard to obtain the key value instruction that the user wants to input, and then the character corresponding to the key value instruction is searched according to the built-in input method in the AR application, and output in the form of a pop-up window in the virtual scene after the user selects and confirms it.

[0056] Preferably, the above embodiment further includes: connecting the smart terminal to other smart devices by signal, so as to realize synchronous input of the output characters to the smart devices. The purpose of such setting is to realize the extended application of the input method based on AR technology.

[0057] The application scenarios of the method for implementing the input method based on AR technology in the above embodiment are various, and one of the scenarios is used as an example below:

[0058] In the process of selling a house, sellers often give buyers a detailed introduction to the building through sand tables and drawings, but with the emergence of AR, more and more sellers will use AR technology in the future to complete this operation, making it look more vivid and more real. At present, in the process of selling a house, when buyers need to make comments on specific floor locations, sellers often need to use other methods to make records, such as recording with paper and pen or laptop computers. After using the solution of this embodiment, the seller can mark his suggestions in the AR world and record the customer's purchase needs. After the customer leaves, the sales staff uses the AR application to review and can clearly see the corresponding annotations. It can also be transmitted to other smart devices using Bluetooth for viewing. The whole process is convenient and fast.

[0059] Embodiment 2

[0060] This embodiment provides an implementation device of an input method based on AR technology, including:

[0061] A conversion display unit is used to shoot a real scene picture using an intelligent terminal with a camera function, and convert the real scene picture into a virtual picture displayed by the terminal after processing by an AR application;

[0062] A virtual keyboard switch unit, used to turn on the virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual screen;

[0063] The recognition output unit is used to obtain the key value instructions input by the user through the virtual keyboard, and output corresponding characters after processing.

[0064] Preferably, the virtual keyboard switch unit comprises:

[0065] The partition module is used to divide the virtual screen into a plurality of partitions and set any partition as a display partition of the virtual keyboard.

[0066] Preferably, it also includes:

[0067] A virtual coordinate unit, used to construct a virtual coordinate system in the virtual screen and obtain the virtual coordinates of the center point of each key in the virtual keyboard;

[0068] The input recognition unit recognizes the virtual coordinate range corresponding to each key based on the virtual display size of each key and the corresponding virtual coordinate.

[0069] Compared with the prior art, the beneficial effects of the device for implementing an input method based on AR technology provided by the embodiment of the present invention are the same as the beneficial effects of the method for implementing an input method based on AR technology provided by the above-mentioned embodiment 1, and are not described in detail here.

[0070] Embodiment 3

[0071] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for implementing the input method based on AR technology are executed.

[0072] Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this embodiment are the same as the beneficial effects of the network testing method provided by the above technical solution, which will not be elaborated here.

[0073] A person skilled in the art can understand that all or part of the steps in the above-mentioned inventive method can be implemented by instructing related hardware through a program, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above-mentioned embodiment method, and the storage medium can be: ROM / RAM, disk, optical disk, memory card, etc.

[0074] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for implementing an input method based on AR technology, characterized in that: include: Using a smart terminal with a camera function to shoot a real scene picture, and converting the real scene picture into a virtual picture displayed by the terminal after processing by an AR application; Turning on a virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual screen; The method for presenting the virtual keyboard in the virtual screen includes: dividing the virtual screen into a plurality of partitions, and setting any partition as a display partition of the virtual keyboard; any one or more of the display angle, display position, keyboard size, keyboard color, and display transparency of the virtual keyboard in the display partition have a manual adjustment function; Obtaining key value instructions input by the user through the virtual keyboard, and outputting corresponding characters after processing; Wherein, before the step of obtaining the key value instruction input by the user through the virtual keyboard and outputting the corresponding characters after processing, it also includes: Constructing a virtual coordinate system in the virtual picture, and obtaining the virtual coordinates of the center point of each key in the virtual keyboard; Based on the virtual display size of each key and the corresponding virtual coordinates, identifying the virtual coordinate range corresponding to each key; The method of obtaining the key value instruction input by the user through the virtual keyboard and outputting the corresponding characters after processing includes: The intelligent terminal presents the virtual screen converted from the real scene screen to the user in real time, and also presents the virtual keyboard in the virtual screen to the user at the same time; Based on the virtual keyboard, the user simulates a key input action at a position corresponding to the virtual keyboard in the real scene; The AR application obtains the key value input action and the coordinates of the area where the key value input action falls, and identifies the key value instruction of the user tapping the corresponding key based on the virtual coordinate range; The AR application searches for the corresponding character from the built-in input method based on the key value instruction, and outputs it in a pop-up window after the user selects and confirms it.

2. The method according to claim 1, characterized in that Also includes: The smart terminal is connected to other smart device signals to achieve synchronous input of output characters to the smart device.

3. A device for implementing an input method based on AR technology, characterized in that: include: A conversion display unit is used to capture a real scene using a smart terminal with a camera function, and convert the real scene into a virtual scene displayed by the terminal after processing by an AR application; A virtual keyboard switch unit, used to turn on the virtual keyboard function of the AR application so that the virtual keyboard is presented in the virtual screen; The virtual keyboard switch unit includes: a partition module, which is used to divide the virtual screen into multiple partitions and set any partition as a display partition of the virtual keyboard; any one or more of the display angle, display position, keyboard size, keyboard color, and display transparency of the virtual keyboard in the display partition have a manual adjustment function; A virtual coordinate unit, used to construct a virtual coordinate system in the virtual screen and obtain the virtual coordinates of the center point of each key in the virtual keyboard; An input recognition unit, based on the virtual display size of each key and the corresponding virtual coordinate, recognizes the virtual coordinate range corresponding to each key; An identification and output unit is used to obtain key value instructions input by the user through the virtual keyboard, and output corresponding characters after processing; wherein, the method of obtaining key value instructions input by the user through the virtual keyboard, and outputting corresponding characters after processing comprises: the intelligent terminal presents the virtual screen converted from the real scene screen to the user in real time, and also presents the virtual keyboard in the virtual screen to the user at the same time; the user simulates a key value input action at a position corresponding to the virtual keyboard in the real scene screen based on the virtual keyboard; the AR application obtains the key value input action and the coordinates of the area where the key value input action falls, and identifies the key value instruction of the user tapping the corresponding key based on the virtual coordinate range; the AR application searches for the corresponding character from the built-in input method based on the key value instruction, and outputs it in the form of a pop-up window after the user selects and confirms it.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in claim 1 or 2 are executed.

Images