A cloud-based input method and device

Abstract

This invention discloses an input method and apparatus for a cloud-based virtual machine, relating to the field of computer technology. One specific embodiment of the method includes: responding to a user's click operation on a video in a browser, sending a click command to the cloud-based virtual machine; the click command indicating a first position of the click operation on the video; obtaining element information returned by the cloud-based virtual machine in response to the click command; if the element information indicates that the first position is an inputable character, creating an input box at the first position on the video and activating a target input method; receiving characters entered by the user using the target input method in the input box; and sending the characters to the cloud-based virtual machine. This embodiment, by creating an input box and activating the input method on the browser side, can directly send the entered characters to the cloud-based virtual machine, eliminating the need to activate the input method and remotely input characters on the cloud-based virtual machine. This also eliminates the need to adapt different activation methods on the cloud-based virtual machine platform, making it convenient, fast, and highly versatile.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an input method and apparatus for a cloud-based virtual machine. Background Technology

[0002] Cloud-based virtual device (YDV) is a physical mobile phone that uses the internet to project its screen onto a browser, allowing users to remotely control the device from the browser. In existing input methods, the input method needs to be activated on the cloud-based virtual device, and keystrokes from the browser need to be sent to the device to complete character input. However, the cloud-based virtual device itself cannot activate the input method via browser shortcuts. Therefore, a new activation method for the input method needs to be adapted for the cloud-based virtual device platform. Adapting the activation method to different operating systems, cloud-based virtual device brands, and operating system versions requires specific development solutions, placing high demands on development and making maintenance difficult. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an input method and apparatus for a cloud-based real device. By creating an input box and waking up the input method on the browser side, the input characters can be directly sent to the cloud-based real device without waking up the input method and remotely inputting characters on the cloud-based real device. This eliminates the need to adapt different wake-up methods on the cloud-based real device platform, making it convenient, fast, and highly versatile.

[0004] To achieve the above objectives, according to one aspect of the present invention, a method for inputting data on a cloud-based real device is provided.

[0005] An embodiment of the present invention provides an input method for a cloud-based real device, applied to a browser, comprising: responding to a user's click operation on a video in the browser, sending a click instruction to the cloud-based real device; the click instruction indicating a first position of the click operation on the video; acquiring element information returned by the cloud-based real device in response to the click instruction; if the element information indicates that the first position is an inputable character, creating an input box at the first position on the video and activating a target input method; receiving a character entered by the user using the target input method in the input box; and sending the character to the cloud-based real device.

[0006] Optionally, before responding to the user's click operation on the video on the browser, the method further includes: sending a connection request to the cloud device; establishing a network socket between the cloud device and the browser based on the matching information returned by the cloud device in response to the connection request, so as to realize communication between the browser and the cloud device using the network socket.

[0007] Optionally, the network socket includes a video stream and a control stream; wherein the video stream is used to display the interface information of the cloud real device on the browser; and the control stream is used to send the user's click operation on the video to the cloud real device.

[0008] Optionally, before creating the input box at the first position on the video and activating the target input method, the method further includes configuring the attributes, length, height, and duration of the input box;

[0009] Creating an input box at the first position on the video and activating the target input method includes: creating the input box according to pre-configured attributes, length, and height, and displaying the input box according to the duration.

[0010] To achieve the above objectives, according to another aspect of the present invention, another method for inputting data on a cloud-based real device is provided.

[0011] Another cloud-based real device input method according to an embodiment of the present invention, applied to a browser, includes: receiving a click instruction from the browser; the click instruction indicating a first position of the click operation on a browser video; determining a second position on the current display interface corresponding to the first position; determining element information of the first position based on whether the second position is inputtable; returning the element information to the browser; and receiving characters sent by the browser when the element information indicates that the first position is inputtable.

[0012] Optionally, before receiving the click instruction from the browser, the method further includes: receiving a connection request sent by the browser; generating matching information in response to the connection request, and returning the matching information to the browser, so that a network socket is established between the cloud device and the browser, and communication between the browser and the cloud device is realized using the network socket.

[0013] To achieve the above objectives, according to another aspect of the present invention, an input device for a cloud-based virtual machine is provided.

[0014] An input device for a cloud-based real device according to an embodiment of the present invention, applied to a browser, includes: a first sending module, configured to send a click instruction to the cloud-based real device in response to a user's click operation on a video in the browser; the click instruction indicates a first position of the click operation on the video; and send the character to the cloud-based real device; an acquisition module, configured to acquire element information returned by the cloud-based real device in response to the click instruction; a creation module, configured to create an input box at the first position on the video and activate a target input method when the element information indicates that the first position is an inputable character; and a first receiving module, configured to receive characters entered by the user in the input box using the target input method.

[0015] To achieve the above objectives, according to another aspect of the present invention, another input device for a cloud-based real device is provided.

[0016] Another cloud-based input device according to an embodiment of the present invention, applied to a cloud-based real device, includes: a second receiving module, configured to receive a click instruction from a browser; the click instruction indicates a first position of the click operation on a browser video; and, if element information indicates that the first position is an input character, receive characters sent by the browser; a determining module, configured to determine a second position on the current display interface corresponding to the first position, and determine element information of the first position based on whether the second position is an input character; and a second sending module, configured to return the element information to the browser.

[0017] To achieve the above objectives, according to another aspect of the present invention, a cloud-based real device system is provided.

[0018] An embodiment of the present invention provides a cloud-based real device system, comprising: a browser client and a cloud-based real device, wherein the browser client, in response to a user's click operation on a video on the browser client, sends a click instruction to the cloud-based real device; the click instruction indicates a first position of the click operation on the video; the cloud-based real device receives the click instruction from the browser client; determines a second position on the current display interface corresponding to the first position, and determines element information of the first position based on whether the second position is inputtable; the cloud-based real device returns the element information to the browser client; the browser client obtains the element information returned by the cloud-based real device in response to the click instruction; if the element information indicates that the first position is inputtable, it creates an input box at the first position on the video and activates a target input method; the browser client receives characters input by the user using the target input method in the input box and sends the characters to the cloud-based real device.

[0019] Optionally, before the step of responding to the user's click operation on the video on the browser, the method further includes: displaying one or more cloud-based virtual machines to the user through the browser; the browser receiving the user's selection instruction; the selection instruction indicating a target cloud-based virtual machine selected by the user from the one or more cloud-based virtual machines; the browser sending a connection request to the target cloud-based virtual machine; the target cloud-based virtual machine generating matching information in response to the connection request sent by the browser, and returning the matching information to the browser, so as to establish a network socket between the cloud-based virtual machine and the browser, and use the network socket to realize communication between the browser and the cloud-based virtual machine.

[0020] To achieve the above objectives, according to another aspect of the present invention, an electronic device for cloud-based real-device input is provided.

[0021] An embodiment of the present invention provides a cloud-based real-device input electronic device, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a cloud-based real-device input method according to an embodiment of the present invention.

[0022] To achieve the above objectives, according to another aspect of the present invention, a computer-readable storage medium is provided.

[0023] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements an input method for a cloud-based real device according to an embodiment of the present invention.

[0024] One embodiment of the above invention has the following advantages or beneficial effects: by creating an input box and waking up the input method on the browser side, the input characters can be sent directly to the cloud real device without waking up the input method and remotely inputting characters on the cloud real device side, and thus without needing to adapt different wake-up methods on the cloud real device platform, which is convenient, fast and highly versatile.

[0025] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0026] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0027] Figure 1 This is a schematic diagram of the main flow of a cloud-based real device input method according to an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram illustrating the main process of connecting the browser client with the cloud real device according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the main flow of another cloud-based real device input method according to an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of the main process of connecting a cloud-based physical device with a browser according to an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of the main modules of an input device for a cloud-based virtual machine according to an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of the main modules of an input device for a cloud-based virtual machine according to an embodiment of the present invention;

[0033] Figure 7 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0034] Figure 8 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0035] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0036] It should be noted that, unless otherwise specified, the embodiments of the present invention and the technical features thereof can be combined with each other.

[0037] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0038] Figure 1 This is a schematic diagram of the main flow of the cloud real device input method according to an embodiment of the present invention.

[0039] like Figure 1 As shown, the cloud-based real device input method of this invention is applied to the browser side and mainly includes the following steps:

[0040] Step S101: In response to the user's click operation on the video on the browser, a click instruction is sent to the cloud real device; the click instruction indicates the first position of the click operation on the video;

[0041] Step S102: Obtain the element information returned by the cloud real device in response to the click command;

[0042] Step S103: If the element information indicates that the first position is an input character, create an input box at the first position on the video and activate the target input method;

[0043] Step S104: Receive the characters entered by the user in the input box using the target input method;

[0044] Step S105: Send the characters to the cloud real device.

[0045] Among them, Cloud True Device is a physical mobile phone that is realized through the Internet. It projects the phone screen to a browser via video stream, allowing users to remotely control the physical phone through the browser. For users, the only operation is through the browser, and the video displayed in the browser is the phone interface of Cloud True Device. This invention achieves remote control of the phone interface through user operations on the video.

[0046] It should be noted that before data transmission (remote control) occurs between the browser and the cloud device, a communication connection between the browser and the cloud device must be established beforehand. Data transmission can only be completed on the basis of a stable communication connection. Therefore, before step S101, in one optional embodiment, as follows: Figure 2 As shown, it includes:

[0047] Step S201: Send a connection request to the cloud physical device;

[0048] Step S202: Based on the matching information returned by the cloud physical device in response to the connection request, establish a network socket between the cloud physical device and the browser to realize communication between the browser and the cloud physical device using the network socket.

[0049] Multiple cloud virtual machines can exist, and users can select the target cloud virtual machine to connect to from among them in the browser, and then send a connection request to the target cloud virtual machine. The matching information can indicate whether the match is successful or unsuccessful. If the matching information indicates a successful match, a network socket is established between the cloud virtual machine and the browser.

[0050] In a further optional embodiment of step S202, the network socket can be two, including a video stream and a control stream, wherein different network sockets are used to implement different data transmission functions. Specifically, the video stream is used to display the interface information of the cloud real device on the browser, and the control stream is used to send the user's click operations on the video to the cloud real device.

[0051] Through the above process, a communication connection between the cloud virtual machine and the browser was established, and remote display of the cloud virtual machine interface and remote control of browser operations were achieved through two network sockets.

[0052] In step S101, the click instruction is generated based on the user's click operation. Specifically, the corresponding click instruction can be generated based on the coordinates or pixel position of the first position where the user clicks on the video, so that after the cloud-based device receives the click instruction, it can obtain the specific click position of the user on the mobile phone interface. The click operation of this invention is not limited to a single mouse click, but may also include double clicks, swipes, etc., and this invention does not limit it in this way.

[0053] It should be noted that for the mobile phone interface on the cloud-based real device, each pixel (position) has a corresponding element type, element size, and control type, and each interface has an element tree that contains all the element information in the interface. Therefore, the element information returned in step S102 can specifically include: element type, control type, and element size.

[0054] Furthermore, in step S103, the browser needs to judge the element information to determine whether the element information indicates that the first position is an input character. For example, it can judge whether the control type is an input box or whether the element type is an input type. This invention does not limit this.

[0055] In an optional embodiment, before step S103, the method further includes configuring the attributes, length, height, and duration of the input box. It is understood that when a user clicks on the video, the location where the input box pops up, its size (height and length), color, and whether it is transparent can all be set according to the user's needs. Preferably, the input box is displayed in a location that does not affect the display of the phone's interface in the video. Specifically, step S103 may include creating the input box according to the pre-configured attributes, length, and height, and displaying the input box according to the duration. The duration can be understood as the display time of the input box, such as 10 seconds or 20 seconds. If the user did not intend to input characters but accidentally clicked on the video, the input box can be deleted after the duration has expired.

[0056] In one alternative embodiment, when a user performs consecutive click operations, when the next click operation is triggered, the created input box will be deleted even if the input box corresponding to the previous position has not exceeded the duration, and only the input box corresponding to the last click operation will be displayed.

[0057] Regarding the target input method activated in step S103, in one optional embodiment, it can be preset according to the specific type of operation. For example, an English input method is activated during a mouse click operation, and a Chinese input method is activated during a mouse double-click operation. Different input methods are activated through different click operations to facilitate simultaneous input of Chinese and English by the user. It should be noted that since the input method is activated locally in the browser, the cloud-based real device directly obtains the characters input locally in the browser. Therefore, it is generally not necessary to install multiple different types of Chinese input methods (such as Sogou Input Method, Wubi Input Method, etc., which are all used for Chinese input). Only one commonly used one needs to be installed.

[0058] The cloud-based real device input method according to embodiments of the present invention can directly send the input characters to the cloud-based real device by creating an input box and waking up the input method on the browser side. This eliminates the need to wake up the input method on the cloud-based real device and remotely input characters, and thus eliminates the need to adapt different wake-up methods on the cloud-based real device platform. It is convenient, fast, and highly versatile.

[0059] Figure 3 This is a schematic diagram of the main flow of another cloud-based real device input method according to an embodiment of the present invention.

[0060] like Figure 3 As shown, the cloud-based real device input method of this invention is applied to a cloud-based real device and mainly includes the following steps:

[0061] Step S301: Receive a click instruction from the browser; the click instruction indicates the first position of the click operation on the browser video;

[0062] Step S302: Determine the second position on the current display interface that corresponds to the first position, and determine the element information of the first position based on whether characters can be input at the second position;

[0063] Step S303: Return the element information to the browser;

[0064] Step S304: If the element information indicates that the first position is an input character, receive the character sent by the browser.

[0065] The browser can only display the mobile interface of the cloud-based device. For videos, it's impossible to obtain the corresponding element information from the first click position on the video. Therefore, the cloud-based device needs to determine the element information. Step S302 can be understood as determining the specific second position on the mobile interface based on the first click position on the video. In other words, it determines the second position the user wants to click on the mobile interface based on the first position, and then determines the element type, size, and control type corresponding to the second position. For example, if the user clicks the first position on the video in the browser, the browser sends the click instruction containing the first position to the cloud-based device. The cloud-based device then determines the second position on the currently displayed interface corresponding to the first position and whether the second position is an input box. If so, it determines that the element information for the first position is an input box.

[0066] In one alternative embodiment, before receiving a click instruction from the browser, such as Figure 4 As shown, it also includes:

[0067] Step S401: Receive a connection request sent by the browser;

[0068] Step S402: In response to the connection request, generate matching information and return the matching information to the browser so that a network socket is established between the cloud physical device and the browser, and the network socket is used to realize communication between the browser and the cloud physical device.

[0069] According to the cloud-based real device input method of the present invention, by creating an input box on the browser side and waking up the input method, the input characters can be directly sent to the cloud-based real device without waking up the input method on the cloud-based real device and remotely inputting characters. Therefore, there is no need to adapt different wake-up methods on the cloud-based real device platform, which is convenient, fast and highly versatile.

[0070] Figure 5 This is a schematic diagram of the main modules of the input device of the cloud-based virtual machine according to an embodiment of the present invention.

[0071] like Figure 5 As shown, the input device 500 for the browser side of the cloud real device according to an embodiment of the present invention includes:

[0072] The first sending module 501 is used to send a click instruction to the cloud device in response to a user's click operation on the video in the browser; the click instruction indicates a first position of the click operation on the video; and sends the character to the cloud device.

[0073] The acquisition module 502 is used to acquire element information returned by the cloud real device in response to the click instruction;

[0074] A creation module 503 is used to create an input box at the first position on the video and activate the target input method when the element information indicates that the first position is an input character.

[0075] The first receiving module 504 is used to receive characters entered by the user in the input box using the target input method.

[0076] In an optional embodiment of the present invention, the first sending module 501 is further configured to send a connection request to the cloud virtual machine; and establish a network socket between the cloud virtual machine and the browser based on the matching information returned by the cloud virtual machine in response to the connection request, so as to realize communication between the browser and the cloud virtual machine using the network socket.

[0077] In an optional embodiment of the present invention, the network socket includes a video stream and a control stream; wherein the video stream is used to display the interface information of the cloud real device on the browser; and the control stream is used to send the user's click operation on the video to the cloud real device.

[0078] In an optional embodiment of the present invention, the device further includes a configuration module for configuring the attributes, length, height, and duration of the input box before creating the input box at the first position on the video and waking up the target input method; the creation module 503 is further configured to create the input box according to the pre-configured attributes, length, and height, and display the input box according to the duration.

[0079] The cloud-based input device for a real device applied to a browser, according to an embodiment of the present invention, can directly send the input characters to the real device by creating an input box and waking up the input method on the browser side. This eliminates the need to wake up the input method on the real device side and remotely input characters, and thus eliminates the need to adapt different wake-up methods on the real device platform. This is convenient, fast, and highly versatile.

[0080] Figure 6 This is a schematic diagram of the main modules of the input device for a cloud virtual machine according to an embodiment of the present invention.

[0081] like Figure 6 As shown, the input device 500 of the cloud virtual machine in this embodiment of the invention includes:

[0082] The second receiving module 601 is used to receive a click instruction from the browser; the click instruction indicates the first position of the click operation on the browser video; and when the element information indicates that the first position is an input character, it receives the character sent by the browser.

[0083] The determining module 602 is used to determine the second position on the current display interface corresponding to the first position, and to determine the element information of the first position based on whether the second position can be input with characters.

[0084] The second sending module 603 is used to return the element information to the browser.

[0085] In an optional embodiment of the present invention, the second receiving module 601 is further configured to, before receiving the click instruction from the browser, include: receiving a connection request sent by the browser; generating matching information in response to the connection request, and returning the matching information to the browser, so as to establish a network socket between the cloud virtual machine and the browser, and use the network socket to realize communication between the browser and the cloud virtual machine.

[0086] According to the embodiments of the present invention, the input device for a cloud-based virtual machine can directly send the input characters to the cloud-based virtual machine by creating an input box and waking up the input method on the browser side. This eliminates the need to wake up the input method on the cloud-based virtual machine and remotely input characters, and thus eliminates the need to adapt different wake-up methods on the cloud-based virtual machine platform. This is convenient, fast, and highly versatile.

[0087] Figure 7 An exemplary system architecture 700 is shown, in which the cloud-based input method or input device of the present invention can be applied.

[0088] like Figure 7 As shown, the system architecture 700 may include browser clients 701, 702, and 703, a network 704, a server 705, and cloud physical devices 706, 707, and 708. The network 704 serves as the medium for providing communication links between the browser clients 701, 702, and 703 and the server 705, and between the server 705 and the cloud physical devices 706, 707, and 708. The network 704 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0089] Users can use browser clients 701, 702, and 703 to interact with server 705 via network 704 to receive or send data. Various communication client applications can be installed on browser clients 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and social media platforms.

[0090] The browser side 701, 702, and 703 can be various electronic devices with a display screen that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0091] Server 705 can be a server that provides various services, such as a backend management server that supports click operations sent by users through browser terminals 701, 702, and 703. The backend management server can analyze and process the received click operations and input characters, and send the processing results (such as the input characters) to cloud physical devices 706, 707, and 708.

[0092] It should be understood that Figure 7 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0093] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system 800 suitable for implementing a terminal device of the present invention. Figure 8 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0094] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 802 or programs loaded from storage section 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the system 800. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) first interface 805 is also connected to the bus 804.

[0095] The following components are connected to the I / O first interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network first interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O first interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0096] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit (CPU) 801, it performs the functions defined above in the system of this invention.

[0097] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0099] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a first sending module, an acquisition module, a creation module, and a first receiving module. The names of these modules do not necessarily limit the module itself; for example, the first sending module may also be described as "a module that sends a click command to the cloud-based real device in response to a user's click operation on a video in the browser."

[0100] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to include: sending a click instruction to the cloud-based device in response to a user's click operation on a video in the browser; the click instruction indicating a first position of the click operation on the video; acquiring element information returned by the cloud-based device in response to the click instruction; creating an input box at the first position on the video and activating a target input method when the element information indicates that the first position is an input character; receiving characters entered by the user in the input box using the target input method; and sending the characters to the cloud-based device.

[0101] According to the technical solution of the present invention, by creating an input box and waking up the input method on the browser side, the input characters can be directly sent to the cloud real device without waking up the input method and remotely inputting characters on the cloud real device side. Therefore, there is no need to adapt different wake-up methods on the cloud real device platform, which is convenient, fast and highly versatile.

[0102] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for inputting data on a cloud-based real device, characterized in that: Applied to the browser, including: In response to a user's click operation on the video in the browser, a click instruction is sent to the cloud-based real device; the click instruction indicates the first position of the click operation on the video; the cloud-based real device is a physical mobile phone, the video displayed in the browser is the mobile phone interface of the cloud-based real device, and the user remotely controls the cloud-based real device in the browser. The cloud device obtains the element information returned in response to the click command; the cloud device determines the second position on the current display interface corresponding to the first position, and determines the element information of the first position based on whether the second position can input characters; If the element information indicates that the first position is an input character, an input box is created at the first position on the video and the target input method is activated; the browser judges the element information to determine whether the element information indicates that the first position is an input character. Receive characters entered by the user in the input box using the target input method; The characters are sent to the cloud device.

2. The method according to claim 1, characterized in that, Prior to responding to a user's click action on the video in the browser, the method further includes: Send a connection request to the cloud device; Based on the matching information returned by the cloud device in response to the connection request, a network socket is established between the cloud device and the browser to enable communication between the browser and the cloud device.

3. The method according to claim 2, characterized in that, The network socket includes a video stream and a control stream; wherein... The video stream is used to display the interface information of the cloud real device on the browser. The control flow is used to send the user's click operation on the video to the cloud real device.

4. The method according to claim 1, characterized in that, Before creating the input box at the first position on the video and activating the target input method, the method further includes: Configure the input box's attributes, length, height, and duration; Creating an input box at the first position on the video and activating the target input method includes: The input box is created according to the pre-configured attributes, length, and height, and displayed according to the duration.

5. A method for inputting data on a cloud-based real device, characterized in that, Applied to cloud-based real devices, including: The browser receives a click instruction; the click instruction indicates the first position of the click operation on the browser video; the cloud device is a physical mobile phone, the video displayed on the browser is the mobile phone interface of the cloud device, and the user remotely controls the cloud device on the browser. Determine the second position on the current display interface that corresponds to the first position, and determine the element information of the first position based on whether the second position allows input of characters; Return the element information to the browser. When the element information indicates that the first position is an input character, the browser receives the character sent by the browser; when the element information indicates that the first position is an input character, the browser creates an input box at the first position on the video and activates the target input method; receives the character entered by the user in the input box using the target input method; sends the character to the cloud device; the browser judges the element information to determine whether the element information indicates that the first position is an input character.

6. The method according to claim 5, characterized in that, Before receiving the click instruction from the browser, the following is also included: Receive the connection request sent by the browser; In response to the connection request, matching information is generated and returned to the browser, so that a network socket is established between the cloud device and the browser, and the network socket is used to realize communication between the browser and the cloud device.

7. An input device for a cloud-based virtual machine, characterized in that, Applied to the browser, including: The first sending module is used to respond to a user's click operation on the video on the browser end and send a click instruction to the cloud real device; the click instruction indicates the first position of the click operation on the video; and sends characters to the cloud real device; the cloud real device is a physical mobile phone, the video displayed on the browser end is the mobile phone interface of the cloud real device, and the user remotely controls the cloud real device on the browser end; The acquisition module is used to acquire element information returned by the cloud real device in response to the click instruction; determine a second position on the current display interface corresponding to the first position; and determine the element information of the first position based on whether the second position can be input with characters. A creation module is configured to create an input box at the first position on the video and activate the target input method when the element information indicates that the first position is an inputable character; the creation module is also configured to judge the element information to determine whether the element information indicates that the first position is an inputable character. The first receiving module is used to receive characters entered by the user in the input box using the target input method.

8. An input device for a cloud-based virtual machine, characterized in that, Applied to cloud-based real devices, including: The second receiving module is used to receive a click instruction from the browser; the click instruction indicates the first position of the click operation on the browser video; when the element information indicates that the first position is an input character, the module receives the character sent by the browser; the cloud real device is a physical mobile phone, the video displayed on the browser is the mobile phone interface of the cloud real device, and the user remotely controls the cloud real device on the browser. The determination module is used to determine the second position on the current display interface corresponding to the first position, and to determine the element information of the first position based on whether the second position can be input with characters. The second sending module is used to return the element information to the browser. When the element information indicates that the first position is an input character, the browser creates an input box at the first position on the video and activates the target input method; receives the character entered by the user in the input box using the target input method; sends the character to the cloud device; and the browser judges the element information to determine whether the element information indicates that the first position is an input character.

9. A cloud-based real-device system, characterized in that, include: The browser client of claim 7 and the cloud-based real device of claim 8; wherein, In response to a user's click on the video on the browser, the browser sends a click instruction to the cloud-based real device; the click instruction indicates the first position of the click operation on the video. The cloud-based real device receives a click command from the browser; determines a second position on the current display interface corresponding to the first position; and determines the element information of the first position based on whether characters can be input at the second position. The cloud-based real device returns the element information to the browser. The browser obtains the element information returned by the cloud real device in response to the click command; If the element information indicates that the first position is an input character, an input box is created at the first position on the video and the target input method is activated. The browser receives characters entered by the user in the input box using the target input method and sends the characters to the cloud device.

10. The system according to claim 9, characterized in that, Prior to the step of responding to a user's click on a video on the browser side, the method further includes: One or more cloud-based physical devices are displayed to the user through the browser. The browser receives the user's selection instruction; the selection instruction indicates the target cloud physical device selected by the user from the one or more cloud physical devices; The browser sends a connection request to the target cloud device; The browser client of the target cloud device sends a connection request, generates matching information in response to the connection request, and returns the matching information to the browser client, so as to establish a network socket between the cloud device and the browser client, and use the network socket to realize communication between the browser client and the cloud device.

11. An electronic device for cloud-based real-device input, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

12. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images