Multi-screen input method, electronic equipment, storage medium and screen projection system

Abstract

The invention provides a multi-screen input method, electronic equipment, a storage medium and a screen projection system, the method is applied to first electronic equipment, the first electronic equipment is in communication connection with second electronic equipment, the first electronic equipment comprises an entity display screen and a virtual display screen, and the virtual display screen is used for displaying on the second electronic equipment. The method comprises the following steps: associating a first input method corresponding to an entity display screen and associating a second input method corresponding to a virtual display screen; obtaining a screen identifier, wherein the screen identifier is used for indicating a screen acted by the user operation; when the screen identifier is a virtual screen identifier, displaying a second input method on a virtual display screen; and when the screen identifier is an entity screen identifier, displaying the first input method on an entity display screen. By means of the method, input operations of multiple screens of the electronic equipment do not affect one another, and multi-screen input of the electronic equipment is achieved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a multi-screen input method, electronic device, storage medium and screen projection system. Background Art

[0002] When an electronic device has multiple screens, inputs from each screen will affect each other. In the same electronic device, the input method can only be called by one application at the same time. When one screen of the electronic device calls the input method, if another screen also calls the input method for input, the input method will be switched from the previous screen to the other screen, and the input method in the previous screen will be destroyed by the system, making it impossible to continue inputting on the previous screen. This reduces the user experience.

[0003] The car computer can be an electronic device installed in a smart vehicle, which can be used to realize information communication and interactive control between people and cars, and between cars and the outside world. The car computer can be connected to a mobile phone, and the mobile phone can be operated on the car computer's display screen. When the car computer and the mobile phone are connected, when the user operates the mobile phone application on the car computer display screen to input, the system will forcibly destroy the input method being input on the mobile phone and display the input method in the car computer, which seriously affects the user's input experience.

[0004] It can be seen that although electronic devices can have multiple screens, they cannot implement multi-screen input, which affects the user experience. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a multi-screen input method, an electronic device, a storage medium and a screen projection system, so as to ensure that the input operations of multiple screens of the electronic device do not affect each other, thereby realizing multi-screen input of the electronic device.

[0006] A first aspect of an embodiment of the present application provides a multi-screen input method, the method being applied to a first electronic device, the first electronic device being communicatively connected to a second electronic device, the first electronic device comprising a physical display screen and a virtual display screen, the virtual display screen being used for displaying on the second electronic device, the method comprising:

[0007] Associating a first input method corresponding to the physical display screen and associating a second input method corresponding to the virtual display screen;

[0008] Obtaining a screen identifier, wherein the screen identifier includes a virtual screen identifier and a physical screen identifier, and the screen identifier is used to indicate the screen on which the user operation is performed;

[0009] When the screen identifier is the virtual screen identifier, displaying the second input method on the virtual display screen;

[0010] When the screen identifier is the physical screen identifier, the first input method is displayed on the physical display screen.

[0011] In a possible implementation, the screen identifier is a physical screen identifier, the first electronic device is installed with a first application, and the physical display screen is used to display a first user interface of the first application, and obtaining the screen identifier includes:

[0012] A first input operation of a user is detected, where the first input operation is performed on an input box of the first user interface;

[0013] In response to the first input operation, a window token of the first user interface is acquired, wherein the window token includes the physical screen identifier.

[0014] In a possible implementation manner, displaying the first input method on the physical display screen includes:

[0015] Calling an input method management service based on the window token;

[0016] The input method management service determines the associated first input method based on the physical screen identifier, calls the first input method and generates a keyboard interface of the first input method;

[0017] A keyboard interface of the first input method is displayed on the first user interface.

[0018] In a possible implementation, the screen identifier is a virtual screen identifier, the first electronic device is installed with a second application, the virtual display screen is used to display a second user interface of the second application, and obtaining the screen identifier includes:

[0019] receiving a second input operation from the second electronic device, where the second input operation is performed on an input box of the second user interface;

[0020] In response to the second input operation, a window token of the second user interface is acquired, wherein the window token includes the virtual screen identifier.

[0021] In a possible implementation manner, displaying the second input method on the virtual display screen includes:

[0022] Invoke an input method management service based on the window token;

[0023] The input method management service determines an associated second input method based on the virtual screen identifier, calls the second input method and generates a keyboard interface of the second input method;

[0024] A keyboard interface of the second input method is displayed on the second user interface.

[0025] In a possible implementation manner, the first electronic device is installed with a third application, and the method further includes:

[0026] receiving a third startup operation of the user on the physical display screen, wherein the third startup operation is used to instruct to start the third application program;

[0027] Displaying a user interface of the third application on the physical display screen;

[0028] The third application is associated with the first input method based on the physical screen identifier.

[0029] A second aspect of an embodiment of the present application provides a multi-screen input method, the method being applied to a second electronic device, the second electronic device being communicatively connected to a first electronic device, the first electronic device comprising a physical display screen and a virtual display screen, the virtual display screen being used for displaying on the second electronic device, the method comprising:

[0030] Detecting a second input operation of a user on the second electronic device, where the second input operation is performed on an input box of a second user interface, where the second user interface is a user interface of a second application of the first electronic device, and the second user interface is displayed on the virtual display screen;

[0031] The second input operation is sent to the first electronic device, where the second input operation is used to trigger the first electronic device to obtain a window token of the second user interface, where the window token includes a virtual screen identifier, and the virtual screen identifier is used to display a keyboard interface of a second input method on the second user interface.

[0032] In a possible implementation, before detecting a second input operation of the user on the virtual display screen, the method further includes:

[0033] Detecting a second startup operation performed by a user on the second electronic device, where the second startup operation includes the virtual screen identifier, and the second startup operation is used to instruct to start the second application;

[0034] The second startup operation is sent to the first electronic device, where the second startup operation is used to trigger the first electronic device to display the second user interface on the virtual display screen, and associate the second application with the second input method based on the virtual screen identifier.

[0035] A third aspect of an embodiment of the present application provides a first electronic device, comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the first electronic device executes the method described in the first aspect or any possible implementation method of the first aspect.

[0036] A fourth aspect of an embodiment of the present application provides a second electronic device, comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the second electronic device executes the method described in the second aspect or any possible implementation method of the second aspect.

[0037] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any possible implementation method of the first aspect is implemented.

[0038] A sixth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the second aspect or any possible implementation of the second aspect is implemented.

[0039] A seventh aspect of an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the first electronic device executes the method described in the first aspect or any possible implementation of the first aspect.

[0040] An eighth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on a second electronic device, the second electronic device executes the method described in the second aspect or any possible implementation of the second aspect.

[0041] A ninth aspect of an embodiment of the present application provides a screen projection system, including a first electronic device and one or more second electronic devices, wherein the first electronic device and the second electronic device have a communication connection, and the second electronic device executes the method described in the second aspect or any possible implementation of the second aspect; the first electronic device executes the method described in the first aspect or any possible implementation of the first aspect.

[0042] Compared with the prior art, the embodiments of the present application have the following advantages:

[0043] In the embodiment of the present application, since the physical display screen and the virtual display screen are respectively associated with different input methods, different screens can call different input methods, and different screens can display keyboard interfaces of different input methods for input, so the input operations on different screens do not affect each other. That is, when input is performed in the physical display screen, the keyboard interface in the virtual display screen will not be destroyed; when input is performed in the virtual display screen, the keyboard interface in the virtual display screen will not be destroyed. In the embodiment of the present application, the physical display screen and the virtual display screen are isolated from each other in calling the input method, and the physical display screen and the virtual display screen can call the input method without interfering with each other, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art.

[0045] Figure 1 is a schematic diagram of a multi-screen input scenario provided in an embodiment of the present application;

[0046] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0047] Figure 3 It is a schematic diagram of the steps of a multi-screen input method provided by an embodiment of the present application;

[0048] Figure 4 It is a flowchart of an associative input method provided in an embodiment of the present application;

[0049] Figure 5 is a schematic diagram of different screens displaying different user interfaces provided by an embodiment of the present application;

[0050] Figure 6 It is a schematic diagram of the steps of a multi-screen input method provided by an embodiment of the present application;

[0051] Figure 7 This is a schematic diagram of a display effect provided by an embodiment of the present application;

[0052] Figure 8 This is a framework diagram of an input method provided in an embodiment of the present application;

[0053] Fig. 9 It is a flowchart of associating a screen with an input method provided in an embodiment of the present application;

[0054] Fig.10It is a flow chart of a screen display input method provided in an embodiment of the present application;

[0055] Fig.11 It is a schematic diagram of another multi-screen input method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0057] At present, electronic devices equipped with the Android system cannot support multi-screen input in the case of a single user. When there are two screens, when input is performed on the first screen, if you click the first screen, the input method will switch to the first screen, resulting in the inability to continue input on the second screen. Similarly, when input is performed on the second screen, if you click the second screen, the input method will switch to the second screen, resulting in the inability to continue input on the first screen. That is, when an electronic device has multiple screens, the calls to the input method on multiple screens will affect each other, reducing the user experience. For example, a vehicle may include multiple display screens, such as a central control screen, a co-pilot screen, and a rear screen. In the screen projection usage scenario, the input of each screen will affect each other. When the mobile phone and the car are interconnected, input on the mobile phone will cause the input method keyboard interface on the car display to be destroyed by the system; input on the car will cause the input method keyboard interface on the mobile phone display to be destroyed by the system.

[0058] As an example, the embodiment of the present application provides a multi-screen input scenario, such as Figure 1 As shown, the multi-screen input scenario may include: an electronic device 100 and one or more electronic devices 200 ( Figure 1 Only one is shown in the figure). There is a communication connection between the electronic device 100 and any electronic device 200, so that screen projection can be performed between the electronic device 100 and any electronic device 200.

[0059] As an example, after screen projection is established between electronic device 100 and electronic device 200, the screen content of electronic device 100 can be projected to electronic device 200. In other words, a virtual display screen is created for electronic device 100 on electronic device 200 to display the screen content in electronic device 100.

[0060] The electronic device 100 or the electronic device 200 may include but is not limited to a personal computer (PC), a smart phone, a netbook, a tablet computer, a smart camera, a PDA, a smart TV, a personal digital assistant (PDA), a portable multimedia player (PMP), a projection device, a smart screen device, an augmented reality (AR) / virtual reality (VR) device, a mixed reality (MR) device, a television or a somatosensory game console in a human-computer interaction scene, etc. The embodiments of the present application do not limit the specific functions and structures of the electronic device.

[0061] The electronic device 100 and the electronic device 200 may be different types of electronic devices. For example, the electronic device 100 may be a mobile phone, a tablet computer, or the like, and the electronic device 200 may be a smart car or the like, which is not limited in the embodiments of the present application. The electronic device 100 and the electronic device 200 may support the user to input text / letters, emoticons, or languages ​​using a virtual keyboard. As an example, the present application is described with the electronic device 100 being a mobile phone and the electronic device 200 being a car computer.

[0062] One or more applications may be installed in the electronic device 100. After the electronic device 100 and the electronic device 200 establish screen projection, the virtual display screen of the electronic device 100 may be operated through the display screen of the electronic device 200. For example, the application on the electronic device 100 may be started through the display screen of the electronic device 200, or the user interface of the application of the electronic device 100 may be displayed on the display screen of the electronic device 200.

[0063] In the embodiment of the present application, the electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The application layer includes applications such as a browser, an address book, a word processing software, and an instant messaging software.

[0064] Figure 2 The structure of an electronic device provided by an embodiment of the present application is shown. Figure 2As shown, the electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a wireless charging coil 140A, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a microphone 170B, an earphone interface 170C, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera module 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The processor 110 may be directly or indirectly connected to other components through a system bus.

[0065] The processor 110 may include one or more processing units, for example, the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (NPU), etc. Different processing units may be independent devices or integrated into one or more processors. The processor 110 may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of reading and executing instructions.

[0066] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function, etc. The data storage area may store data created during the use of the electronic device, etc. The internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (Universal Flash Storage, UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0067] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, baseband processor, etc. In some embodiments, the antenna 1 of the electronic device is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device can communicate with other devices through a wireless communication network.

[0068] The mobile communication module 150 can provide electronic devices with solutions for wireless communications including 2G / 3G / 4G / 5G. The wireless communication module 160 can provide solutions for wireless communications including wireless local area networks (WLAN), wireless fidelity (Wi-Fi), Bluetooth (BT), global navigation satellite system (GNSS), near field communication technology (NFC), infrared technology (IR), etc., which are applied to electronic devices.

[0069] The electronic device implements the display function through a graphics processor GPU, a display screen 194, and an application processor AP. Among them, the GPU connects the display screen 194 and the AP. The GPU can be used to perform mathematical and geometric calculations, as well as graphics rendering. The processor 110 may include one or more GPUs. The display screen 194 is also called a screen, which is used to display images and videos, etc. The display screen 194 can be one or more. In the present application, the display screen 194 can be a physical display screen (also called a "physical screen" or a "physical display screen"), or a virtual display screen (also called a "virtual screen"). In some examples, the physical screen may include a touch screen, also called a "touch screen". The touch screen is used to detect touch operations acting on or near it, such as click operations, sliding operations, etc., and pass the detected touch operations to the application processor to determine the type of touch event. The electronic device can provide visual output related to the touch operation through the display screen 194.

[0070] It is understood that the above example structures do not constitute a specific limitation on the electronic device. In some other examples, the electronic device may include more or fewer components, or combine certain components, or split certain components, or arrange the components differently. In some examples, the same type of components may be one or more, such as a speaker or a microphone. The illustrated components may include hardware, software, or a combination of software and hardware.

[0071] Based on this, an embodiment of the present application provides a multi-screen input method, which is used to enable independent input between multiple screens without affecting each other.

[0072] Figure 3 A schematic diagram of a multi-screen input method provided in an embodiment of the present application is shown, comprising the following steps:

[0073] S301: The first electronic device associates a first input method corresponding to the physical display screen and associates a second input method corresponding to the virtual display screen.

[0074] Among them, the first electronic device can be a mobile phone, and the second electronic device can be a car computer. The first electronic device includes one or more physical screens and virtual screens. The first electronic device can be operated through both the physical screen and the virtual screen. The mobile phone and the car computer can be networked to establish a communication connection, and the screen can be interconnected and projected. The communication connection between the mobile phone and the car computer can be implemented based on the communication protocol between devices. The discovery protocol between devices helps electronic devices find and identify other connectable electronic devices, and establishes a communication connection between different electronic devices, so that the virtual screen of the mobile phone is displayed on the physical screen of the car computer.

[0075] For example, the applications NearBy and NioLink for networking between mobile phones and vehicle computers can be developed in advance, wherein NearBy is used by mobile phones to discover and identify connectable vehicle computers, and NioLink is used to realize data transmission, resource sharing and collaborative work between vehicle computers and mobile phones in the scenario of vehicle computer interconnection.

[0076] The mobile phone can associate different input methods with different display screens. The mobile phone can associate the physical screen with the first input method and the virtual screen with the second input method. Association is to establish a corresponding mapping relationship between the unique identification information of the display screen and the input method in the mobile phone. Exemplarily, based on the unique identification information of the physical screen, the first input method corresponding to the display screen can be determined; based on the unique identification information of the virtual screen, the second input method corresponding to the virtual screen can be determined. Among them, the unique identification information can be a screen identification. For example, in the Android system, DisplayId is the above-mentioned screen identification. The unique identification information of the physical screen is the physical screen identification, and the unique identification information of the virtual screen is the virtual screen identification.

[0077] In one example, the first input method and the second input method may be different input methods, for example, the first input method may be the Sogou input method, and the second input method may be the Baidu input method. In another example, the second input method may be an application clone of the first input method. The mobile phone generates an application clone of the first input method according to the first input method, obtains the second input method, and installs the second input method in the mobile phone.

[0078] S302, obtaining a screen identifier, where the screen identifier includes a virtual screen identifier and a physical screen identifier, and the screen identifier is used to indicate the screen on which the user operation is performed.

[0079] When the user performs input operations on the physical screen of the mobile phone, the mobile phone can obtain the physical screen ID. When the user performs input operations on the virtual screen displayed on the vehicle computer, the mobile phone can obtain the virtual screen ID.

[0080] S303: When the screen identifier is the virtual screen identifier, display the second input method on the virtual screen.

[0081] The memory stores the association between the screen identifier and the input method. According to the virtual screen identifier, the mobile phone can search the associated second input method from the memory. The second input method is called and the keyboard interface of the second input method can be displayed on the virtual screen.

[0082] The virtual screen is displayed in the target area of ​​the physical screen of the vehicle computer. Based on the keyboard interface displayed on the virtual screen, the user can input on the mobile phone in the target area of ​​the physical screen of the vehicle computer.

[0083] S304: When the screen identifier is the physical screen identifier, display the first input method on the physical screen.

[0084] According to the physical screen identifier, the mobile phone can search the associated first input method from the memory, call the first input method, and display the keyboard interface of the first input method on the physical screen. Based on the keyboard interface displayed on the physical screen, the user can input on the physical screen of the mobile phone.

[0085] In this embodiment, different screens of the first electronic device are associated with different input methods. When inputting an operation on the physical screen, the first input method associated with the physical screen may be called. When operating the virtual screen, the first input method associated with the virtual screen may be called. The physical screen and the virtual screen do not affect each other in calling the input method, and the user's input operations on the physical screen and the virtual screen do not affect each other.

[0086] The mobile phone can install multiple applications, and the user can operate the applications through the physical screen and the virtual screen. When any application in the mobile phone is started on any screen, the mobile phone can execute step S301. Step S301 can be specifically performed by Figure 4 The steps shown are implemented. Figure 4 As shown:

[0087] S401, a first electronic device receives a first startup operation performed by a user on the physical screen, where the first startup operation is used to instruct startup of the first application.

[0088] The first startup operation may be a preset operation for the application displayed on the physical screen, and the preset operation may be a pre-agreed click operation, a long press operation, other gesture operations, etc. If the user's preset operation on the physical screen is captured, it is determined that the first startup operation is received.

[0089] The mobile phone can identify the first startup operation through the main screen desktop launcher (Launcher). The main screen desktop launcher is used to provide an interface for users to interact with the physical screen of the mobile phone. The main screen desktop launcher can generate a corresponding icon for the application on the physical screen; according to the coordinate information of the first startup operation in the physical screen, determine the icon targeted by the first startup operation, and obtain the first application identifier corresponding to the icon. According to the first application identifier, the mobile phone can determine that the object targeted by the received first startup operation is the first application.

[0090] S402: The first electronic device displays the user interface of the first application on the physical screen.

[0091] The mobile phone can load the executable file and startup configuration information of the first application according to the first application identifier, thereby starting the first application. The startup configuration information includes interface component configuration information. During the startup of the first application, a first user interface can be created according to the interface component configuration information, and the first user interface can be displayed on the physical screen.

[0092] S403: The first electronic device obtains the physical screen identifier in response to starting the first application.

[0093] The first startup operation carries a physical screen identifier. The main screen desktop launcher can obtain the physical screen identifier corresponding to the first startup operation when detecting the first startup operation. The mobile phone can obtain the physical screen identifier from the first startup operation.

[0094] The mobile phone may also obtain a screen identifier of the screen displayed on the first user interface to obtain a physical screen identifier.

[0095] S404: The first electronic device associates the first application with the first input method based on the physical screen identifier.

[0096] The configuration information of the mobile phone may include an input method. If there are multiple input methods and multiple display screens, different display screens may be configured with different input methods. Thus, each physical screen identifier has a corresponding associated first input method.

[0097] Optionally, the configuration information of the mobile phone may not include input method configuration information. In this case, the input method used by the physical screen can be configured by the user, so that each physical screen identifier has a corresponding associated first input method.

[0098] The mobile phone can establish a mapping relationship between the first application and the first input method, thereby associating the first input method with the first application. After the first application is associated with the first input method, the first application can call the first input method to perform input operations.

[0099] When any application is started on the physical screen, the mobile phone can associate the application with the first input method. For example, when the user switches the first application to the third application on the physical screen, it is equivalent to performing a third startup operation on the physical screen, and the third startup operation is used to instruct to start the third application. The mobile phone can receive the third startup operation of the user on the physical screen and associate the third application with the first input method.

[0100] S405: The second electronic device receives a second start-up operation from the user on the virtual screen, where the second start-up operation is used to instruct to start the second application.

[0101] The second start-up operation may be a preset operation for the icon displayed on the virtual screen. If the vehicle computer captures the user's preset operation on the physical screen displayed in the target area, it is determined that the second start-up operation is received.

[0102] S406: The second electronic device sends the second starting operation to the first electronic device.

[0103] The vehicle computer can send a control signal corresponding to the second startup operation to the mobile phone. The mobile phone can identify the second startup operation through the second screen desktop launcher (SecondLauncher). The second screen desktop launcher is used to provide an interface for the user to interact with the virtual screen of the mobile phone. The second screen desktop launcher can generate an icon for the application in the virtual screen, determine the icon targeted by the second startup operation based on the coordinate information of the second startup operation in the virtual screen, and obtain the second application identifier corresponding to the icon. Based on the second application identifier, the mobile phone can determine that the object targeted by the received second startup operation is the second application.

[0104] S407: The first electronic device displays the user interface of the second application on the virtual screen.

[0105] The mobile phone can load the executable file and startup configuration information of the second application according to the second application identifier, thereby starting the second application. The startup configuration information includes interface component configuration information. During the startup of the second application, a second user interface can be created according to the interface component configuration information, and the second user interface can be displayed on the virtual screen.

[0106] S408: The first electronic device associates the second application with the second input method based on the virtual screen identifier.

[0107] Each virtual screen identifier has a corresponding associated second input method. Optionally, the configuration information of the mobile phone may not include input method configuration information. In this case, the input method used by the virtual screen can be configured by the user, so that each virtual screen identifier has a corresponding associated second input method.

[0108] The mobile phone can establish a mapping relationship between the second application and the second input method, thereby associating the second input method with the second application. After the second application is associated with the second input method, the second application can call the second input method to perform input operations.

[0109] When any application is started on the virtual screen, the mobile phone can associate the application with the second input method. For example, when the user switches the second application to the fourth application on the virtual screen, it is equivalent to performing a fourth startup operation on the virtual screen, and the fourth startup operation is used to instruct to start the fourth application. The mobile phone can receive the fourth startup operation of the user on the virtual screen and associate the fourth application with the second input method.

[0110] After the applications in the physical screen and the virtual screen are started, the corresponding user interfaces can be displayed in the physical screen and the virtual screen respectively. Figure 5 shown. Figure 5 The memo in is the first application program, and the recorder is the second application program. Figure 5 As shown, the user interface of the memo is displayed on the physical screen of the mobile phone, the virtual screen of the mobile phone is displayed in the target area of ​​the physical screen of the vehicle computer, and the user interface of the recorder is displayed on the virtual screen.

[0111] By associating an application with an input method, a data channel can be established between the application and the input method. The input method and the application can perform data transmission and event processing through the data channel. After associating an application with an input method, the application can call the services provided by the input method. Applications on different screens are associated with different input methods. In the processes of different applications, the input method client can independently call the corresponding associated input method. Therefore, the calls to the input method by the applications on the two screens are independent of each other.

[0112] It should be noted that the physical screen and virtual screen of the mobile phone can switch input methods. After the input method is switched, the physical screen and virtual screen of the mobile phone are associated with different input methods. For example, the physical screen of the mobile phone is associated with the first input method, and the virtual screen is associated with the second input method. When the first input method is switched to the third input method, the physical screen of the mobile phone is associated with the third input method. The third input method can be an input method different from both the first input method and the second input method.

[0113] The mobile phone displays the corresponding input method keyboard interface on different screens according to the input operation, such as Figure 6 As shown:

[0114] S601: A first electronic device detects a first input operation of a user.

[0115] When the user performs a preset operation in the input box of the first user interface of the mobile phone, the first input operation is triggered. The input box refers to a visual area in the user interface that allows the user to input data.

[0116] S602: The first electronic device obtains a window token of the first user interface in response to the first input operation, where the window token includes the physical screen identifier.

[0117] The mobile phone can call the window token acquisition method to acquire the window token of the first user interface.

[0118] S603: The first electronic device displays a keyboard interface of the first input method on the first user interface based on the physical screen identifier.

[0119] S604: The second electronic device detects a second input operation of the user on the second electronic device.

[0120] When the user performs a preset operation in the input box of the second user interface of the mobile phone, a second input operation is triggered.

[0121] S605: The second electronic device sends the second input operation to the first electronic device.

[0122] The vehicle computer can send the control signal generated by the second input operation to the mobile phone.

[0123] S606: The first electronic device obtains a window token of the second user interface in response to the second input operation, where the window token includes the virtual screen identifier.

[0124] Based on the control signal sent by the vehicle computer, the mobile phone can call the window token acquisition method to obtain the window token of the second user interface.

[0125] S607: The first electronic device displays a keyboard interface of the second input method on the second user interface.

[0126] The second user interface is displayed in the virtual screen, and the virtual screen is displayed in the target area of ​​the vehicle computer's physical screen. Therefore, the target area of ​​the vehicle computer's physical screen can display the keyboard interface of the second input method for the user to input on the vehicle computer.

[0127] For example, when a user clicks on the browser application in the virtual screen on the mobile phone, the mobile phone can detect the electrical communication signal when the screen is triggered, and based on the electrical communication signal, it can be determined that the first startup operation is received. The mobile phone can determine that the application corresponding to the first startup operation is a browser, and display the user interface of the browser on the physical screen, thereby starting the browser on the physical screen. The physical screen in the mobile phone is associated with the first input method, so the mobile phone associates the browser with the first input method. In subsequent inputs, the browser can be input through the second input method. When the user clicks on the search input box of the browser, the first input operation can be triggered, and the keyboard interface of the first input method is displayed on the physical screen. When the user clicks on the characters on the keyboard interface, he can enter the keywords required for the search into the browser.

[0128] For example, when the user clicks on the recorder application on the virtual screen on the car computer, the virtual screen can collect the electrical communication signal when the screen is triggered, and the electrical communication signal can be sent to the mobile phone as the second startup operation. After the mobile phone receives the second startup operation, it can determine that the application corresponding to the second startup operation is the recorder, and display the user interface of the recorder on the virtual screen, thereby starting the recorder on the virtual screen. The virtual screen of the mobile phone is associated with the second input method, so the recorder can be associated with the second input method. In subsequent input, the recorder can be input through the second input method. When the user clicks the edit button of the recorder to name the recording, the second input operation can be triggered, and the keyboard interface of the second input method is displayed on the virtual screen. The user can edit the name of the recording by clicking the characters on the keyboard interface.

[0129] Based on the above method, the calling of input methods can be isolated in different screens of the first electronic device, and input can be performed simultaneously in different screens. Figure 7 is a schematic diagram of a display effect provided by an embodiment of the present application, such as Figure 7 As shown, the browser (first application) on the physical screen of the mobile phone displays the keyboard interface of the first input method; the memo (second application) on the virtual screen can display the keyboard interface of the second input method, and the target area of ​​the vehicle computer can display the keyboard interface of the second input method. The user can input on the mobile phone and the vehicle computer at the same time.

[0130] Based on the method in this embodiment, the virtual screen and the physical screen of the first electronic device correspond to the same user. When the first electronic device performs multi-screen input, it is not necessary to create a user for each screen, thereby avoiding the consumption of computing resources caused by creating users and reducing costs. When implementing multi-screen input, there is no need to isolate user data. One user can correspond to multiple screens, avoiding more system overhead and having less impact on data storage, performance, power consumption, etc. In this embodiment, the multiple screen inputs of a single user do not affect each other, and multiple screens can be input simultaneously without interfering with each other, thereby improving the user experience.

[0131] The method in the embodiment of the present application can be specifically implemented through the input method framework of the first electronic device. Figure 8 This is a diagram of an input method framework provided by an embodiment of the present application. Figure 8 As shown, the input method framework may include the input method client InputMethodManager, the input method management service InputMethodManagerService and the input method service InputMethodService. Among them, the input method client can be used to initialize an input method client instance for each application, and the input method client instance can run in the process that needs to use the input method, such as the process of social, browser and other applications. The input method management service is a global system service that can be used to manage all interactions between processes, such as managing the binding, hiding, and display of input methods. The input method management service includes the input method binding control class InputMethodBindingController, and the input method binding control class can be used to control different screen identifiers to bind different input method applications. The input method service is a specific input method application, such as the first input method and the second input method mentioned above, which is an application that truly implements the input method interface and controls character input.

[0132] Figure 8 The DisplayId in is the screen identifier. When an application is started on the screen whose DisplayId is x, the input method client can be initialized to obtain an input method client instance x.

[0133] The input method management service can associate and save the input method client instance and the screen ID. Based on the ClientState, the addClient interface is called, and the input method management service associates and saves the input method client instance and the corresponding DisplayId. The addClient interface is an interface used to add client-related information in software development. ClientState is the client state, which can represent an application in the input method management service.

[0134] Different input method binding control classes can correspond to an application on the screen respectively, and each application can be associated with the corresponding input method service. When the input method is not bound, you can initialize an input method binding control class according to DisplayId, and bind the input method to the corresponding application according to DisplayId in the input method binding control class. The input method can be bound by associating the input method client instance and the startInput method of the input method service. startInput is a key method in the input method framework, which is used to start the input process of the input method.

[0135] When performing input operations, the input method management service can use the window token as a parameter to call the display input method showSoftinput or hide input method hideSoftinput through DisplayId. The input method management service determines the corresponding input method based on the DisplayId in the window token, and calls the corresponding input method service showSoftinput or hideSoftinput to display or hide the keyboard interface of the input method.

[0136] Based on the input method framework, the process of associating the screen with the input method can be as follows Fig. 9 shown.

[0137] like Fig. 9 As shown, when the application (APP) on the physical screen is turned on, the APP can obtain the screen identifier and call the addClient of the input method client to initialize the input method client. The input method management service associates and saves the APP's ClientState and the physical screen identifier. The input method management service binds the APP's startInput class to the startInput class of the first input method APP1, which is equivalent to associating the physical screen identifier with the first input method.

[0138] Similarly, when starting the APP in the virtual screen, the input method management service binds the startInput class of the APP to the startInput class of the second input method APP2, thereby associating the virtual screen identifier and the input method APP2.

[0139] Based on the input method framework, the process of displaying the input method on the screen can be shown as 10.

[0140] like Fig.10 As shown in the figure, when the input box of the APP in the physical screen is clicked, the first input operation can be triggered. The input box is a View, and the View obtains the unique window token of the application window through the view.getWindowToken() method; and passes the window token as a parameter to the input method client. The view.getWindowToken() method is the above-mentioned window token acquisition method.

[0141] Based on the window token, the input method client calls the showSoftInput method to display the software disk. When calling the show software disk method, the input method client can send the window token as a parameter value to the input method management service to trigger the input method management service to call the show software disk method. The input method management service determines that the input method service to be called is the first input method based on the physical screen identifier in the window token, and sends a show request to the first input method. After receiving the show request, the first input method can generate a keyboard interface and display the keyboard interface of the first input method on the physical screen.

[0142] Similarly, when clicking the input box of the APP in the virtual screen, the second input operation can be triggered and a window token can be obtained. Based on the window token, the display software disk method of the second input method can be called to display the keyboard interface of the second input method on the virtual screen.

[0143] based on Figure 8 The input method framework can be implemented Fig.11 As shown in the multi-screen input. Fig.11 As shown:

[0144] S1101, start a first application in the mobile phone.

[0145] S1102: The first application program requests an input method on the physical screen.

[0146] S1103, project the mobile phone screen onto the car computer.

[0147] The mobile phone and the car computer are connected through screen projection through communication connection.

[0148] S1104, the mobile phone creates a virtual screen.

[0149] As an example, a sky window module may be developed in advance, based on which a virtual screen may be displayed in a target area of ​​a physical screen of the vehicle computer, wherein the target area may be referred to as a sky window.

[0150] S1105, start the second application in the vehicle computer.

[0151] S1106: The second application program requests an input method on the virtual screen.

[0152] S1107, determining whether the screen is associated with an input method according to the screen identifier.

[0153] S1108: If the screen identifier is not associated with the input method, associate the screen identifier with the input method.

[0154] S1109: If the screen identifier is associated with an input method, determine the associated input method according to the screen identifier.

[0155] S1110, displaying the input method on the corresponding screen.

[0156] As an example, the driver sits in the driver's seat and the passenger sits in the co-pilot seat. When the passenger in the co-pilot seat does not carry a mobile phone, in order to avoid the passenger's boredom on the road, the driver can project the mobile phone onto the co-pilot screen. At this time, the driver can view information or use applications through the mobile phone. The passenger can use applications through the co-pilot screen. The input operations of the driver and the passenger do not affect each other. In addition, when the driver is driving, it is inconvenient to operate the mobile phone, so the mobile phone can be projected onto the central control screen to facilitate the use of mobile phone services. Based on the method in this embodiment, the input operations on the mobile phone, the central control screen and the co-pilot screen do not interfere with each other, which can improve the user experience.

[0157] An embodiment of the present application also provides a screen projection system, including a first electronic device and one or more second electronic devices, wherein a screen projection connection is provided between the first electronic device and the second electronic device, and multi-screen input can be performed based on the first electronic device and the second electronic device.

[0158] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0159] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0160] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A multi-screen input method, characterized in that: The method is applied to a first electronic device, the first electronic device is communicatively connected with a second electronic device, the first electronic device includes a physical display screen and a virtual display screen, the virtual display screen is used for displaying on the second electronic device, and the method includes: Associating a first input method corresponding to the physical display screen and associating a second input method corresponding to the virtual display screen; Obtaining a screen identifier, wherein the screen identifier includes a virtual screen identifier and a physical screen identifier, and the screen identifier is used to indicate the screen on which the user operation is performed; When the screen identifier is the virtual screen identifier, displaying the second input method on the virtual display screen; When the screen identifier is the physical screen identifier, the first input method is displayed on the physical display screen.

2. The method according to claim 1, characterized in that The first electronic device is installed with a first application, and the associating with the first input method corresponding to the physical display screen includes: Receiving a first startup operation of a user on the physical display screen, where the first startup operation is used to instruct to start the first application; Displaying a user interface of the first application on the physical display screen; In response to starting the first application, acquiring the physical screen identifier; The first application is associated with the first input method based on the physical screen identifier.

3. The method according to claim 2, characterized in that The associating the first application with the first input method based on the physical screen identifier includes: Determine whether the physical screen identifier has an associated first input method; If there is an associated first input method, associating the first application with the first input method; If there is no associated first input method, the physical screen identifier is associated with the first input method, and the first application is associated with the first input method.

4. The method according to claim 1, characterized in that The first electronic device is installed with a second application program, and the associating the second input method corresponding to the virtual display screen includes: receiving a second startup operation from the second electronic device, where the second startup operation includes the virtual screen identifier, and the second startup operation is used to instruct to start the second application; Displaying a user interface of the second application on the virtual display screen; The second application is associated with the second input method based on the virtual screen identifier.

5. The method according to claim 4, characterized in that The associating the second application with the second input method based on the virtual screen identifier includes: Determine whether there is an associated second input method for the virtual screen identifier; If there is an associated second input method, associating the second application with the second input method; If there is no associated second input method, the virtual screen identifier is associated with the second input method, and the second application is associated with the second input method.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Based on the first input method, the second input method is generated and installed, where the second input method is an application clone of the first input method.

7. A multi-screen input method, characterized in that: The method is applied to a second electronic device, the second electronic device is communicatively connected to a first electronic device, the first electronic device includes a physical display screen and a virtual display screen, the virtual display screen is used for displaying on the second electronic device, and the method includes: Detecting a second input operation of a user on the second electronic device, where the second input operation is performed on an input box of a second user interface, where the second user interface is a user interface of a second application of the first electronic device, and the second user interface is displayed on the virtual display screen; The second input operation is sent to the first electronic device, where the second input operation is used to trigger the first electronic device to obtain a window token of the second user interface, where the window token includes a virtual screen identifier, and the virtual screen identifier is used to display a keyboard interface of a second input method on the second user interface.

8. An electronic device, characterized in that: include: One or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

10. A screen projection system, characterized in that: The device comprises a first electronic device and one or more second electronic devices, wherein the first electronic device and the second electronic device are connected to each other by communication. The second electronic device executes the method according to claim 7; The first electronic device executes the method according to any one of claims 1 to 6.

Images