Interaction event processing method and device

Abstract

The present application provides a method and device for processing interactive events, the method comprising: obtaining a key combination event, the key combination event being an interactive event for triggering a zoom function; converting the key combination event into a zoom touch event; and zooming the interface in response to the zoom touch event. The solution mainly achieves the effect of controlling the zoom function of a touch-screen electronic device through a keyboard and mouse by converting an unresponsive key combination event into a responsive zoom touch event and then responding to the event, and the solution is easy to implement.

Description

Technical Field

[0001] The present application relates to the field of electronic information technology, and in particular to a method and device for processing interactive events. Background Art

[0002] With the development of science and technology, people's use of touch-screen electronic devices such as mobile phones and tablets is no longer limited to direct control of these electronic devices. There are also situations where external input devices are used for control, and it is possible to project these touch-screen electronic devices onto other electronic devices for control. For example, you can connect an external keyboard and mouse to a touch-screen electronic device such as a mobile phone, and use the mobile phone as an office computer, using the keyboard and mouse to control the touch-screen electronic device. Or you can project the mobile phone screen onto a laptop or computer for joint office work, and then use the laptop's built-in keyboard to control the touch-screen electronic device, or use the laptop or computer's external keyboard and mouse to control the touch-screen electronic device.

[0003] In the above scenario, taking a mobile phone as an example, since the keyboard and mouse are essentially designed for computers, the control of the keyboard and mouse cannot be fully matched with the mobile phone. In other words, although the keyboard and mouse can realize the control of the computer, when the control object becomes a mobile phone, some original control functions on the mobile phone cannot be realized. For example, when using the keyboard and mouse to control the mobile phone, there is currently no way to realize the zoom function on the mobile phone.

[0004] Therefore, how to achieve zooming on a touch-screen electronic device through the manipulation of a keyboard and a mouse is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a method and device for processing interactive events, which can realize zooming on a touch-screen electronic device through the manipulation of a keyboard and a mouse.

[0006] In a first aspect, a method for processing an interactive event is provided, the method comprising: obtaining a combination key event, the combination key event being an interactive event for triggering a zoom function; converting the combination key event into a zoom touch event; and zooming an interface in response to the zoom touch event.

[0007] In the technical solution of the present application, the unresponsive key combination event is mainly converted into a responsive zoom touch event and then responded to, thereby achieving the effect of controlling the zoom function of the touch screen electronic device through the keyboard and mouse. The solution is easy to implement and does not require too many changes.

[0008] In combination with the first aspect, in some implementations of the first aspect, when converting a combination key event into a zoom touch event, the process may include: converting parameters of the combination key event into corresponding parameters in the zoom touch event.

[0009] In combination with the first aspect, in certain implementations of the first aspect, the combination key event is a combination event of a control key event and a mouse wheel event; the parameters of the combination key event include the scrolling direction and scrolling amount of the wheel; the parameters of the zoom touch event include the pinching direction and the pinching distance; the parameters of the combination key event are converted into corresponding parameters in the zoom touch event, including: converting the scrolling amount into the pinching distance, and converting the scrolling direction into the pinching direction.

[0010] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0011] Determine whether the key combination event is a zoom key combination event;

[0012] When the key combination event is a zoom key combination event, executing a step of converting the key combination event into a zoom touch event; or

[0013] When the key combination event is not a zoom key combination event, the processing flow of the interaction event ends.

[0014] This implementation can shield the input of pure control key events, or avoid mistakenly responding to pure scroll wheel events as zoom key combinations.

[0015] In combination with the first aspect, in some implementations of the first aspect, when converting the combination key event into a zoom touch event, the method may further include:

[0016] Determine whether the conversion of the previous round of key combination events is still in progress;

[0017] When the conversion of the previous round of key combination events is still in progress, the key combination event is discarded; or

[0018] When the conversion of the previous round of key combination events is not being executed, the key combination events are converted into zoom touch events.

[0019] This implementation can improve the accuracy of processing continuously triggered combination key events.

[0020] In combination with the first aspect, in some implementations of the first aspect, when obtaining a combination key event, the following may be included:

[0021] Get key combination events through hardware drivers or collaborative office software.

[0022] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0023] Determine whether the foreground application supports scaling;

[0024] When the foreground application supports zooming, in response to a zoom touch event, zooming the interface includes: in response to the zoom touch event, zooming the running interface of the foreground application; or

[0025] When the foreground application does not support zooming, the interaction event processing flow ends.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, when determining whether the foreground application supports zooming, the method may include:

[0027] Compare the information of the foreground application with the pre-configured information of the whitelist application to determine whether the foreground application is an application in the whitelist application, where the whitelist application includes at least one application that supports a zoom function;

[0028] When the foreground application is an application in the whitelist, the foreground application is considered to support scaling; or

[0029] When the foreground application is not an application in the whitelist applications, it is considered that the foreground application does not support scaling.

[0030] In combination with the first aspect, in some implementations of the first aspect, the method further includes:

[0031] After the zoom touch event is generated, the modifier key event is discarded.

[0032] In a second aspect, a device for processing interactive events is provided, the device comprising a unit composed of software and / or hardware for executing any one of the methods in the first aspect.

[0033] In one embodiment, the device comprises:

[0034] An acquisition unit, used for acquiring a key combination event, where the key combination event is an interactive event for triggering a zoom function;

[0035] A processing unit for:

[0036] Convert key combination events to zoom touch events;

[0037] In response to a zoom touch event, the interface is zoomed.

[0038] In combination with the second aspect, in some implementations of the second aspect, the processing unit is specifically configured to: convert parameters of the combination key event into corresponding parameters in the zoom touch event.

[0039] In conjunction with the second aspect, in some implementations of the second aspect, the combination key event is a combination event of a control key event and a mouse wheel event; the parameters of the combination key event include a scrolling direction and a scrolling amount of the wheel; the parameters of the zoom touch event include a pinching direction and a pinching distance; and the processing unit is specifically configured to:

[0040] Convert scroll amount to pinch distance and scroll direction to pinch direction.

[0041] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to:

[0042] Determine whether the key combination event is a zoom key combination event;

[0043] When the key combination event is a zoom key combination event, executing a step of converting the key combination event into a zoom touch event; or

[0044] When the key combination event is not a zoom key combination event, the processing flow of the interaction event ends.

[0045] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to:

[0046] Determine whether the conversion of the previous round of key combination events is still in progress;

[0047] When the conversion of the previous round of key combination events is still in progress, the key combination event is discarded; or

[0048] When the conversion of the previous round of key combination events is not being executed, the key combination events are converted into zoom touch events.

[0049] In conjunction with the second aspect, in some implementations of the second aspect, the acquiring unit is specifically configured to:

[0050] Get key combination events through hardware drivers or collaborative office software.

[0051] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to:

[0052] Determine whether the foreground application supports scaling;

[0053] When the foreground application supports zooming, in response to a zoom touch event, zooming the interface includes: in response to the zoom touch event, zooming the running interface of the foreground application; or

[0054] When the foreground application does not support zooming, the interaction event processing flow ends.

[0055] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is specifically configured to:

[0056] Compare the information of the foreground application with the pre-configured information of the whitelist application to determine whether the foreground application is an application in the whitelist application, where the whitelist application includes at least one application that supports a zoom function;

[0057] When the foreground application is an application in the whitelist, the foreground application is considered to support scaling; or

[0058] When the foreground application is not an application in the whitelist applications, it is considered that the foreground application does not support scaling.

[0059] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to:

[0060] After the zoom touch event is generated, the modifier key event is discarded.

[0061] According to a third aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the methods of the first aspect can be implemented.

[0062] In a fourth aspect, a chip is provided, comprising a processor, wherein the processor is used to read and execute a computer program stored in a memory, and when the computer program is executed by the processor, any one of the methods of the first aspect can be implemented.

[0063] Optionally, the chip also includes a memory, and the memory is electrically connected to the processor.

[0064] Optionally, the chip may also include a communication interface.

[0065] According to a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of the methods of the first aspect can be implemented.

[0066] According to a sixth aspect, a computer program product is provided. The computer program product comprises a computer program, and when the computer program is executed by a processor, any one of the methods of the first aspect can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of an interaction scenario of an embodiment of the present application.

[0068] Figure 2 It is a schematic diagram of another interaction scenario of an embodiment of the present application.

[0069] Figure 3 It is a schematic diagram of the response process of the zoom function.

[0070] Figure 4 It is a structural schematic diagram of an electronic device according to an embodiment of the present application.

[0071] Figure 5 It is a software structure block diagram of an electronic device according to an embodiment of the present application.

[0072] Figure 6 It is a schematic diagram of the interactive event processing process of the first type of interactive scenario in an embodiment of the present application.

[0073] Figure 7 It is a schematic diagram of the interactive event processing process of the second type of interactive scenario in an embodiment of the present application.

[0074] Figure 8 It is a schematic flowchart of a method for processing an interactive event in an embodiment of the present application.

[0075] Fig. 9 It is a schematic flowchart of another method for processing an interactive event according to an embodiment of the present application.

[0076] Fig.10 It is a schematic flowchart of another method for processing an interaction event according to an embodiment of the present application.

[0077] Fig.11 It is a schematic diagram of a device for processing interactive events according to an embodiment of the present application. DETAILED DESCRIPTION

[0078] The following is an introduction to the scheme of the embodiment of the present application in conjunction with the accompanying drawings. The method for processing interactive events provided by the present application can be applied to keyboard and mouse control scenarios of various touch-screen electronic devices. The method for processing interactive events provided by the present application can be used in touch-screen electronic devices that can support external keyboard and mouse devices for control.

[0079] Figure 1 It is a schematic diagram of an interactive scenario of an embodiment of the present application. It should be understood that the embodiment of the present application mainly includes two types of interactive scenarios. The first type of interactive scenario is that the touch-screen electronic device is directly connected to an external keyboard and mouse, and the touch-screen electronic device is controlled by the external keyboard and mouse; the second type of interactive scenario is that the touch-screen electronic device is projected to a projection device, and the touch-screen electronic device is controlled by the external keyboard and mouse of the projection device. Figure 1 is an example of the first type of interaction scenario. Figure 2 is an example of the second type of interaction scenario.

[0080] like Figure 1 As shown, in this scenario, the electronic device D is a touch screen electronic device, that is, an electronic device that can be operated by touching the screen, such as clicking, double-clicking, or sliding. A touch screen electronic device can also be called a touch screen device or a touch screen terminal. The electronic device D can be, for example, a mobile phone or a tablet computer, or other electronic devices that can be operated by touch screen. Figure 1 The electronic device D is mainly taken as a tablet computer as an example. It should be understood that the main input method of the electronic device D is touch screen operation, but there may also be an input method of key operation using mechanical keys such as a power key and a volume key.

[0081] That is to say, the touch screen electronic device in the embodiment of the present application mainly refers to the original main input method is touch screen operation, and key operation is not its original main input method, that is to say, the keyboard and mouse are not the original input devices of the touch screen electronic device, but only the input devices extended from it. For example, for mobile phones, the touch operation of the screen is its original main input method. It is with the development of science and technology that the use scenario of external keyboard and mouse to control the mobile phone has emerged. However, since the keyboard and mouse are essentially designed for computers, the control of the keyboard and mouse cannot be fully matched with the mobile phone. In other words, although the keyboard and mouse can realize the control of the computer, when the control object becomes a mobile phone, some original control functions on the mobile phone cannot be realized.

[0082] exist Figure 1 In the illustrated scenario, electronic device D is connected to keyboard B and mouse C via wireless connection. The wireless connection method may be, for example, Bluetooth connection, but it should be understood that it may also be other wireless connection methods. In addition, the wireless connection method may also be replaced by a wired connection method, and there is no limitation. There is no limitation on the shape, model, and key layout of keyboard B and mouse C, for example Figure 1 The Fn key in the keyboard B shown may not exist, or its actual position may be different from the position shown in the figure. For example, the icon of the direction key may be in other forms of representation. For example, there may also be a numeric keypad area, that is, a small keypad area. For example, there may be a volume button, etc. Other possible situations will not be described one by one.

[0083] exist Figure 1 In the scenario shown, user A can control electronic device D by operating keyboard B and mouse C.

[0084] It should be noted that in the embodiments of the present application, interactive events include key events and touch events. Key events are used to represent interactive events generated by operating keys or buttons on a keyboard and / or mouse. Touch events are used to represent interactive events generated by touching the screen. It should be understood that the touch event here is an interactive event generated by a touch operation on the screen of a touch-screen electronic device, which is different from the touchpad (touchpad) on a laptop computer.

[0085] If user A pushes the scroll wheel (scroll lock) of mouse C while pressing the ctrl key of keyboard B, a combination key event of ctrl+scroll wheel will be formed. It should be understood that pushing the scroll wheel includes scrolling up and scrolling down, that is, the scroll wheel can have two scrolling directions, and the two scrolling directions are described as upward and downward respectively. In the operation of the external keyboard and mouse of a traditional computer, ctrl+scroll wheel can realize the zoom function, wherein ctrl+up scroll wheel can realize the zoom-in function, and ctrl+down scroll wheel can realize the zoom-out function. If it is just a simple scroll wheel operation, the page turning function is realized, and the upward scroll wheel corresponds to the upward page turning, and the downward scroll wheel corresponds to the downward page turning. It should be understood that since the ctrl key is a control key, simply pressing the ctrl key will not cause the controlled device to generate a corresponding operation, so the ctrl in the ctrl+scroll wheel combination key event can be regarded as the state value of the scroll wheel event, that is, the ctrl+scroll wheel combination key event can be regarded as a scroll wheel event carrying the ctrl state value.

[0086] For current touch-screen electronic devices, if they are connected to a keyboard and mouse for control, even if a ctrl+scroll wheel key combination event is generated by operating the keyboard and mouse, the touch-screen electronic device cannot respond to the key combination event, that is, zooming cannot be achieved.

[0087] In view of the above problems, the solution of the embodiment of the present application mainly increases some processing of the combination key events, thereby realizing the control of the zoom of the touch screen electronic device through the combination key events. Figure 1 In the illustrated scenario, if user A presses the ctrl key of keyboard B and scrolls the mouse wheel of mouse C, through the solution of the embodiment of the present application, electronic device D processes the ctrl+scroll key combination event, and the interface on electronic device D can be zoomed. The specific process will be described below and will not be expanded for the time being.

[0088] Figure 2 is a schematic diagram of another interactive scenario of an embodiment of the present application. As described above, Figure 2 is an example of the second type of interaction scenario. Figure 2 As shown, in this scenario, the electronic device D is a touch screen electronic device, which can be referred to Figure 1 Related description of electronic device D.

[0089] exist Figure 2In the scenario shown, electronic device D projects its screen to electronic device E through near field communication, that is, short-range wireless communication technology. However, it should be understood that the embodiment of the present application does not limit the communication method between electronic device D and electronic device E. Electronic device E can be called a screen projection device. Electronic device E can be an electronic device such as a laptop or a computer that can be controlled by a keyboard and a mouse. For laptops, they usually come with their own keyboard, so you can use the laptop's own keyboard and an external mouse for control. For computers, an external keyboard and mouse are usually required for control.

[0090] It should be understood that electronic device E and electronic device D are two types of electronic devices with different control methods. Electronic device D is a touch screen electronic device, that is, an electronic device that can be controlled by touch operations such as clicking, double-clicking or sliding on the screen. The main input method of electronic device D is touch screen operation. And electronic device E is an auxiliary device of electronic device D in the use scenario of joint office or collaborative office. The main function of electronic device E is to provide a screen for electronic device D, synchronously display the screen image of electronic device D, and the interface and input and output devices of electronic device E can be called by electronic device D. However, it should be understood that in the use scenario of collaborative office, the processing of data, such as the processing of input interactive events is still performed by electronic device D.

[0091] As mentioned above, the touch screen electronic device in the embodiment of the present application mainly refers to the original main input mode of the touch screen operation, and the key operation is not its original main input mode, that is, the keyboard and mouse are not the original input devices of the touch screen electronic device, but only the input devices extended from it. For the projection screen device, the original main input mode is key operation, that is, the keyboard and mouse are the original input devices of the projection screen device, not the input devices extended from it.

[0092] exist Figure 2 In the illustrated scenario, the electronic device E is connected to a keyboard B and a mouse C via a wired connection. However, it should be understood that a wireless connection may also be used, and there is no limitation. The description of the keyboard B and the mouse C can also be referred to in Figure 1 Related content.

[0093] exist Figure 2 In the scenario shown, user A can control electronic device D by operating keyboard B and mouse C.

[0094] If user A pushes the ctrl key of keyboard B and the scroll wheel of mouse C at the same time, a combination key event of ctrl+scroll wheel is formed. The combination key event is reported to electronic device E, and then transmitted to electronic device D through the communication channel between electronic device D and electronic device E. However, since electronic device D cannot respond to this combination key event, electronic device D cannot achieve zooming.

[0095] To address this problem, in the solution of the embodiment of the present application, some processing of the combination key event by the electronic device D is added, so as to realize the zoom function of the touch screen electronic device by the combination key event. Figure 2 In the scenario shown, if user A scrolls the mouse wheel of mouse C while pressing the ctrl key of keyboard B, electronic device D processes the ctrl+scroll key combination event using the solution of the embodiment of the present application, and the interface of electronic device D can be scaled. It should be understood that since electronic device E is the screen projection device of electronic device D, the interface of electronic device D is synchronized with the screen projection interface of electronic device D on the screen of electronic device E. Therefore, the interface on electronic device D and the screen projection interface of electronic device D on electronic device E will be scaled synchronously.

[0096] In one example, electronic device D is running an Android operating system, and electronic device E is running a Windows operating system.

[0097] from Figure 1 and Figure 2 It can be seen that in the embodiments of the present application, zooming on the touch-screen electronic device is achieved through the manipulation of the keyboard and mouse, including: zooming in and out of the interface on the touch-screen electronic device through an external keyboard and mouse of the touch-screen electronic device; or zooming in and out of the interface of the touch-screen electronic device and the projection interface of the touch-screen electronic device on the projection device through the keyboard and mouse of the projection device of the touch-screen electronic device.

[0098] It should be noted that, although there is a type of laptop computer that can combine the functions of an ordinary laptop computer and a tablet computer, that is, it has the processing capabilities of both touch screen interaction events and keyboard and mouse interaction events, and can be called a touch screen laptop computer. However, these laptop computers are all Windows operating systems, and these two types of interaction events correspond to two independent processing flows. If such a touch screen laptop computer is directly operated, its zoom function is realized by directly responding to the user's touch operation or directly responding to the user's keyboard and mouse operation, that is, the user can zoom by touching the screen, or by keyboard and mouse operation. This dual processing flow method cannot be used in devices such as electronic device D, because if this dual processing flow is forcibly applied to electronic device D, electronic device D must adopt the Windows system, and the software architecture and supporting hardware architecture of the entire electronic device must be changed and redesigned, but this change is equivalent to completely transforming a mobile phone into a laptop computer, which is costly and complicated, and there is no need for transformation. Because there is no need to forcibly transform an existing product category into another existing product category. In short, the above-mentioned touch-screen laptop computer originally has input devices for both touch operation and key operation, and has been equipped with processing devices and processes for two types of interactive events. Therefore, this type of touch-screen laptop computer is not the touch-screen electronic device described in the embodiment of the present application, that is, electronic device D is not this type of touch-screen laptop computer.

[0099] However, a touch-screen laptop can be used as a screen projection device in the embodiment of the present application, that is, the electronic device E can be a touch-screen laptop. If a touch-screen electronic device such as a mobile phone or a tablet is projected onto a touch-screen laptop, taking a mobile phone as an example, even if the keyboard and an external mouse of the touch-screen laptop are used for zooming, since the touch-screen laptop only acts as a data carrier, the mobile phone is still the one that actually performs the calculation operation. Therefore, at this time, even if the keyboard and mouse connected to the touch-screen laptop are operated, zooming still cannot be performed, and the scheme of the embodiment of the present application is still required to convert the key events of the keyboard and mouse into touch events before responding, so as to achieve the zooming of the mobile phone interface and the projection interface of the mobile phone on the touch-screen laptop.

[0100] Figure 3 This is a schematic diagram of the response process of the scaling function. Figure 3 As shown, if a zoom touch operation is performed on the interface 310, that is, two fingers slide in a direction in which the distance between the fingers increases, a zoom touch event can be generated. Figure 3The zoom-in touch event is represented by two-finger pinch #1, and the electronic device processes the two-finger pinch #1 touch event to obtain interface 320. It can be seen that the picture A in interface 320 is larger than the picture A in interface 310, that is, the proportion of picture A on the screen of the electronic device is increased.

[0101] If the zoom key operation is performed on the interface 310, that is, the scroll wheel is pushed upward while the ctrl key is pressed, a combination key event of ctrl+scroll wheel upward can be generated. Figure 3 The combination key event of zooming in is represented by ctrl+upward scroll wheel, and the electronic device processes the combination key event of ctrl+upward scroll wheel to obtain interface 320. That is, both the zoom combination key event and the zoom touch event can zoom in the interface of the electronic device.

[0102] If a pinch-out touch operation is performed on the interface 330, that is, two fingers slide in a direction in which the distance between the fingers decreases, a pinch-out touch event may be generated. Figure 3 The zoom-out touch event is represented by two-finger pinch #2, and the electronic device processes the two-finger pinch #2 touch event to obtain interface 340. It can be seen that the picture A in interface 340 is smaller than the picture A in interface 330, that is, the proportion of the picture A on the screen of the electronic device is reduced.

[0103] If the interface 330 is zoomed out, that is, the scroll wheel is pushed down while the ctrl key is pressed, a combination key event of ctrl+scroll wheel down is generated. Figure 3 The combination key event of shrinking is represented by ctrl+roll down, and the electronic device processes the combination key event of ctrl+roll down to obtain interface 340. That is, both the shrink combination key event and the shrink touch event can shrink the interface of the electronic device.

[0104] Figure 3It can be regarded as an example of an electronic device's foreground application being a picture viewing application. The zoom function realized by a two-finger pinch touch event is a function that the electronic device originally possesses, and the solution of the embodiment of the present application is to realize the zoom function by the key operation of ctrl+scroll wheel. In the embodiment of the present application, it is mainly considered that the touch screen electronic device itself already has a touch operation corresponding to the zoom function, so as long as the combination key event of the zoom is converted into a zoom touch event, the electronic device can respond to the zoom touch event. Or it can be understood as establishing a mapping relationship between the key event and the touch event of the zoom function, so that when the combination key event of the zoom is input to the electronic device, the electronic device only needs to add a conversion step, first convert the zoom combination key event into a zoom touch event, and then continue to process according to the original processing flow for the touch event, so that from the user's perspective, the control of the zoom function of the electronic device through the keyboard and mouse is realized. This solution is to add an additional conversion step to the existing touch event processing flow, so the transformation difficulty is relatively small and easy to implement. There is no need to change the hardware structure or redesign the complex processing flow.

[0105] Ctrl+Scroll Wheel is the most commonly used key operation for zooming, so using this key combination event to trigger the zooming function is more in line with user habits. However, it should be understood that other key events can also be designed to replace this key combination event, but some adaptive changes need to be added, and it is necessary to explain to the user that the corresponding function of other key events is zooming. For example, a key combination event that is not currently occupied can be used to set the key combination operation for the zooming function.

[0106] It can be seen that Figure 3 It is mainly shown that zooming can be achieved through touch operation or combination key operation, that is, a zoom touch event and a zoom combination key event are compared to illustrate that the two can achieve the same effect. Figure 3 This is mainly to illustrate that the solution of the embodiment of the present application can enable the combination key event of zooming to achieve the same zooming effect as the two-finger pinch touch event.

[0107] Figure 4 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4As shown, the electronic device 400 may include a processor 410, an external memory interface 420, an internal memory 421, a universal serial bus (USB) interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, an antenna 2, a mobile communication module 450, a wireless communication module 460, an audio module 470, a speaker 470A, a receiver 470B, a microphone 470C, an earphone interface 470D, a sensor module 480, a button 490, a motor 491, an indicator 492, a camera 493, a display screen 494, and a subscriber identification module (SIM) card interface 495, etc. Among them, the sensor module 480 may include a pressure sensor 480A, a gyroscope sensor 480B, an air pressure sensor 480C, a magnetic sensor 480D, an acceleration sensor 480E, a distance sensor 480F, a proximity light sensor 480G, a fingerprint sensor 480H, a temperature sensor 480J, a touch sensor 480K, an ambient light sensor 480L, a bone conduction sensor 480M, etc.

[0108] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 400. In other embodiments of the present application, the electronic device 400 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0109] For example, Figure 4 The processor 410 shown may include one or more processing units, for example, the processor 410 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0110] The controller may be the nerve center and command center of the electronic device 400. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0111] The processor 410 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. The memory may store instructions or data that the processor 410 has just used or cyclically used. If the processor 410 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0112] In some embodiments, the processor 410 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0113] In some embodiments, the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The processor 410 may include multiple groups of I2C buses. The processor 410 may be coupled to the touch sensor 480K, the charger, the flash, the camera 493, etc. through different I2C bus interfaces. For example, the processor 410 may be coupled to the touch sensor 480K through the I2C interface, so that the processor 410 communicates with the touch sensor 480K through the I2C bus interface to realize the touch function of the electronic device 400.

[0114] In some embodiments, the I2S interface can be used for audio communication. The processor 410 can include multiple I2S buses. The processor 410 can be coupled to the audio module 470 via the I2S bus to achieve communication between the processor 410 and the audio module 470.

[0115] In some embodiments, the audio module 470 can transmit audio signals to the wireless communication module 460 through the I2S interface to implement the function of answering calls through a Bluetooth headset.

[0116] In some embodiments, the PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. The audio module 470 and the wireless communication module 460 can be coupled via the PCM bus interface.

[0117] In some embodiments, the audio module 470 can also transmit audio signals to the wireless communication module 460 through the PCM interface to implement the function of answering calls through a Bluetooth headset. It should be understood that both the I2S interface and the PCM interface can be used for audio communication.

[0118] In some embodiments, the UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. The UART interface is generally used to connect the processor 410 and the wireless communication module 460. For example, the processor 410 communicates with the Bluetooth module in the wireless communication module 460 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 470 can transmit an audio signal to the wireless communication module 460 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0119] In some embodiments, the MIPI interface can be used to connect the processor 410 with peripheral devices such as the display screen 494 and the camera 493. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. The processor 410 and the camera 493 communicate via the CSI interface to realize the shooting function of the electronic device 400. The processor 410 and the display screen 494 communicate via the DSI interface to realize the display function of the electronic device 400.

[0120] In some embodiments, the GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. The GPIO interface can be used to connect the processor 410 with the camera 493, the display screen 494, the wireless communication module 460, the audio module 470, the sensor module 480, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0121] Exemplarily, the USB interface 430 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 400, and can also be used to transfer data between the electronic device 400 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0122] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 400. In other embodiments of the present application, the electronic device 400 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0123] The charging management module 440 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 440 may receive charging input from a wired charger through the USB interface 430. In some wireless charging embodiments, the charging management module 440 may receive wireless charging input through a wireless charging coil of the electronic device 400. While the charging management module 440 is charging the battery 442, it may also power the electronic device through the power management module 441.

[0124] The power management module 441 is used to connect the battery 442, the charging management module 440 and the processor 410. The power management module 441 receives input from the battery 442 and / or the charging management module 440, and supplies power to the processor 410, the internal memory 421, the external memory, the display screen 494, the camera 493, and the wireless communication module 460. The power management module 441 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 441 can also be set in the processor 410. In other embodiments, the power management module 441 and the charging management module 440 can also be set in the same device.

[0125] The wireless communication function of the electronic device 400 can be implemented through the antenna 1, the antenna 2, the mobile communication module 450, the wireless communication module 460, the modem processor and the baseband processor.

[0126] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 400 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0127] The mobile communication module 450 can provide a wireless communication solution applied to the electronic device 400, such as at least one of the following solutions: a second generation (2G) mobile communication solution, a third generation (3G) mobile communication solution, a fourth generation (5G) mobile communication solution, and a fifth generation (5G) mobile communication solution. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 450 can receive electromagnetic waves from the antenna 1, and filter and amplify the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 450 can also amplify the signal modulated by the modulation and demodulation processor, and the amplified signal is converted into an electromagnetic wave and radiated out through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 410. In some embodiments, at least some of the functional modules of the mobile communication module 450 can be set in the same device as at least some of the modules of the processor 410.

[0128] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 470A, a receiver 470B, etc.), or displays an image or video through a display screen 494. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 410 and be set in the same device as the mobile communication module 450 or other functional modules.

[0129] The wireless communication module 460 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 400. The wireless communication module 460 can be one or more devices integrating at least one communication processing module. The wireless communication module 460 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 410. The wireless communication module 460 can also receive the signal to be sent from the processor 410, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0130] In some embodiments, the antenna 1 of the electronic device 400 is coupled to the mobile communication module 450, and the antenna 2 of the electronic device 400 is coupled to the wireless communication module 460, so that the electronic device 400 can communicate with the network and other electronic devices through wireless communication technology. The wireless communication technology may include at least one of the following communication technologies: global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, IR technology. The GNSS may include at least one of the following positioning technologies: global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and satellite based augmentation systems (SBAS).

[0131] The electronic device 400 implements the display function through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 410 may include one or more GPUs, which execute program instructions to generate or change display information.

[0132] The display screen 494 is used to display images, videos, etc. The display screen 494 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, Micro-Led, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 400 may include 1 or N display screens 494, where N is a positive integer greater than 1.

[0133] The electronic device 400 can realize the shooting function through ISP, camera 493, video codec, GPU, display screen 494 and application processor.

[0134] ISP is used to process the data fed back by camera 493. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 493.

[0135] The camera 493 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 400 may include 1 or N cameras 493, where N is a positive integer greater than 1.

[0136] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 400 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0137] Video codecs are used to compress or decompress digital videos. The electronic device 400 may support one or more video codecs. Thus, the electronic device 400 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0138] NPU is a neural network (NN) computing processor, which can quickly process input information and continuously self-learn by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain. NPU can realize applications such as intelligent cognition of electronic device 400, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0139] The external memory interface 420 can be used to connect an external memory card, such as a secure digital (SD) card, to expand the storage capacity of the electronic device 400. The external memory card communicates with the processor 410 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0140] The internal memory 421 can be used to store computer executable program codes, which include instructions. The processor 410 executes various functional applications and data processing of the electronic device 400 by running the instructions stored in the internal memory 421. The internal memory 421 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 400 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 421 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0141] The electronic device 400 can implement audio functions such as music playing and recording through the audio module 470, the speaker 470A, the receiver 470B, the microphone 470C, the headphone jack 470D, and the application processor.

[0142] The audio module 470 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 470 can also be used to encode and decode audio signals. In some embodiments, the audio module 470 can be arranged in the processor 410, or some functional modules of the audio module 470 can be arranged in the processor 410.

[0143] The speaker 470A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 400 can listen to music or listen to a hands-free call through the speaker 470A.

[0144] The receiver 470B, also called a "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device 400 receives a call or a voice message, the voice can be received by placing the receiver 470B close to the human ear.

[0145] Microphone 470C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 470C to input the sound signal into microphone 470C. The electronic device 400 can be provided with at least one microphone 470C. In other embodiments, the electronic device 400 can be provided with two microphones 470C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 400 can also be provided with three, four or more microphones 470C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0146] The earphone interface 470D is used to connect a wired earphone and can be a USB interface 430 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0147] The pressure sensor 480A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 480A can be set on the display screen 494. There are many types of pressure sensors 480A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 480A, the capacitance between the electrodes changes. The electronic device 400 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 494, the electronic device 400 detects the touch operation intensity according to the pressure sensor 480A. The electronic device 400 can also calculate the touch position according to the detection signal of the pressure sensor 480A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0148] The gyroscope sensor 480B can be used to determine the motion posture of the electronic device 400. In some embodiments, the angular velocity of the electronic device 400 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 480B. The gyroscope sensor 480B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 480B detects the angle of the electronic device 400 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 400 through reverse movement to achieve anti-shake. The gyroscope sensor 480B can also be used for navigation and somatosensory game scenes.

[0149] The air pressure sensor 480C is used to measure air pressure. In some embodiments, the electronic device 400 calculates the altitude through the air pressure value measured by the air pressure sensor 480C to assist in positioning and navigation.

[0150] The magnetic sensor 480D includes a Hall sensor. The electronic device 400 can use the magnetic sensor 480D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 400 is a flip phone, the electronic device 400 can detect the opening and closing of the flip cover according to the magnetic sensor 480D; according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, set the flip cover automatic unlocking and other features.

[0151] The acceleration sensor 480E can detect the magnitude of the acceleration of the electronic device 400 in all directions (generally three axes). When the electronic device 400 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0152] The distance sensor 480F is used to measure the distance. The electronic device 400 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 400 can use the distance sensor 480F to measure the distance to achieve fast focusing.

[0153] The proximity light sensor 480G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The electronic device 400 emits infrared light outward through the light-emitting diode. The electronic device 400 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 400. When insufficient reflected light is detected, the electronic device 400 can determine that there is no object near the electronic device 400. The electronic device 400 can use the proximity light sensor 480G to detect that the user holds the electronic device 400 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 480G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0154] Ambient light sensor 480L is used to sense the brightness of ambient light. Electronic device 400 can adaptively adjust the brightness of display screen 494 according to the perceived brightness of ambient light. Ambient light sensor 480L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 480L can also cooperate with proximity light sensor 480G to detect whether electronic device 400 is in a pocket to prevent accidental touch.

[0155] The fingerprint sensor 480H is used to collect fingerprints. The electronic device 400 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0156] The temperature sensor 480J is used to detect temperature. In some embodiments, the electronic device 400 uses the temperature detected by the temperature sensor 480J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 480J exceeds a threshold, the electronic device 400 reduces the performance of the processor located near the temperature sensor 480J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 400 heats the battery 442 to avoid abnormal shutdown of the electronic device 400 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 400 performs a boost on the output voltage of the battery 442 to avoid abnormal shutdown caused by low temperature.

[0157] The touch sensor 480K is also called a "touch panel". The touch sensor 480K can be set on the display screen 494, and the touch sensor 480K and the display screen 494 form a touch screen, also called a "touch screen". The touch sensor 480K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 494. In other embodiments, the touch sensor 480K can also be set on the surface of the electronic device 400, which is different from the position of the display screen 494.

[0158] The bone conduction sensor 480M can obtain a vibration signal. In some embodiments, the bone conduction sensor 480M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 480M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 480M can also be set in an earphone and combined into a bone conduction earphone. The audio module 470 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 480M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 480M to realize a heart rate detection function.

[0159] The key 490 includes a power key, a volume key, etc. The key 490 may be a mechanical key or a touch key. The electronic device 400 may receive key input and generate key signal input related to user settings and function control of the electronic device 400.

[0160] Motor 491 can generate vibration prompts. Motor 491 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 494, motor 491 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0161] Indicator 492 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0162] The SIM card interface 495 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 400 by inserting it into the SIM card interface 495 or pulling it out from the SIM card interface 495. The electronic device 400 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 495 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 495 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 495 can also be compatible with different types of SIM cards. The SIM card interface 495 can also be compatible with external memory cards. The electronic device 400 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 400 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 400 and cannot be separated from the electronic device 400.

[0163] The software system of the electronic device 400 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The present application embodiment takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device. Figure 5 Taking an electronic device with an Android system having a layered architecture as an example, the software structure of the electronic device is described exemplarily.

[0164] Figure 5 It is a software structure block diagram of an electronic device according to an embodiment of the present application. Figure 5 The electronic devices in the Figure 4 The electronic device 400 shown, Figure 5 As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor; the layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely the application layer, the application framework layer, the system library layer, and the kernel layer. The application layer can include a series of application packages. However, it should be understood that Figure 5 This is just an example of how to layer the Android system. In actual scenarios, there may be other ways of layering. In addition, there may be other ways of naming the modules in each layer, and there is no limitation. For example, the application framework layer can be called the framework layer, or the system library layer can be merged into the application framework layer.

[0165] like Figure 5As shown, the application package may include short message, calendar, camera, video, navigation, call, gallery and other applications (application, APP). For convenience, the application is referred to as application below. The running application can also be called foreground application or front-end application.

[0166] Since some applications support the zoom function, such as short messages, gallery or navigation, and some applications do not support the zoom function, such as settings, remote control, etc., electronic devices of different manufacturers may be different. In short, applications can be divided into applications that support the zoom function and applications that do not support the zoom function. In the embodiment of the present application, the combination key control of the application that supports the zoom function can be realized.

[0167] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0168] like Figure 5 As shown, the application framework layer may include a window manager, a resource manager, a notification manager, an input manager, and the like.

[0169] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0170] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0171] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0172] The input manager (InputManagerService) is mainly used to manage the input part of the entire system, including keyboard, mouse, touch screen, etc.

[0173] The system library layer can include the Android runtime and the system library. The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0174] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0175] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0176] The system library may include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), an input service module, etc.

[0177] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0178] The media library supports playback and recording of multiple audio formats, playback and recording of multiple video formats, and still image files. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, moving picture experts group audio layer III (MP3), advanced audio coding (AAC), adaptive multi-rate (AMR), joint photo graphic experts group (JPG) and portable network graphics (PNG).

[0179] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing, etc.

[0180] The input service module may include InputReader and InputDispatcher. InputReader is used to read input events and report the read events to InputDispatcher, and InputDispatcher distributes the input events.

[0181] The kernel layer refers to the layer between hardware and software; the kernel layer includes at least sensor driver, camera driver, display driver, etc.

[0182] It should be noted that the above Figure 4A possible electronic device structure diagram is shown below. Figure 5 A possible software architecture diagram of an electronic device is shown in FIG. Figures 1 to 3 Any one of the touch screen electronic devices (electronic device D) shown in FIG. 1 may have Figure 4 The structure and Figure 5 The software structure shown.

[0183] Figure 6 It is a schematic diagram of the interactive event processing process of the first type of interactive scenario in an embodiment of the present application. Figure 6 The process shown can be applied to Figure 1 The interactive scene shown, Figure 6 The processing process can be executed by the above-mentioned electronic device D or the electronic device 400.

[0184] like Figure 6 As shown, the application layer includes multiple applications such as app1, app2, app3, and app4. The application running in the foreground can be called the foreground application. When the user inputs the ctrl+wheel key combination operation through the keyboard and mouse external to the touch-screen electronic device, the corresponding driver of the kernel layer captures the key combination event and reports it to the InputReader in the native framework layer. The InputReader reads the key combination event and reports it to the InputDispatcher in the native framework layer. The InputDispatcher can distribute the key combination event. In the scheme of the embodiment of the present application, the InputDispatcher will first distribute the key combination event to the InputFilter in the InputManagerService of the java framework layer. The InputFilter filters the key combination event before event distribution, and injects the key combination event into the InputDispatcher to form a corresponding touch event. Then the InputDispatcher distributes the touch event to the view framework, and then sends it to the foreground application through the view framework. Since the touch-screen electronic device (electronic device D or electronic device 400) can respond to the touch event. Specifically, the process is that InputDispatcher distributes events to the application process, and the view framework running in the application process further distributes and processes the events according to the view tree of the current interface layout, and passes the events to the corresponding application controls. Thus, the interface of the foreground application can be scaled.

[0185] In some cases, the native framework layer can be seen as Figure 5 The Java framework layer can be seen as an example of the system library layer. Figure 5 An example of the application framework layer shown in FIG. However, it should be understood that there is no unique layering method for the Android system, that is, it is not necessary to completely adopt the layering method. Figure 5 For example, in some other cases, the native framework layer and the Java framework layer can be collectively referred to as the framework layer.

[0186] It should be noted that, if it is a traditional solution, InputDispatcher will directly distribute the key combination event to the view framework, and then send it to the foreground application. However, since the touch screen electronic device (electronic device D or electronic device 400) cannot respond to the key combination event, that is, it cannot process the key combination event, the interface of the foreground application cannot be scaled. Different from this, in the solution of the embodiment of the present application, the key combination event is first converted into a touch event, and then the touch event is distributed to the view framework, thereby realizing the zoom function.

[0187] Figure 7 It is a schematic diagram of the interactive event processing process of the second type of interactive scenario in an embodiment of the present application. Figure 7 The process shown can be applied to Figure 2 The interactive scene shown, Figure 7 The processing process can be executed by the above-mentioned electronic device D or the electronic device 400.

[0188] like Figure 7 As shown in Figure 1, the application layer includes multiple applications such as app1 and app2. Device E can be regarded as Figure 2 An example of an electronic device E is shown. When the user subsequently inputs the ctrl+scroll wheel key combination operation through the device E's own keyboard or an external keyboard and mouse, the device E transmits the key combination event through the communication channel between the device E and the touch-screen electronic device (electronic device D or electronic device 400), and is received by the collaborative office software of the touch-screen electronic device. The collaborative office software sends the key combination event to the InputManagerService, and the InputManagerService injects the key combination event into the InputDispatcher to form a corresponding touch event. The InputDispatcher then distributes the touch event to the view framework, and then sends it to the foreground application through the view framework. Since the touch-screen electronic device (electronic device D or electronic device 400) can respond to the touch event, the interface of the foreground application can be scaled. The above-mentioned response process to touch events can also be referred to. Figure 6 The relevant description will not be repeated here.

[0189] It should be noted that, if it is a traditional solution, InputManagerService will not inject this key combination event into InputDispatcher to form a corresponding touch event, but will directly send the key combination event to InputDispatcher, and then InputDispatcher will distribute the key combination event to the view framework, and then send it to the foreground application. However, since the touch-screen electronic device (electronic device D or electronic device 400) cannot respond to the key combination event, that is, it cannot process the key combination event, the interface of the foreground application cannot be scaled. The embodiment of the present application first converts the key combination event into a touch event, and then distributes the touch event to the view framework, thereby realizing the zoom function.

[0190] Figure 8 1 is a schematic flow chart of a method for processing an interactive event according to an embodiment of the present application. The method can be applied to the electronic device D or the electronic device 400. Figure 8 Each step is introduced.

[0191] S801. Acquire a key combination event, where the key combination event is an interactive event for triggering a zoom function.

[0192] In one implementation, the key combination event may be acquired through a hardware driver or collaborative office software.

[0193] In the first type of interaction scenario, step S801 may be to capture the operation of the control key and the scroll wheel through the hardware driver, generate a key combination event, and report the key combination event to the InputReader.

[0194] In the second type of interaction scenario, step S801 may be to obtain the key combination event through collaborative office software. For example, the hardware driver of the screen projection device may capture the control key and wheel operation, generate a key combination event, and transmit the key combination event to the touch screen electronic device through the communication path between the screen projection device and the touch screen electronic device, and then the touch screen electronic device may obtain the key combination event through the collaborative office software.

[0195] In one implementation, the key combination event includes an event composed of a control key (ctrl key) and a mouse wheel, that is, the key combination event is a combination event of a control key event and a mouse wheel event. Optionally, the key combination event may be ctrl+wheel, and ctrl+wheel may include "ctrl+wheel up" and "ctrl+wheel down".

[0196] As mentioned above, since the ctrl key is a control key, simply pressing the ctrl key will not cause the controlled device to perform a corresponding operation. Therefore, the ctrl in the ctrl+wheel key combination event can be regarded as the state value of the wheel event. In other words, the ctrl+wheel key combination event can be regarded as a wheel event carrying the ctrl state value.

[0197] In actual scenarios, when a scroll wheel event occurs and it is determined that the current ctrl key has been pressed, the current scroll wheel event object motionevent can obtain the current event status through the interface getMetaState(). The event status is reflected as an integer (int) value, where the bit 0x1000 represents the ctrl key status. If the first bit is 1, the ctrl key is pressed, and if the first bit is 0, the ctrl key is not pressed, and it is a simple scroll wheel.

[0198] S802: Convert the key combination event into a zoom touch event.

[0199] In one implementation, the parameters of the combination key event may be converted into corresponding parameters in the zoom touch event.

[0200] Optionally, the combination key event can be converted into a zoom touch event by injecting a touch event of a segment event. That is, the zoom touch event can be generated by starting to inject touch events when the combination key event is received and ending the injection until a touch event equivalent to a zoom gesture on the touch screen can be generated.

[0201] In the first type of interaction scenario, step S802 may be that InputDispatcher distributes the key combination event to InputFilter, InputFilter filters the key combination event before event distribution, and injects the key combination event into InputDispatcher to form a corresponding touch event.

[0202] In the second type of interaction scenario, step S802 may be that the InputManagerService injects the key combination event into the InputDispatcher to form a corresponding touch event.

[0203] For a set of touch events, a certain duration is required, during which a down event is the start and an up event is the end, which can be understood as the time period from pressing to releasing. In this process, a move event can be injected into each preset time period, thereby generating a set of touch events.

[0204] A set of touch events often includes the location of the press and the distance and direction of movement (mostly sliding). Figure 3 The two-finger pinch in the above may include the coordinates of the two fingers pressed on the screen, as well as the sliding distance and sliding direction of the two fingers during the pinching process, or the change in the distance between the two fingers. The positive and negative values ​​of the distance change correspond to zooming in and out, that is, the change in the coordinates of the two fingers pressed. For example, a two-finger pinch event may be to detect that if the change in the coordinates pressed within a preset time interval is +Δ, then it corresponds to zooming in, and the zooming ratio is positively correlated with Δ. If the change is -Δ, then it corresponds to zooming out, and the zooming ratio is positively correlated with Δ.

[0205] For ease of understanding, it can be considered that the parameters of the zoom touch event (two-finger pinch event) include a pinch distance and a pinch direction, wherein the pinch direction corresponds to zooming in or out, and the pinch distance corresponds to the zoom ratio.

[0206] The mouse wheel scroll event is a motion event, which contains the coordinate information of the current mouse cursor. Therefore, the coordinate information of the cursor can be matched with the sliding distance of the two fingers.

[0207] For ease of understanding, the parameters of the scroll wheel event can be considered to include the scroll direction and scroll amount. The scroll direction includes, for example, up and down. The scroll amount can be understood as the amount of change in the scroll wheel when the user pushes the scroll wheel. For example, it can be expressed in grids, that is, how many grids the scroll wheel has scrolled up or down. The scroll direction corresponds to zooming in or out. In conventional operations, scrolling up often corresponds to zooming in, and scrolling down often corresponds to zooming out. The scroll amount corresponds to the zoom ratio.

[0208] If it is a simple wheel event, the scrolling direction corresponds to the page turning direction. In common operations, scrolling up corresponds to turning up pages, scrolling down corresponds to turning down pages, and the scrolling amount corresponds to the amount of page turning up or down. The difference between a simple wheel event and a control key and wheel combination event is that the control key status bit in the wheel event is marked differently, but it does not affect the two parameters of the wheel's scrolling direction and scrolling amount, as described above, and will not be repeated here.

[0209] Therefore, it can be seen that a mapping relationship can be established between the parameters of the combination key event and the parameters of the zoom touch event, thereby achieving the purpose of converting the combination key event into a touch event.

[0210] The parameters of the zoom touch event include the pinch direction and the pinch distance, and the parameters of the combination key event include the scroll direction and the scroll distance. There are two pinch directions, which correspond to zoom in and zoom out, and there are two scroll directions, which correspond to zoom in and zoom out, so the pinch direction and the scroll direction can be mapped one to one. The pinch distance corresponds to the zoom ratio, and the scroll distance also corresponds to the zoom ratio, so the pinch distance can be mapped to the scroll distance.

[0211] In one implementation, the combination key event is a combination event of a control key event and a mouse wheel event; the parameters of the combination key event include the scrolling direction and scrolling amount of the wheel; the parameters of the zoom touch event include the pinching direction and pinching distance; when converting the parameters of the combination key event into corresponding parameters in the zoom touch event, it can include: converting the scrolling amount into the pinching distance, and converting the scrolling direction into the pinching direction.

[0212] In one implementation, the above injection process, that is, the step of converting the combination key event into the zoom touch event, may include:

[0213] Get the scroll direction and scroll amount of the scroll wheel;

[0214] By injection, the wheel event is injected to form a zoom touch event, wherein the zoom mark of the zoom touch event (corresponding to the pinch direction) corresponds to the wheel direction, and the zoom ratio of the zoom touch event (corresponding to the pinch distance) corresponds to the scroll amount. The zoom mark is used to indicate whether this interaction event is zooming in or out, for example, positive and negative values ​​can be used to indicate zooming in and out, respectively.

[0215] In one example, the coordinates of the down event initiated by pinching two fingers correspond to two symmetrical points with the cursor position as the center point. The distances between the two points on the x-axis and y-axis are preset values, which can be 400dp, for example. The injected up event coordinates are the starting coordinates offset by 100dp in the x-axis and y-axis directions, and the x and y coordinate offsets of each move event are calculated based on the time of the event injection process. The zoom-in and zoom-out gestures are determined by the direction of the scroll wheel event, and at the same time, it is determined whether the x and y coordinates of the coordinate points are calculated as positive or negative values. The injected touch event coordinates need to be restricted to the screen boundaries.

[0216] dp (density-independent pixel) is a unit of length used in Android development. 1dp means 1 pixel length when the screen pixel density is 160 resolution.

[0217] In one implementation, the injected event coordinates are limited according to the screen boundaries.

[0218] In one implementation, step S802 may further include:

[0219] Determine whether the conversion of the previous round of key combination events is still in progress;

[0220] When the conversion of the previous round of key combination events is still in progress, the key combination event is discarded; or

[0221] When the conversion of the previous round of key combination events is not being executed, the key combination events are converted into zoom touch events.

[0222] This implementation can improve the accuracy of processing continuously triggered combination key events.

[0223] S803: In response to the zoom touch event, zoom the interface.

[0224] Optionally, the view can respond to the zoom touch event to zoom the interface.

[0225] The interface may be an interface of a touch-screen electronic device or a projection interface of a touch-screen electronic device, corresponding to the above two types of interaction scenarios respectively.

[0226] In one implementation, the method may further include: determining whether the key combination event is a zoom key combination event;

[0227] When the key combination event is a zoom key combination event, executing a step of converting the key combination event into a zoom touch event; or

[0228] When the key combination event is not a zoom key combination event, the processing flow of the interaction event ends.

[0229] That is to say, it is determined whether the input key combination event is a key combination event corresponding to the zoom function. If it is a key combination event corresponding to the zoom function, step S802 is executed. That is, a discrimination operation is added to step S802. If the input interaction event is not the expected key combination event, the subsequent operation is no longer executed, or the conversion and response steps are continued only when the input interaction event is the expected key combination event. This implementation method can shield the input of pure control key events, or avoid mistakenly responding to pure scroll wheel events as zoom key combinations.

[0230] In one implementation, the method may further include: determining whether the foreground application supports zooming, and when the foreground application supports zooming, executing step S803. That is to say, a discrimination operation is added. Since some applications support zooming and some applications do not support zooming, the solution of the embodiment of the present application may be executed only for applications that support zooming. However, it should be understood that the judgment action may be executed at any step node before step S803, for example, it may be executed before or after step S801, or it may be executed before or after step S802, and there is no limitation. For example, assuming that the judgment of whether the foreground application supports zooming is executed before step S801, then if the judgment result is no, steps S801-S803 may no longer be executed, thereby omitting the processing flow. If the judgment result is yes, steps S801-S803 are executed to achieve zooming under keyboard and mouse control. Assuming that after step S801 and before step S802, it is determined whether the foreground application supports zooming, then if the determination result is no, steps S802 and S803 are not performed, and if the determination result is yes, steps S802 and S803 are continued to be performed to achieve zooming under keyboard and mouse control.

[0231] In one example, in the above determination of whether the foreground application supports zooming, if the foreground application supports zooming, the process of executing step S803 may include:

[0232] Determine whether the foreground application supports scaling;

[0233] When the foreground application supports zooming, step S803 includes: in response to the zoom touch event, zooming the running interface of the foreground application; or

[0234] When the foreground application does not support zooming, the interaction event processing flow ends.

[0235] In one example, an application whitelist may be set, and the applications in the whitelist are all capable of supporting the zoom function.

[0236] Optionally, the InputManagerService may perform the above step of determining whether the foreground application supports scaling.

[0237] Optionally, when determining whether the foreground application supports zooming, the following steps may be included:

[0238] Compare the information of the foreground application with the pre-configured information of the whitelist application to determine whether the foreground application is an application in the whitelist application, where the whitelist application includes at least one application that supports a zoom function;

[0239] When the foreground application is an application in the whitelist, the foreground application is considered to support scaling; or

[0240] When the foreground application is not an application in the whitelist applications, it is considered that the foreground application does not support scaling.

[0241] In another implementation, the method further includes: discarding the key combination event after the zoom touch event is generated. That is, the key combination event can be discarded after step S802 is executed, and the discarded event does not need to be processed later.

[0242] Figure 8 The method shown mainly converts the unresponsive key combination event into a responsive zoom touch event and then responds to it, thereby achieving the effect of controlling the zoom function of the touch screen electronic device through the keyboard and mouse. The solution is easy to implement and does not require too many changes.

[0243] In the embodiment of the present application, it is mainly considered that the touch screen electronic device itself already has the touch operation corresponding to the zoom function, so as long as the zoom combination key event is converted into a zoom touch event, the electronic device can respond to the zoom touch event. Or it can be understood as establishing a mapping relationship between the key event of the zoom function and the touch event, so that when the zoom combination key event is input into the electronic device, the electronic device only needs to add a conversion step, first converting the zoom combination key event into a zoom touch event, and then continue to process according to the original processing flow for the touch event, so that from the user's perspective, the zoom function of the electronic device is controlled by the keyboard and mouse. This solution adds an additional conversion step to the existing touch event processing flow, so the modification difficulty is relatively small and easy to implement. There is no need to change the hardware structure, nor is there a need to redesign the complex processing flow.

[0244] Fig. 9 It is a schematic flowchart of another method for processing an interactive event according to an embodiment of the present application. Fig. 9 Can be seen as Figure 8 An example of the method shown.

[0245] S901. Obtain a combination key event.

[0246] Step S901 may be viewed as an example of step S801.

[0247] S902, determining whether the key combination event is a ctrl+scroll event, if the determination result is yes, executing step S903, if the determination result is no, executing step S904.

[0248] S903, determine whether the foreground application supports zooming, when the determination result is yes, execute step S905, when the determination result is no, execute step S904.

[0249] S904: Report the key combination event to the foreground application.

[0250] Step S904 can report the key combination event to the view for the InputDispatcher, and then the view sends it to the foreground application. Assuming that the actual event obtained in step S901 is only the control key event, that is, the ctrl event, the ctrl event is directly reported to the foreground application, and the subsequent steps of implementing the zoom function will not be executed. Assuming that the actual event obtained in step S901 is only the scroll wheel event, the scroll wheel event is directly reported to the foreground application, and the subsequent steps of implementing the zoom function will not be executed. The foreground application may respond to the scroll wheel event and turn the page.

[0251] Steps S902 and S904 are mainly to prejudge the key combination event. Only when the key combination event is indeed a key combination event of the zoom function will the subsequent steps be executed. Otherwise, the key combination event is directly reported to the foreground application.

[0252] Steps S903 and S904 are mainly to prejudge the foreground application. Only when the foreground application supports the zoom function will the subsequent steps be executed. Otherwise, the key combination event is directly reported to the foreground application.

[0253] It should be understood that there is no limitation on the order of step S902 and step S903. Step S903 can also be executed before, after or during step S901, without limitation.

[0254] S905. Determine whether the previous round of injection is still being executed. If the determination result is yes, execute step S906; if the determination result is no, execute step S907.

[0255] S906: Discard the key combination event.

[0256] S907: Inject a period of time to generate a zoom touch event.

[0257] Step S907 can be regarded as an example of step S802.

[0258] Steps S905-S907 can mainly realize that for continuous key combination operations, through each round of injection, the generated zoom touch event is equivalent to the actual two-finger pinch event.

[0259] S908: Report the zoom touch event to the foreground application.

[0260] Optionally, when step S907 is executed, in addition to step S908, a step of discarding the key combination event may also be executed.

[0261] Fig.10 It is a schematic flowchart of another method for processing an interaction event according to an embodiment of the present application. Fig.10 Can be seen as Figure 8 or Fig. 9 An example of the method shown.

[0262] S1001. The hardware driver reports a key combination event to InputReader.

[0263] Step S1001 may be considered as an example of step S801 or S901.

[0264] S1002. InputReader reports the key combination event to InputDispatcher.

[0265] S1003. InputDispatcher reports the key combination event to InputFilter in InputManagerService, and InputFilter performs filtering before event distribution.

[0266] S1004. InputFilter determines whether the key combination event is a zoom key combination event.

[0267] Step S1004 can be regarded as an example of S902.

[0268] S1005. InputFilter determines whether the foreground application supports scaling.

[0269] Step S1005 can be regarded as an example of S903.

[0270] S1006. InputFilter injects zoom touch events into InputDispatcher.

[0271] Step S1006 may be viewed as an example of step S802 or S907.

[0272] S1007. InputDispatcher sends a zoom touch event to the foreground application.

[0273] Step S1007 may be viewed as an example of step S908.

[0274] In step S1007, the InputDispatcher may report the zoom touch event to the view, which then sends the event to the foreground application.

[0275] It can be seen that Fig.10 The method shown is mainly introduced by taking the first type of interaction scenario as an example, and Fig.10The method shown is a modification based on the existing interactive event processing framework and processing flow, adding steps S1003-S1006, so that step S1007 sends a zoom touch event to the foreground instead of a combination key event, so that the view can respond to the zoom touch event, thereby realizing the zoom function triggered by the combination key event as a whole. It should be understood that those skilled in the art can also use other suitable processing modules or design a separate processing module to realize the conversion from the combination key event to the zoom touch event, but the required modification is compared with Fig.10 There will be more. Fig.10 The changes are based on the existing processing framework and processing flow, with smaller changes, lower development costs, and greater convenience.

[0276] The above mainly introduces the method of the embodiment of the present application in conjunction with the accompanying drawings. It should be understood that although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence, these steps are not necessarily performed in sequence in the order shown in the figure. Unless clearly stated in this article, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the various embodiments described above may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a part of the steps or stages in other steps. The device of the embodiment of the present application is introduced below in conjunction with the accompanying drawings.

[0277] Fig.11 Schematic diagram of a device for processing interactive events according to an embodiment of the present application. Fig.11 As shown, the device 2000 includes an acquisition unit 2001 and a processing unit 2002. The device 2000 may be any of the above-mentioned touch screen electronic devices, such as the electronic device D or the electronic device 400.

[0278] The device 2000 can be used to execute any of the above-mentioned methods for processing interactive events. For example, the acquisition unit 2001 can be used to execute step S801, and the processing unit 2002 can be used to execute steps S802 and S803. For another example, the processing unit 2002 can also be used to execute the step of determining whether the combination key event is a combination key event corresponding to the zoom function, and / or to execute the step of determining whether the foreground application supports the zoom function. For another example, the acquisition unit 2001 can also be used to execute step S901, and the processing unit 2002 can also be used to execute steps S902 to S908. For another example, the acquisition unit 2001 can also be used to execute step S1001, and the processing unit 2002 can also be used to execute steps S1002 to S1007.

[0279] In one implementation, the device 2000 may further include a storage unit for storing data such as interactive events, configuration information of applications supporting the zoom function, etc. The storage unit may be integrated in the processing unit 2002 or may be a unit independent of the acquisition unit 2001 and the processing unit 2002.

[0280] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0281] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0282] An embodiment of the present application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above methods when executing the computer program.

[0283] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0284] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0285] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0286] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0287] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0288] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0289] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0290] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0291] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0292] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0293] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0294] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0295] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for processing an interaction event, characterized in that: include: The first electronic device displays a first interface of a first application; The first electronic device projects a screen onto a second electronic device, wherein the second electronic device displays a screen projection window, and the second electronic device is connected to a keyboard and a mouse; In response to the keyboard and mouse operation on the projection window on the second electronic device, the first electronic device uses InputReader to obtain the interaction event sent by the second electronic device to the first electronic device after obtaining the keyboard and mouse operation; the content displayed in the projection window on the second electronic device is the first interface; The first electronic device filters the interaction event using InputFilter to obtain a combination key event, where the combination key event is an interaction event for triggering a zoom function, and the combination key event is a combination event of a control key event and a mouse wheel event; The first electronic device uses InputFilter to inject the key combination event into InputDispatcher to form a corresponding touch event, thereby converting the key combination event into a zoom touch event; The first electronic device uses InputDispatcher to distribute the zoom touch event to the current view of the first application, and the current view sends it to the first application, and the first application zooms the display element corresponding to the interactive event in the first interface.

2. The method according to claim 1, characterized in that The first electronic device uses InputFilter to inject the key combination event into InputDispatcher to form a corresponding touch event, thereby converting the key combination event into a zoom touch event, including: The first electronic device converts the parameters of the combination key event into corresponding parameters in the zoom touch event.

3. The method according to claim 2, characterized in that The parameters of the combination key event include the scrolling direction and scrolling amount of the scroll wheel; the parameters of the zoom touch event include the pinching direction and pinching distance; The first electronic device converts the parameters of the combination key event into corresponding parameters in the zoom touch event, including: The scrolling amount is converted into the pinching distance, and the scrolling direction is converted into the pinching direction.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The first electronic device determines whether the key combination event is a zoom key combination event; When the key combination event is the zoom key combination event, the first electronic device executes the step of converting the key combination event into the zoom touch event; or, When the key combination event is not the zoom key combination event, the processing flow of the interaction event is terminated.

5. The method according to any one of claims 1 to 3, characterized in that The first electronic device uses InputFilter to inject the key combination event into InputDispatcher to form a corresponding touch event, thereby converting the key combination event into a zoom touch event, including: The first electronic device determines whether the conversion of the combination key event in the previous round is still being executed; When the conversion of the combination key event of the previous round is still being executed, the first electronic device discards the combination key event; or, When the conversion of the combination key event of the previous round is not being executed, the first electronic device converts the combination key event into the zoom touch event.

6. The method according to any one of claims 1 to 3, characterized in that The interaction event is sent from the second electronic device to the first electronic device via collaborative office software.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Determining, by the first electronic device, whether the first application supports zooming; When the first application supports zooming, when the first application zooms the display element in the first interface, in response to the zoom touch event, the display element corresponding to the interaction event in the first interface is zoomed; or When the first application does not support scaling, when the first application scales the display elements in the first interface, the processing flow of the interaction event ends.

8. The method according to claim 7, characterized in that The first electronic device determines whether the first application supports zooming, including: The first electronic device compares the information of the first application with pre-configured information of whitelist applications to determine whether the first application is an application in the whitelist applications, wherein the whitelist applications include at least one application that supports a zoom function; When the first application is an application in the whitelist application, it is considered that the first application supports scaling; or, When the first application is not an application in the whitelist applications, it is considered that the first application does not support scaling.

9. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The first electronic device discards the combination key event after the zoom touch event is generated.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images