Interaction event processing method and device

Abstract

The invention provides an interaction event processing method and device, and the method comprises the steps: obtaining a combination key event which is an interaction event used for triggering a zooming function; converting the combination key event into a zoom touch event; and zooming the interface in response to the zooming touch event. According to the scheme, the combined key event which cannot be responded is converted into the zooming touch event which can be responded, and then the zooming touch event is responded, so that the effect of controlling the zooming function of the touch screen electronic equipment through a keyboard and a mouse is achieved, and the scheme is easy to implement.

Description

[0001] This application is a divisional application of a Chinese patent application with the application number 202211069768.5, the application date of September 2, 2022, and the invention title of "Method and Device for Processing Interaction Events". Technical Field

[0002] This application relates to the field of electronic information technology, and in particular to a method and device for processing interaction events. Background Art

[0003] With the development of technology, people's use of touch-screen electronic devices such as mobile phones and tablet computers is no longer limited to directly manipulating these electronic devices. There are also cases of using external input devices for manipulation, and cases where these touch-screen electronic devices may be cast to other electronic devices for manipulation. For example, a keyboard and a mouse can be externally connected to a touch-screen electronic device such as a mobile phone, and the mobile phone can be used as an office computer, and the keyboard and mouse can be used to control the touch-screen electronic device. Or the mobile phone can be cast to a laptop or a computer for joint work, and then the built-in keyboard of the laptop can be used to control the touch-screen electronic device, or the external keyboard and mouse of the laptop or computer can be used to control the touch-screen electronic device.

[0004] In the above scenarios, taking the mobile phone as an example, since the keyboard and mouse are essentially designed for computers, the manipulation of the keyboard and mouse does not fully match the mobile phone. That is to say, although the keyboard and mouse can realize the manipulation of the computer, when the manipulation object becomes the mobile phone, some original manipulation functions on the mobile phone cannot be realized. For example, when using the keyboard and mouse to manipulate the mobile phone, there is currently no way to realize the zoom function on the mobile phone.

[0005] Therefore, how to realize the zoom on the touch-screen electronic device through the manipulation of the keyboard and mouse is a technical problem to be solved urgently. Summary of the Invention

[0006] This application provides a method and device for processing interaction events, which can realize the zoom on the touch-screen electronic device through the manipulation of the keyboard and mouse.

[0007] In a first aspect, a method for processing interaction events is provided. The method includes: obtaining a combination key event, where the combination key event is an interaction event for triggering a zoom function; converting the combination key event into a zoom touch event; and in response to the zoom touch event, zooming the interface.

[0008] In the technical solution of this application, mainly by converting the unresponsive combination key event into a responsive zoom touch event and then making a response, the effect of realizing the zoom function of the touch-screen electronic device through the keyboard and mouse is achieved. And this solution is easy to implement and does not require too many modifications.

[0009] In combination with the first aspect, in certain implementations of the first aspect, when converting a combined key event into a zoom touch event, it may include: converting the parameters of the combined key event into corresponding parameters in the zoom touch event.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the combined key event is a combined event of a control key event and a mouse wheel event; the parameters of the combined key event include the scrolling direction and scrolling amount of the wheel; the parameters of the zoom touch event include the pinch direction and pinch distance; converting the parameters of the combined key event into corresponding parameters in the zoom touch event includes: converting the scrolling amount into the pinch distance and converting the scrolling direction into the pinch direction.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the above method further includes:

[0012] Determine whether the combined key event is a zoom combined key event;

[0013] When the combined key event is a zoom combined key event, perform the step of converting the combined key event into a zoom touch event; or,

[0014] When the combined key event is not a zoom combined key event, end the processing flow of the interaction event.

[0015] This implementation can block the input of pure control key events or avoid mistakenly responding to pure wheel events as zoom combined key events.

[0016] In combination with the first aspect, in certain implementations of the first aspect, when converting a combined key event into a zoom touch event, it may further include:

[0017] Determine whether the conversion of the combined key event in the previous round is still in progress;

[0018] When the conversion of the combined key event in the previous round is still in progress, discard the combined key event; or,

[0019] When the conversion of the combined key event in the previous round is not in progress, convert the combined key event into a zoom touch event.

[0020] This implementation can improve the accuracy of processing continuously triggered combined key events.

[0021] In combination with the first aspect, in certain implementations of the first aspect, when obtaining the combined key event, it may include:

[0022] Obtain the combined key event through a hardware driver or collaborative office software.

[0023] In combination with the first aspect, in certain implementations of the first aspect, the above method further includes:

[0024] Determine whether the foreground application supports zooming;

[0025] When the foreground application supports zooming, in response to a zoom touch event, zoom the interface, including: in response to a zoom touch event, zoom the running interface of the foreground application; or,

[0026] When the foreground application does not support zooming, end the processing flow of the interaction event.

[0027] In combination with the first aspect, in certain implementations of the first aspect, when determining whether the foreground application supports zooming, it may include:

[0028] Compare the information of the foreground application with the pre-configured information of the whitelist applications to determine whether the foreground application is an application in the whitelist applications, and the whitelist applications include at least one application that supports the zoom function;

[0029] When the foreground application is an application in the whitelist applications, it is considered that the foreground application supports zooming; or,

[0030] When the foreground application is not an application in the whitelist applications, it is considered that the foreground application does not support zooming.

[0031] In combination with the first aspect, in certain implementations of the first aspect, the above method further includes:

[0032] After the zoom touch event is generated, discard the combination key event.

[0033] In a second aspect, there is provided a processing device for interaction events, and the device includes units composed of software and / or hardware for executing any one of the methods in the first aspect.

[0034] In one embodiment, the device includes:

[0035] An acquisition unit, configured to acquire a combination key event, where the combination key event is an interaction event for triggering the zoom function;

[0036] A processing unit, configured to:

[0037] Convert the combination key event into a zoom touch event;

[0038] In response to the zoom touch event, zoom the interface.

[0039] In combination with the second aspect, in certain implementations of the second aspect, the processing unit is specifically configured to: convert the parameters of the combination key event into corresponding parameters in the zoom touch event.

[0040] In combination with the second aspect, in some implementation manners of the second aspect, the combined key event is a combined event of a control key event and a mouse wheel event; the parameters of the combined key event include the scrolling direction and the scrolling amount of the scroll wheel; the parameters of the zoom touch event include the pinching direction and the pinching distance; specifically, the processing unit is configured to:

[0041] Convert the scrolling amount into a pinching distance and convert the scrolling direction into a pinching direction.

[0042] In combination with the second aspect, in some implementation manners of the second aspect, the processing unit is further configured to:

[0043] Determine whether the combined key event is a zoom combined key event;

[0044] When the combined key event is a zoom combined key event, perform the step of converting the combined key event into a zoom touch event; or,

[0045] When the combined key event is not a zoom combined key event, end the processing flow of the interaction event.

[0046] In combination with the second aspect, in some implementation manners of the second aspect, the processing unit is further configured to:

[0047] Determine whether the conversion of the combined key event in the previous round is still in progress;

[0048] When the conversion of the combined key event in the previous round is still in progress, discard the combined key event; or,

[0049] When the conversion of the combined key event in the previous round is not in progress, convert the combined key event into a zoom touch event.

[0050] In combination with the second aspect, in some implementation manners of the second aspect, the obtaining unit is specifically configured to:

[0051] Obtain the combined key event through a hardware driver or collaborative office software.

[0052] In combination with the second aspect, in some implementation manners of the second aspect, the processing unit is further configured to:

[0053] Determine whether the foreground application supports zooming;

[0054] When the foreground application supports zooming, in response to the zoom touch event, zoom the interface, including: in response to the zoom touch event, zoom the running interface of the foreground application; or,

[0055] When the foreground application does not support zooming, end the processing flow of the interaction event.

[0056] In combination with the second aspect, in some implementation manners of the second aspect, the processing unit is specifically configured to:

[0057] Compare the information of the foreground application with the pre-configured information of the whitelist applications to determine whether the foreground application is an application in the whitelist applications, where the whitelist applications include at least one application that supports the zoom function;

[0058] When the foreground application is an application in the whitelist applications, it is considered that the foreground application supports zooming; or,

[0059] When the foreground application is not an application in the whitelist applications, it is considered that the foreground application does not support zooming.

[0060] Combined with the second aspect, in some implementation manners of the second aspect, the processing unit is further configured to:

[0061] After the zoom touch event is generated, discard the combination key event.

[0062] In a third aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, any method of the first aspect can be implemented.

[0063] In a fourth aspect, a chip is provided, including a processor, and the processor is configured to read and execute a computer program stored in a memory, and when the computer program is executed by the processor, any method of the first aspect can be implemented.

[0064] Optionally, the chip further includes a memory, and the memory is electrically connected to the processor.

[0065] Optionally, the chip may further include a communication interface.

[0066] In a fifth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any method of the first aspect can be implemented.

[0067] In a sixth aspect, a computer program product is provided, and the computer program product includes a computer program, and when the computer program is executed by a processor, any method of the first aspect can be implemented. Description of the Drawings

[0068] Figure 1 It is a schematic diagram of an interaction scenario of an embodiment of the present application.

[0069] Figure 2 It is a schematic diagram of another interaction scenario of an embodiment of the present application.

[0070] Figure 3 It is a schematic diagram of the response process of a zoom function.

[0071] Figure 4It is a schematic structural diagram of an electronic device according to an embodiment of the present application.

[0072] Figure 5 It is a software structure block diagram of an electronic device according to an embodiment of the present application.

[0073] Figure 6 It is a schematic diagram of the interaction event processing process of the first type of interaction scenario according to an embodiment of the present application.

[0074] Figure 7 It is a schematic diagram of the interaction event processing process of the second type of interaction scenario according to an embodiment of the present application.

[0075] Figure 8 It is a schematic flowchart of a method for processing an interaction event according to an embodiment of the present application.

[0076] Figure 9 It is a schematic flowchart of another method for processing an interaction event according to an embodiment of the present application.

[0077] Figure 10 It is a schematic flowchart of yet another method for processing an interaction event according to an embodiment of the present application.

[0078] Figure 11 It is a schematic diagram of a device for processing an interaction event according to an embodiment of the present application. Detailed implementation manners

[0079] The solutions of the embodiments of the present application will be introduced below with reference to the accompanying drawings. The method for processing interaction events provided by the present application can be applied to the mouse and keyboard control scenarios of various touch-screen electronic devices. The method for processing interaction events provided by the present application can be used in touch-screen electronic devices that support external mouse and keyboard devices for operation.

[0080] Figure 1 It is a schematic diagram of an interaction scenario according to an embodiment of the present application. It should be understood that the embodiments of the present application mainly include two types of interaction scenarios. The first type of interaction scenario is that a touch-screen electronic device is directly externally connected to a keyboard and a mouse, and the external keyboard and mouse are used to control the touch-screen electronic device; the second type of interaction scenario is that the touch-screen electronic device is projected onto a projection device, and the external keyboard and mouse of the projection device are used to control the touch-screen electronic device. Figure 1 It is an example of the first type of interaction scenario, Figure 2 while it is an example of the second type of interaction scenario.

[0081] Such as Figure 1As shown, in this scenario, the electronic device D is a touch-screen electronic device, that is, an electronic device that can be controlled by touch operations such as clicking, double-clicking, or swiping on the screen. The touch-screen electronic device can also be referred to as a touch-screen device or a touch-screen terminal. The electronic device D can be, for example, a mobile phone or a tablet computer, or other electronic devices that can perform touch-screen control. Figure 1 Taking the electronic device D as a tablet computer as an example. It should be understood that the main input method of the electronic device D is touch-screen operation, but there may also be input methods such as mechanical keys like the power button and volume button for key operations.

[0082] That is to say, the touch-screen electronic device in the embodiments of the present application mainly refers to an electronic device whose original main input method is touch-screen operation, and key operation is not its original main input method. That is to say, the keyboard and mouse are not the original input devices of the touch-screen electronic device, but only its extended input devices. For example, for a mobile phone, the touch operation on the screen is its original main input method. It is only with the development of technology that there has emerged a usage scenario where a keyboard and a mouse can be externally connected to control the mobile phone. However, since the keyboard and mouse are essentially designed for computers, the control of the keyboard and mouse does not fully match the mobile phone. That is to say, although the keyboard and mouse can achieve the control of a computer, when the control object becomes a mobile phone, some original control functions on the mobile phone cannot be realized.

[0083] In Figure 1 In the shown scenario, the electronic device D externally connects a keyboard B and a mouse C through a wireless connection. This wireless connection method can be, for example, a Bluetooth connection. However, it should be understood that it can also be other wireless connection methods. In addition, the wireless connection method can also be replaced by a wired connection method, without limitation. There is no limitation on the shape, model of the keyboard B and the mouse C, and the key layout on the keyboard B either. For example Figure 1 The Fn key in the shown keyboard B may not exist, or its actual position may be different from the position shown in the figure. For another example, the icon of the arrow keys can be in other forms of representation. For another example, there may be a numeric keypad area, that is, a small keyboard area. For another example, there may be volume buttons, etc. Other possible situations will not be described one by one.

[0084] In Figure 1 In the shown scenario, the user A can control the electronic device D by operating the keyboard B and the mouse C.

[0085] It should be noted that in the embodiments of the present application, the interaction events include two types: key events and touch events. Key events are used to represent interaction events generated by operating the keys or buttons on the keyboard and / or mouse. Touch events are used to represent interaction events generated by touching the screen. It should be understood that the touch events here are interaction events generated by touching the screen of a touch-screen electronic device, which are different from the touchpad on a laptop computer.

[0086] If user A presses the ctrl key on keyboard B while pushing the scroll wheel of mouse C, a combined key event of ctrl + scroll wheel will be formed. It should be understood that pushing the scroll wheel includes scrolling up and scrolling down, that is, the scroll wheel can have two scrolling directions, and the two scrolling directions are respectively described as up and down. In the operation of an external keyboard and mouse of a traditional computer, ctrl + scroll wheel can implement the zoom function, where ctrl + up scroll wheel can implement the zoom-in function, and ctrl + down scroll wheel can implement the zoom-out function. If it is just a simple scroll wheel operation, the function implemented is page turning, with the up scroll wheel corresponding to turning the page up and the down scroll wheel corresponding to turning the page down. It should be understood that since the ctrl key is a control key, simply pressing the ctrl key will not cause the controlled device to perform corresponding operations. Therefore, in the combined key event of ctrl + scroll wheel, ctrl can be regarded as the status value of the scroll wheel event, that is, the combined key event of ctrl + scroll wheel can be regarded as a scroll wheel event carrying the ctrl status value.

[0087] For current touch-screen electronic devices, even if a combined key event of ctrl + scroll wheel is generated by operating the keyboard and mouse when connected to the keyboard and mouse, the touch-screen electronic device cannot respond to this combined key event, that is, it cannot implement zooming.

[0088] In response to the above problems, the solution in the embodiments of the present application mainly realizes the zooming of the touch-screen electronic device through a combined key event by adding some processing for the combined key event. In Figure 1 In the shown scenario, if user A presses the ctrl key on keyboard B while scrolling the scroll wheel of mouse C, through the solution in the embodiments of the present application, the electronic device D processes the combined key event of ctrl + scroll wheel, and the interface zooming on the electronic device D can be realized. The specific process will be introduced below and will not be elaborated for the time being.

[0089] Figure 2 is a schematic diagram of another interaction scenario in the embodiments of the present application. As described above, Figure 2 is an example of the second type of interaction scenario. As Figure 2 shown, in this scenario, the electronic device D is a touch-screen electronic device, and the relevant description of the electronic device D can be referred to Figure 1 for relevant descriptions.

[0090] In Figure 2 the scenario shown, the electronic device D is screen-cast to the electronic device E through near-field communication, that is, short-range wireless communication technology. However, it should be understood that the embodiments of the present application do not limit the communication method between the electronic device D and the electronic device E. The electronic device E can be referred to as a screen-casting device. The electronic device E can be an electronic device such as a laptop or a computer that can be manipulated through a keyboard and a mouse. For a laptop, a keyboard is usually built-in, so it can be manipulated using the built-in keyboard of the laptop and an external mouse. For a computer, an external keyboard and mouse are usually required for manipulation.

[0091] It should be understood that the electronic device E and the electronic device D are two types of electronic devices with different manipulation methods. The electronic device D is a touch-screen electronic device, that is, an electronic device that can be manipulated through touch operations such as clicking, double-clicking, or swiping on the screen. The main input method of the electronic device D is touch-screen operation. The electronic device E is an auxiliary device for the electronic device D in the usage scenario of joint work or so-called collaborative work. The main function of the electronic device E is to provide a screen for the electronic device D, synchronously display the screen image of the electronic device D, and the interface and input / output devices of the electronic device E can be called by the electronic device D. However, it should be understood that in the usage scenario of collaborative work, the processing of data, such as the processing of input interaction events, is still performed by the electronic device D.

[0092] As described above, the touch-screen electronic device in the embodiments of the present application mainly refers to an electronic device whose original main input method is touch-screen operation, and button operation is not its original main input method. That is to say, the keyboard and mouse are not the original input devices of the touch-screen electronic device, but only the extended input devices. For the screen-casting device, the original main input method is button operation, that is, the keyboard and mouse are the original input devices of the screen-casting device, not the extended input devices.

[0093] In Figure 2 the scenario shown, the electronic device E is externally connected to a keyboard B and a mouse C in a wired connection manner. However, it should be understood that it can also be a wireless connection manner, without limitation. The description of the keyboard B and the mouse C can also refer to Figure 1 the relevant content.

[0094] In Figure 2 the scenario shown, the user A can manipulate the electronic device D by operating the keyboard B and the mouse C.

[0095] If user A presses the Ctrl key on keyboard B while rolling the scroll wheel of mouse C, that is, a combined key event of Ctrl + scroll wheel is formed. This combined key event is reported to electronic device E and then transmitted to electronic device D through the communication channel between electronic device D and electronic device E. However, since electronic device D cannot respond to this combined key event, electronic device D cannot achieve zooming.

[0096] To address this issue, in the solution of the embodiment of the present application, some processing of the combined key event by electronic device D is added, so as to achieve the zoom function of the touchscreen electronic device through the combined key event. In Figure 2 In the scenario shown, if user A presses the Ctrl key on keyboard B while rolling the scroll wheel of mouse C, electronic device D uses the solution of the embodiment of the present application to perform some processing on the combined key event of Ctrl + scroll wheel, and the interface of electronic device D can be zoomed. It should be understood that here, since electronic device E is a projection device of electronic device D, the interface of electronic device D is synchronized with the projection interface of electronic device D on the screen of electronic device E. Therefore, the interface on electronic device D and the projection interface of electronic device D on electronic device E will be zoomed synchronously.

[0097] In one example, electronic device D has an Android operating system, and electronic device E has a Windows operating system.

[0098] From Figure 1 and Figure 2 it can be seen that in the embodiment of the present application, zooming on a touchscreen electronic device is achieved through the manipulation of a keyboard and a mouse, including: achieving interface zooming on the touchscreen electronic device through an external keyboard and mouse of the touchscreen electronic device; or achieving zooming of the interface of the touchscreen electronic device and the projection interface of the touchscreen electronic device on the projection device through the keyboard and mouse of the projection device of the touchscreen electronic device.

[0099] It should be noted that although there is a type of laptop computer that can combine the functions of a regular laptop computer and a tablet computer, that is, it can handle both touch screen interaction events and keyboard and mouse interaction events, and can be called a touch screen laptop computer. However, all such laptop computers use the Windows operating system, and the two types of interaction events correspond to two independent processing processes. If you directly operate such a touch screen laptop computer, its zoom function is directly responsive to the user's touch operation or directly responsive to the user's keyboard and mouse operation, that is, the user can zoom in by touching the screen or by using the keyboard and mouse. This dual-processing process method cannot be used in devices such as electronic device D. The reason is that if this dual-processing process is forcibly applied to electronic device D, then electronic device D must use the Windows system, and the entire software architecture and supporting hardware architecture of the electronic device have to be changed and redesigned. However, this change is equivalent to completely transforming a mobile phone into a laptop computer, which is costly, complex, and there is no need for such a transformation. Because there is no need to forcibly transform one existing product type into another existing product type. In short, for the above-mentioned touch screen laptop computer, it already has an input device with two input methods, namely touch operation and button operation, and has already been equipped with processing devices and processes for two types of interaction events. Therefore, such a touch screen laptop computer is not the touch screen electronic device described in the embodiments of the present application, that is, electronic device D is not such a touch screen laptop computer.

[0100] However, the touch screen laptop computer can be used as a screen mirroring device in the embodiments of the present application, that is, electronic device E can be a touch screen laptop computer. If a touch screen electronic device such as a mobile phone or a tablet computer is screen mirrored to a touch screen laptop computer, taking the mobile phone as an example, even if the keyboard and an external mouse of the touch screen laptop computer are used for zooming, since the touch screen laptop computer only serves as a data carrier and the actual arithmetic operations are still performed by the mobile phone, so at this time, even if the keyboard and mouse connected to the touch screen laptop computer are operated, zooming still cannot be performed. It is still necessary to use the solution of the embodiments of the present application to first convert the key events of the keyboard and mouse into touch events and then respond, so as to achieve the zooming of the mobile phone interface and the screen mirroring interface of the mobile phone on the touch screen laptop computer.

[0101] Figure 3 It is a schematic diagram of the response process of a zoom function. As Figure 3 shown, if a magnifying touch operation is performed on interface 310, that is, two fingers slide in the direction of increasing the distance between the fingers, a magnifying touch event can be generated. Figure 3The medium magnification touch event is represented by a two-finger pinch #1. When the electronic device processes this touch event of the two-finger pinch #1, the interface 320 can be obtained. It can be seen that the picture A in the interface 320 is larger than the picture A in the interface 310, that is, the proportion of the picture A on the screen of the electronic device increases.

[0102] If a magnification key operation is performed on the interface 310, that is, while pressing the ctrl key, the scroll wheel is pushed upward, a combined key event of ctrl + upward scroll wheel can be generated. Figure 3 The combined key event of medium magnification is represented by ctrl + upward scroll wheel. When the electronic device processes this key event of the combined key event of ctrl + upward scroll wheel, the interface 320 can also be obtained. That is to say, both the magnification combined key event and the magnification touch event can perform the magnification of the interface of the electronic device.

[0103] If a reduction touch operation is performed on the interface 330, that is, the two fingers slide in the direction of decreasing the distance between the fingers, a reduction touch event can be generated. Figure 3 The reduction touch event is represented by a two-finger pinch #2. When the electronic device processes this touch event of the two-finger pinch #2, the interface 340 can be obtained. It can be seen that the picture A in the interface 340 is smaller than the picture A in the interface 330, that is, the proportion of the picture A on the screen of the electronic device decreases.

[0104] If a reduction key operation is performed on the interface 330, that is, while pressing the ctrl key, the scroll wheel is pushed downward, a combined key event of ctrl + downward scroll wheel can be generated. Figure 3 The combined key event of reduction is represented by ctrl + downward scroll wheel. When the electronic device processes this key event of the combined key event of ctrl + downward scroll wheel, the interface 340 can also be obtained. That is to say, both the reduction combined key event and the reduction touch event can perform the reduction of the interface of the electronic device.

[0105] Figure 3It can be regarded as an example where the foreground application of the electronic device is a picture viewing application. Implementing the zoom function through a two-finger pinch touch event is a function that the electronic device originally has, while the solution of the embodiment of the present application can implement the zoom function through the key operation of ctrl + scroll wheel. In the embodiment of the present application, mainly considering that the touch screen electronic device already has the corresponding touch operation for the zoom function, as long as the zoom combination key event is converted into a zoom touch event, the electronic device can respond to the zoom touch event. Or it can be understood that a mapping relationship is established between the key event and the touch event of the zoom function. Thus, when the zoom combination key event is input into the electronic device, the electronic device only needs to add a conversion step to first convert the zoom combination key event into a zoom touch event, and then can continue to process according to the original processing flow for touch events. Therefore, from the user's perspective, it realizes the control of the zoom function of the electronic device through the keyboard and mouse. This solution adds an additional conversion step to the existing touch event processing flow, so the transformation difficulty is relatively small and it is easy to implement. It does not require changing the hardware structure nor re-designing a complex processing flow.

[0106] And ctrl + scroll wheel is the most commonly used key operation for the zoom function. Therefore, using this combination key event to trigger the zoom function is more in line with the user's usage habits. However, it should be understood that other key events can also be designed to replace this combination key event, but some adaptive modifications need to be added, and the corresponding function of other key events, which is zoom, needs to be explained to the user. For example, a combination key event that is not currently occupied can be used to set as the combination key operation for the zoom function.

[0107] It can be seen that Figure 3 mainly shows that zoom can be implemented through touch operations or combination key operations, that is, by comparing the zoom touch event and the zoom combination key event to illustrate that the two can achieve the same effect. Figure 3 It is mainly to illustrate that using the solution of the embodiment of the present application can enable the zoom combination key event to achieve the same zoom effect as the two-finger pinch touch event.

[0108] Figure 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As Figure 4As shown, the electronic device 400 may include a processor 410, an external memory interface 420, an internal memory 421, a universal serial bus (USB) interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, an antenna 2, a mobile communication module 450, a wireless communication module 460, an audio module 470, a speaker 470A, a receiver 470B, a microphone 470C, a headphone interface 470D, a sensor module 480, keys 490, a motor 491, an indicator 492, a camera 493, a display screen 494, and a subscriber identification module (SIM) card interface 495, etc. Among them, the sensor module 480 may include a pressure sensor 480A, a gyroscope sensor 480B, a barometric pressure sensor 480C, a magnetic sensor 480D, an acceleration sensor 480E, a distance sensor 480F, a proximity light sensor 480G, a fingerprint sensor 480H, a temperature sensor 480J, a touch sensor 480K, an ambient light sensor 480L, a bone conduction sensor 480M, etc.

[0109] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 400. In other embodiments of the present application, the electronic device 400 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0110] Exemplarily, Figure 4 The illustrated processor 410 may include one or more processing units. For example, the processor 410 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0111] Among them, the controller may be the nerve center and command center of the electronic device 400. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching instructions and executing instructions.

[0112] A memory can also be provided in the processor 410 for storing instructions and data. In some embodiments, the memory in the processor 410 is a cache memory. This memory can hold the instructions or data that the processor 410 has just used or recycled. If the processor 410 needs to use the instruction or data again, it can directly call it from the said memory. This avoids repeated accesses, reduces the waiting time of the processor 410, and thus improves the efficiency of the system.

[0113] In some embodiments, the processor 410 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0114] In some embodiments, the I2C interface is a two-way synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The processor 410 may include multiple groups of I2C buses. The processor 410 can be respectively coupled to the touch sensor 480K, the charger, the flash, the camera 493, etc. through different I2C bus interfaces. For example, the processor 410 can be coupled to the touch sensor 480K through the I2C interface, enabling the processor 410 to communicate with the touch sensor 480K through the I2C bus interface to implement the touch function of the electronic device 400.

[0115] In some embodiments, the I2S interface can be used for audio communication. The processor 410 may include multiple groups of I2S buses. The processor 410 can be coupled to the audio module 470 through the I2S bus to implement communication between the processor 410 and the audio module 470.

[0116] In some embodiments, the audio module 470 may transmit an audio signal to the wireless communication module 460 through the I2S interface to implement the function of answering a call through a Bluetooth headset.

[0117] In some embodiments, the PCM interface may also be used for audio communication to sample, quantize, and encode an analog signal. The audio module 470 and the wireless communication module 460 may be coupled through the PCM bus interface.

[0118] In some embodiments, the audio module 470 may also transmit an audio signal to the wireless communication module 460 through the PCM interface to implement the function of answering a call through a Bluetooth headset. It should be understood that both the I2S interface and the PCM interface can be used for audio communication.

[0119] In some embodiments, the UART interface is a universal serial data bus for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. The UART interface is usually used to connect the processor 410 and the wireless communication module 460. For example, the processor 410 communicates with the Bluetooth module in the wireless communication module 460 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 470 may transmit an audio signal to the wireless communication module 460 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0120] In some embodiments, the MIPI interface may be used to connect the processor 410 to peripheral devices such as the display screen 494 and the camera 493. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. The processor 410 and the camera 493 communicate through the CSI interface to implement the shooting function of the electronic device 400. The processor 410 and the display screen 494 communicate through the DSI interface to implement the display function of the electronic device 400.

[0121] In some embodiments, the GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or a data signal. The GPIO interface can be used to connect the processor 410 to the camera 493, the display screen 494, the wireless communication module 460, the audio module 470, the sensor module 480, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0122] Exemplarily, the USB interface 430 is an interface that conforms to the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 400, and can also be used to transfer data between the electronic device 400 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as AR devices, etc.

[0123] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only for illustrative purposes and does not constitute a structural limitation on the electronic device 400. In other embodiments of the present application, the electronic device 400 can also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0124] The charging management module 440 is used to receive a charging input from a charger. Among them, the charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 440 can receive the charging input from the wired charger through the USB interface 430. In some embodiments of wireless charging, the charging management module 440 can receive the wireless charging input through the wireless charging coil of the electronic device 400. While charging the battery 442, the charging management module 440 can also supply power to the electronic device through the power management module 441.

[0125] The power management module 441 is used to connect the battery 442, the charging management module 440 and the processor 410. The power management module 441 receives the inputs from the battery 442 and / or the charging management module 440 and supplies power to the processor 410, the internal memory 421, the external memory, the display screen 494, the camera 493, the wireless communication module 460, etc. The power management module 441 can also be used to monitor parameters such as the battery capacity, the number of battery cycles, and the battery health status (leakage, impedance). In some other embodiments, the power management module 441 can also be provided in the processor 410. In some other embodiments, the power management module 441 and the charging management module 440 can also be provided in the same device.

[0126] The wireless communication function of the electronic device 400 can be implemented through antenna 1, antenna 2, the mobile communication module 450, the wireless communication module 460, the modulation and demodulation processor, and the baseband processor, etc.

[0127] Antenna 1 and Antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 400 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, Antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0128] The mobile communication module 450 can provide solutions for wireless communication applied to the electronic device 400, such as at least one of the following solutions: second-generation (2G) mobile communication solution, third-generation (3G) mobile communication solution, fourth-generation (4G) mobile communication solution, fifth-generation (5G) mobile communication solution. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 450 can receive electromagnetic waves through Antenna 1, perform processing such as filtering and amplifying the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 450 can also amplify the signal modulated by the modulation and demodulation processor, and the amplified signal is converted into electromagnetic waves and radiated out through Antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be disposed in the processor 410. In some embodiments, at least some functional modules of the mobile communication module 450 can be disposed in the same device as at least some modules of the processor 410.

[0129] The modulation and demodulation processor can include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 470A, receiver 470B, etc.), or displays an image or video through the display screen 494. In some embodiments, the modulation and demodulation processor can be an independent device. In some other embodiments, the modulation and demodulation processor can be independent of the processor 410 and disposed in the same device as the mobile communication module 450 or other functional modules.

[0130] The wireless communication module 460 may provide solutions for wireless communications applied to the electronic device 400, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 460 may be one or more devices integrating at least one communication processing module. The wireless communication module 460 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 410. The wireless communication module 460 may also receive signals to be sent from the processor 410, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0131] In some embodiments, antenna 1 of electronic device 400 is coupled to mobile communication module 450, and antenna 2 of electronic device 400 is coupled to wireless communication module 460, enabling electronic device 400 to communicate with a network and other electronic devices via wireless communication technologies. The wireless communication technologies may include at least one of the following communication technologies: Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, IR technology. The GNSS may include at least one of the following positioning technologies: Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), Beidou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), Satellite Based Augmentation Systems (SBAS).

[0132] Electronic device 400 implements a display function through a GPU, display screen 494, and an application processor, etc. The GPU is a microprocessor for image processing, connected to display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 410 may include one or more GPUs, which execute program instructions to generate or change display information.

[0133] The display screen 494 is used to display images, videos, etc. The display screen X94 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-led, a Micro-Led, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 400 may include one or N display screens 494, where N is a positive integer greater than 1.

[0134] The electronic device 400 can implement the shooting function through an ISP, a camera 493, a video codec, a GPU, a display screen 494, an application processor, etc.

[0135] The ISP is used to process the data fed back by the camera 493. For example, when taking a photo, the shutter is opened, and light passes through the lens and is transmitted to the camera photosensitive element. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image through algorithms. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be provided in the camera 493.

[0136] The camera 493 is used to capture static images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the electronic device 400 may include one or N cameras 493, where N is a positive integer greater than 1.

[0137] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 400 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0138] The video codec is used to compress or decompress digital videos. The electronic device 400 can support one or more video codecs. In this way, the electronic device 400 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0139] The NPU is a neural-network (NN) computing processor. By learning from the biological neural network structure, such as learning from the transmission pattern between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 400 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0140] The external memory interface 420 can be used to connect an external memory card, such as a Secure Digital (SD) card, to expand the storage capacity of the electronic device 400. The external memory card communicates with the processor 410 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0141] The internal memory 421 can be used to store computer-executable program code, and the executable program code includes instructions. The processor 410 executes various functional applications and data processing of the electronic device 400 by running the instructions stored in the internal memory 421. The internal memory 421 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the electronic device 400 (such as audio data, phone book, etc.). In addition, the internal memory 421 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, Universal Flash Storage (UFS), etc.

[0142] The electronic device 400 can implement audio functions through the audio module 470, speaker 470A, receiver 470B, microphone 470C, headphone jack 470D, and the application processor, etc. For example, music playback, recording, etc.

[0143] The audio module 470 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 470 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 470 can be disposed in the processor 410, or some functional modules of the audio module 470 can be disposed in the processor 410.

[0144] The speaker 470A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 400 can listen to music or hands-free calls through the speaker 470A.

[0145] The receiver 470B, also known as the "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device 400 answers a call or a voice message, the voice can be listened to by placing the receiver 470B close to the human ear.

[0146] The microphone 470C, also known as the "microphone" or "transmitter", is used to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak by placing the mouth close to the microphone 470C to input the sound signal into the microphone 470C. The electronic device 400 can be provided with at least one microphone 470C. In some other embodiments, the electronic device 400 can be provided with two microphones 470C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 400 can also be provided with three, four or more microphones 470C, which can collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording.

[0147] The headphone jack 470D is used to connect a wired headphone. The headphone jack 470D can be a USB interface 430, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0148] The pressure sensor 480A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor 480A may be disposed on the display screen 494. There are many types of pressure sensors 480A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates having conductive materials. When a force acts on the pressure sensor 480A, the capacitance between the electrodes changes. The electronic device 400 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 494, the electronic device 400 detects the intensity of the touch operation according to the pressure sensor 480A. The electronic device 400 can also calculate the position of the touch based on the detection signal of the pressure sensor 480A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0149] The gyroscope sensor 480B can be used to determine the motion posture of the electronic device 400. In some embodiments, the angular velocity of the electronic device 400 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 480B. The gyroscope sensor 480B can be used for anti-shake during shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor 480B detects the angle of jitter of the electronic device 400, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the jitter of the electronic device 400 through reverse movement to achieve anti-shake. The gyroscope sensor 480B can also be used for navigation and somatosensory game scenarios.

[0150] The barometric pressure sensor 480C is used to measure barometric pressure. In some embodiments, the electronic device 400 calculates the altitude based on the barometric pressure value measured by the barometric pressure sensor 480C to assist in positioning and navigation.

[0151] The magnetic sensor 480D includes a Hall sensor. The electronic device 400 can use the magnetic sensor 480D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 400 is a flip phone, the electronic device 400 can detect the opening and closing of the flip according to the magnetic sensor 480D; according to the detected opening and closing state of the leather case or the opening and closing state of the flip, features such as automatic flip unlocking are set.

[0152] The acceleration sensor 480E can detect the magnitude of the acceleration of the electronic device 400 in various directions (generally three axes). When the electronic device 400 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0153] The distance sensor 480F is used to measure distance. The electronic device 400 can measure distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 400 can use the distance sensor 480F to measure distance to achieve fast focusing.

[0154] The proximity light sensor 480G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The electronic device 400 emits infrared light outward through the light-emitting diode. The electronic device 400 uses the photodiode to detect the infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 400. When insufficient reflected light is detected, the electronic device 400 can determine that there is no object near the electronic device 400. The electronic device 400 can use the proximity light sensor 480G to detect that the user holds the electronic device 400 close to the ear for a call, so as to automatically turn off the screen to achieve the purpose of power saving. The proximity light sensor 480G can also be used for automatic unlocking and locking of the leather case mode and pocket mode.

[0155] The ambient light sensor 480L is used to sense the ambient light brightness. The electronic device 400 can adaptively adjust the brightness of the display screen 494 according to the sensed ambient light brightness. The ambient light sensor 480L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 480L can also cooperate with the proximity light sensor 480G to detect whether the electronic device 400 is in the pocket to prevent accidental touch.

[0156] The fingerprint sensor 480H is used to collect fingerprints. The electronic device 400 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access to application locks, fingerprint photography, fingerprint answering of incoming calls, etc.

[0157] The temperature sensor 480J is used to detect temperature. In some embodiments, the electronic device 400 uses the temperature detected by the temperature sensor 480J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 480J exceeds the threshold, the electronic device 400 reduces the performance of the processor near the temperature sensor 480J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 400 heats the battery 442 to avoid abnormal shutdown of the electronic device 400 caused by low temperature. In some other embodiments, when the temperature is lower than yet another threshold, the electronic device 400 boosts the output voltage of the battery 442 to avoid abnormal shutdown caused by low temperature.

[0158] The touch sensor 480K, also known as the "touch panel". The touch sensor 480K can be disposed on the display screen 494, and the touch sensor 480K and the display screen 494 together form a touch screen, also known as the "touch display screen". The touch sensor 480K is used to detect touch operations acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 494. In some other embodiments, the touch sensor 480K can also be disposed on the surface of the electronic device 400, at a position different from that of the display screen 494.

[0159] The bone conduction sensor 480M can acquire vibration signals. In some embodiments, the bone conduction sensor 480M can acquire vibration signals of the vibrating bone mass of the human vocal tract. The bone conduction sensor 480M can also contact the human pulse to receive blood pressure pulsation signals. In some embodiments, the bone conduction sensor 480M can also be disposed in the earphone to form a bone conduction earphone. The audio module 470 can analyze the voice signals based on the vibration signals of the vibrating bone mass acquired by the bone conduction sensor 480M to implement the voice function. The application processor can analyze the heart rate information based on the blood pressure pulsation signals acquired by the bone conduction sensor 480M to implement the heart rate detection function.

[0160] The keys 490 include a power-on key, volume keys, etc. The keys 490 can be mechanical keys or touch keys. The electronic device 400 can receive key inputs to generate key signal inputs related to the user settings and function controls of the electronic device 400.

[0161] The motor 491 can generate vibration prompts. The motor 491 can be used for incoming call vibration prompts and can also be used for touch vibration feedback. For example, touch operations on different applications (such as taking pictures, audio playing, etc.) can correspond to different vibration feedback effects. Touch operations on different regions of the display screen 494 can also correspond to different vibration feedback effects for the motor 491. Different application scenarios (such as time reminder, receiving messages, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0162] The indicator 492 can be an indicator light, which can be used to indicate the charging state, power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0163] The SIM card interface 495 is used to connect to the SIM card. The SIM card can be inserted into or removed from the SIM card interface 495 to achieve contact with and separation from the electronic device 400. The electronic device 400 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 495 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 495 simultaneously. The types of the multiple cards can be the same or different. The SIM card interface 495 can also be compatible with different types of SIM cards. The SIM card interface 495 can also be compatible with external memory cards. The electronic device 400 interacts with the network through the SIM card to achieve functions such as calls and data communication. In some embodiments, the electronic device 400 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 400 and cannot be separated from the electronic device 400.

[0164] The software system of the electronic device 400 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of this application, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the electronic device. The following combines Figure 5 Taking the electronic device with an Android system having a layered architecture as an example, the software structure of the electronic device is described exemplarily.

[0165] Figure 5 It is a software structure block diagram of an electronic device according to an embodiment of this application. Figure 5 The electronic device in Figure 4 can be the electronic device 400 shown in Figure 5 As shown, the layered architecture divides the software into several layers, and each layer has a clear role and division of labor; communication between layers is through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely the application layer, the application framework layer, the system library layer, and the kernel layer. The application layer can include a series of application packages. However, it should be understood that Figure 5 this is only an example of layer division for the Android system, and there may be other layer division methods in actual scenarios. In addition, there may be other naming methods for the modules in each layer, and there is no limitation. For example, the application framework layer can be called the framework layer, or the system library layer can be merged into the application framework layer, etc.

[0166] Such as Figure 5As shown in the figure, an application package may include applications (APPs) such as short messages, calendars, cameras, videos, navigation, calls, and galleries. For convenience, applications will be referred to as apps hereinafter. A running application can also be called a foreground application or a front-end application.

[0167] Since some applications support zoom functions, such as short messages, galleries, or navigation, while some applications do not support zoom functions, such as settings and remote controls, different manufacturers' electronic devices may vary. In short, applications can be divided into applications that support zoom functions and applications that do not support zoom functions. In the embodiments of the present application, combined key control for applications that support zoom functions can be achieved.

[0168] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions.

[0169] As Figure 5 shown in the figure, the application framework layer may include a window manager, a resource manager, a notification manager, an input manager, etc.

[0170] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0171] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, and so on.

[0172] The notification manager enables applications to display notification information in the status bar. It can be used to convey message types that need to be informed, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform that the download is completed, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or a scroll bar text, such as the notification of a background running application, or a notification that appears on the screen in the form of a dialogue window. For example, it prompts text information in the status bar, emits a prompt tone, the electronic device vibrates, and the indicator light flashes, etc.

[0173] The input manager (InputManagerService) is mainly used to manage the input part of the entire system, including keyboards, mice, touchscreens, etc.

[0174] The system library layer may include the Android runtime and system libraries. The Android runtime includes core libraries and virtual machines. The Android runtime is responsible for the scheduling and management of the Android system.

[0175] The core library consists of two parts: one is the functional functions that need to be called by the Java language, and the other is the core library of Android.

[0176] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files in the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0177] The system library can include multiple functional modules. For example, the surface manager, media libraries, 3D graphics processing libraries (such as OpenGL ES), input service modules, etc.

[0178] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0179] The media library supports the playback and recording of multiple audio formats, the playback and recording of multiple video formats, and static image files. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, moving picture experts group audio layer III (MP3), advanced audio coding (AAC), adaptive multi-rate (AMR), joint photo graphic experts group (JPG), and portable network graphics (PNG).

[0180] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0181] The input service module can include InputReader and InputDispatcher. InputReader is used to read input events and report the read events to InputDispatcher, and InputDispatcher distributes the input events.

[0182] The kernel layer refers to the layer between hardware and software; the kernel layer at least includes sensor drivers, camera drivers, display drivers, etc.

[0183] It should be noted that the above-listed Figure 4It is a structural diagram of a possible electronic device. Figure 5 It is a software architecture diagram of a possible electronic device. As Figures 1 to 3 any one of the touchscreen electronic devices (electronic device D) shown in the figure can have Figure 4 the described structure and Figure 5 the software structure shown in the figure.

[0184] Figure 6 It is a schematic diagram of the interaction event processing process of the first type of interaction scenario in the embodiment of the present application. Figure 6 The shown processing process can be applied to Figure 1 the interaction scenario shown in the figure, Figure 6 and the described processing process can be executed by the above-mentioned electronic device D or electronic device 400.

[0185] As Figure 6 shown, the application layer includes multiple applications such as app1, app2, app3, app4, etc. The application running in the foreground can be called the foreground application. When the user inputs the combined key operation of ctrl + scroll wheel through the keyboard and mouse external to the touchscreen electronic device, the corresponding driver in the kernel layer captures the combined key event and reports it to InputReader in the native framework layer. InputReader reads the combined key event and reports it to InputDispatcher in the native framework layer. InputDispatcher can then distribute the combined key event. In the solution of the embodiment of the present application, InputDispatcher will first distribute the combined key event to InputFilter in InputManagerService of the java framework layer. InputFilter filters the combined key event before event distribution and injects the combined key event into InputDispatcher to form a corresponding touch event. Then InputDispatcher distributes this touch event to the view framework and then sends it to the foreground application through the view framework. Since the touchscreen electronic device (electronic device D or electronic device 400) can respond to this touch event. Specifically, in this process, InputDispatcher distributes the event to the application process, and the view framework running in the application process further distributes and processes the event according to the view tree of the current interface layout, and passes the event to the corresponding application control. Thus, the interface of the foreground application can be scaled.

[0186] In some cases, the native framework layer can be regarded as Figure 5 an example of the system library layer shown in the figure, and the java framework layer can be regarded asFigure 5 An example of the application framework layer shown. It should be understood that since the layering method of the Android system is not unique, that is, it may not completely adopt Figure 5 the layer structure division method shown. For example, in some other cases, the native framework layer and the java framework layer can also be collectively referred to as the framework layer.

[0187] It should be noted that in the case of a traditional solution, the InputDispatcher will directly distribute the combined key event to the view framework and then send it to the foreground application. However, since the touchscreen electronic device (electronic device D or electronic device 400) cannot respond to the combined key event, that is, it cannot process this combined key event, the interface of the foreground application cannot be zoomed. In contrast, in the solution of the embodiment of the present application, the combined key event is first converted into a touch event, and then the touch event is distributed to the view framework, thus realizing the zoom function.

[0188] Figure 7 It is a schematic diagram of the interaction event processing process of the second type of interaction scenario in the embodiment of the present application. Figure 7 The processing process shown can be applied to Figure 2 the interaction scenario shown, Figure 7 and the processing process can be executed by the above-mentioned electronic device D or electronic device 400.

[0189] Such as Figure 7 shown, the application layer includes multiple applications such as app1 and app2. Device E can be regarded as Figure 2 an example of the electronic device E shown. Subsequently, when the user inputs the combined key operation of ctrl + scroll wheel through the built-in keyboard or the external keyboard and mouse of device E, device E will send the combined key event through the communication channel between device E and the touchscreen electronic device (electronic device D or electronic device 400), and receive it through the collaborative office software of the touchscreen electronic device. The collaborative office software will send the combined key event to the InputManagerService, and the InputManagerService will inject this combined key event into the InputDispatcher to form a corresponding touch event. Then the InputDispatcher will distribute this touch event to the view framework, and then send it to the foreground application through the view framework. Since the touchscreen electronic device (electronic device D or electronic device 400) can respond to this touch event, the interface of the foreground application can be zoomed. The response process of the above touch event can also refer to Figure 6 the relevant description and will not be elaborated here.

[0190] It should be noted that in the traditional solution, the InputManagerService does not inject this combined key event into the InputDispatcher to form a corresponding touch event. Instead, it directly sends the combined key event to the InputDispatcher, and then the InputDispatcher distributes the combined key event to the view framework and then to the foreground application. However, since the touchscreen electronic device (electronic device D or electronic device 400) cannot respond to this combined key event, that is, it cannot process this combined key event, the interface of the foreground application cannot be zoomed. In the embodiments of the present application, the combined key event is first converted into a touch event, and then the touch event is distributed to the view framework, thus realizing the zoom function.

[0191] Figure 8 FIG. is a schematic flowchart of a method for processing an interaction event according to an embodiment of the present application. This method can be applied to the above-mentioned electronic device D or electronic device 400. The following Figure 8 introduces each step.

[0192] S801. Obtain a combined key event, which is an interaction event used to trigger the zoom function.

[0193] In one implementation, the combined key event can be obtained through a hardware driver or collaborative office software.

[0194] In the first type of interaction scenario, step S801 can be to capture the operations of the control key and the scroll wheel through a hardware driver, generate a combined key event, and report the combined key event to the InputReader.

[0195] In the second type of interaction scenario, step S801 can be to obtain the combined key event through collaborative office software. For example, the hardware driver of the projection device can capture the operations of the control key and the scroll wheel, generate a combined key event, and transmit the combined key event to the touchscreen electronic device through the communication path between the projection device and the touchscreen electronic device. Then, the touchscreen electronic device can obtain the combined key event through the collaborative office software.

[0196] In one implementation, the combined key event includes an event composed of a control key (ctrl key) and a scroll wheel, that is, the combined key event is a combined event of a control key event and a mouse scroll wheel event. Optionally, the combined key event can be ctrl + scroll wheel, and ctrl + scroll wheel can include "ctrl + up scroll wheel" and "ctrl + down scroll wheel".

[0197] As described above, since the Ctrl key is a control key, simply pressing the Ctrl key will not cause the controlled device to perform corresponding operations. Therefore, in the Ctrl+scroll wheel key combination event, Ctrl can be regarded as the status value of the scroll wheel event. That is to say, the Ctrl+scroll wheel key combination event can be regarded as a scroll wheel event carrying the Ctrl status value.

[0198] In an actual scenario, when a scroll wheel event occurs and it is determined that the Ctrl key is currently pressed, the current event status can be obtained through the interface getMetaState() in the current scroll wheel event object MotionEvent. The event status is represented as an integer (int) value. Among them, the bit 0x1000 represents the Ctrl key status. If the first bit is 1, it means the Ctrl key is pressed; if the first bit is 0, it means the Ctrl key is not pressed, and it is just a simple scroll wheel.

[0199] S802. Convert the key combination event into a zoom touch event.

[0200] In one implementation, the parameters of the key combination event can be converted into the corresponding parameters in the zoom touch event.

[0201] Optionally, a touch event of a segment of events can be injected to convert the key combination event into a zoom touch event. That is to say, the injection of the touch event can start when the key combination event is received and end until a touch event equivalent to the zoom gesture on the touch screen can be generated, so as to generate a zoom touch event.

[0202] In the first type of interaction scenario, step S802 can be that the InputDispatcher distributes the key combination event to the InputFilter. The InputFilter filters the key combination event before event distribution and injects this key combination event into the InputDispatcher to form a corresponding touch event.

[0203] In the second type of interaction scenario, step S802 can be that the InputManagerService injects this key combination event into the InputDispatcher to form a corresponding touch event.

[0204] For a group of touch events, a certain duration is required. During this duration, starting from the down event and ending with the up event, it can be understood as the time period from pressing to releasing. During this process, a move event can be injected every preset time interval, so as to generate a group of touch events.

[0205] A group of touch events often includes the position where it is pressed and the distance and direction of the move (mostly sliding). For example, forFigure 3 For the two-finger pinch in it, it can include the coordinates where the two fingers press on the screen, as well as the sliding distance and sliding direction of the two fingers during the pinching process. It can also be said to be the change in the distance between the two fingers. The positive and negative of this distance change respectively correspond to zooming in and out, that is, the change amount of the coordinates where the two fingers press. For example, a two-finger pinch event can be that the detected change amount of the coordinates pressed within a preset time interval is +Δ, which corresponds to zooming in, and the zooming-in ratio is positively correlated with Δ. If the change amount is -Δ, it corresponds to zooming out, and the zooming-out ratio is positively correlated with Δ.

[0206] For ease of understanding, it can be considered that the parameters of the zoom touch event (two-finger pinch event) include the pinch distance and the pinch direction, where the pinch direction corresponds to zooming in or out, and the pinch distance corresponds to the zoom ratio.

[0207] The mouse wheel scroll event belongs to the motion event, and the event contains the coordinate information of the current mouse cursor. Therefore, the coordinate information of the cursor can be corresponded to the sliding distance of the above two fingers.

[0208] For ease of understanding, it can be considered that the parameters of the scroll wheel event include the scroll direction and the scroll amount. The scroll direction includes, for example, up and down. The scroll amount can be understood as the change amount when the user pushes the scroll wheel. For example, it can be expressed in units of grids, that is, how many grids the scroll wheel scrolls up or down. The scroll direction corresponds to zooming in or out. In the usual operation, often, scrolling up corresponds to zooming in, and scrolling down corresponds to zooming out. The scroll amount corresponds to the zoom ratio.

[0209] If it is a pure scroll wheel event, then the scroll direction corresponds to the page turning direction. In the usual operation, often, scrolling up corresponds to turning the page up, and scrolling down corresponds to turning the page down. The scroll amount corresponds to the amount of the page turning up or down. The difference between the pure scroll wheel event and the combined key event of the control key and the scroll wheel lies in the different identifications of the control key status bits in the scroll wheel event, but it will not affect the two parameters of the scroll direction and the scroll amount of the scroll wheel. As described above, it will not be elaborated further.

[0210] Therefore, it can be seen that the parameters of the combined key event can be mapped to the parameters of the zoom touch event, so as to achieve the purpose of converting the combined key event into a touch event.

[0211] The parameters of the zoom touch event include the pinch direction and the pinch distance, and the parameters of the combined key event include the scroll direction and the scroll distance. There are two pinch directions, which can respectively correspond to zooming in and out, and there are also two scroll directions, which can respectively correspond to zooming in and out. Therefore, the pinch direction and the scroll direction can be in one-to-one correspondence. The pinch distance corresponds to the zoom ratio, and the scroll distance also corresponds to the zoom ratio. Therefore, the pinch distance can be corresponded to the scroll distance.

[0212] In one implementation, the combined key event is a combined event of a control key event and a mouse wheel event; the parameters of the combined key event include the rolling direction and the rolling amount of the wheel; the parameters of the pinch touch event include the pinch direction and the pinch distance; when converting the parameters of the combined key event into the corresponding parameters in the pinch touch event, it may include: converting the rolling amount into the pinch distance and converting the rolling direction into the pinch direction.

[0213] In one implementation, the above injection process, that is, the step of converting the combined key event into a pinch touch event, may include:

[0214] Obtain the rolling direction and the rolling amount of the wheel;

[0215] Inject the wheel event in an injection manner to form a pinch touch event, where the scaling identifier (corresponding to the pinch direction) of the pinch touch event corresponds to the wheel direction, and the scaling ratio (corresponding to the pinch distance) of the pinch touch event corresponds to the rolling amount. The scaling identifier is used to indicate whether this interaction event is a magnification or a reduction. For example, positive and negative values can be used to represent magnification and reduction respectively.

[0216] In an example, the coordinates of the starting down event of pinching two fingers are respectively corresponding to two symmetric points with the cursor position as the center point, and the distances of the two points on the x-axis and y-axis are respectively a preset value. The preset value can be, for example, 400 dp. The coordinates of the injected up event are the starting coordinates offset by 100 dp in both the x and y axis directions, and the x and y coordinates of each move event are offset to calculate specific coordinate values according to the time of the event injection process. The magnification and reduction gestures are determined according to the direction of the wheel event, and at the same time, it is determined whether to use positive or negative values for the coordinate calculation of the coordinate points x and y. The coordinates of the injected touch event need to be restricted according to the screen boundary.

[0217] dp (density-independent pixel) is a length unit used in Android development. 1 dp represents 1 pixel length when the screen pixel density is 160 dpi.

[0218] In one implementation, restrict the coordinates of the injected event according to the screen boundary.

[0219] In one implementation, step S802 may further include:

[0220] Determine whether the conversion of the combined key event in the previous round is still in progress;

[0221] When the conversion of the combined key event in the previous round is still in progress, discard the combined key event; or,

[0222] When the conversion of the combined key event in the current round is not in execution, convert the combined key event into a zoom touch event.

[0223] This implementation method can improve the accuracy of processing continuously triggered combined key events.

[0224] S803. In response to the zoom touch event, zoom the interface.

[0225] Optionally, the view can be used to respond to the zoom touch event, thereby zooming the interface.

[0226] The interface can be the interface of the touch-screen electronic device or the screen mirroring interface of the touch-screen electronic device, corresponding to the above two types of interaction scenarios respectively.

[0227] In one implementation method, the above method may further include: determining whether the combined key event is a zoom combined key event;

[0228] When the combined key event is a zoom combined key event, execute the step of converting the combined key event into a zoom touch event; or,

[0229] When the combined key event is not a zoom combined key event, end the processing flow of the interaction event.

[0230] That is to say, determine whether the input combined key event is the combined key event corresponding to the zoom function. When it is the combined key event corresponding to the zoom function, execute step S802. That is, add a discrimination operation to step S802. When the input interaction event is not the expected combined key event, the subsequent operations will no longer be executed. Or rather, only when the input interaction event is the correct combined key event, will the steps of conversion and response be continued. This implementation method can block the input of pure control key events or avoid misresponding to pure scroll wheel events as zoom combined key events.

[0231] In one implementation, the above method may further include: determining whether the foreground application supports zooming. When the foreground application supports zooming, step S803 is executed. That is to say, an additional discrimination operation is added. Since some applications support zooming and some do not, the solution of the embodiment of the present application can be executed only for applications that support zooming. However, it should be understood that this determination action can be executed at any step node before step S803. For example, it may be executed before or after step S801, or may be executed before or after step S802, without limitation. For example, assuming that it is determined whether the foreground application supports zooming before step S801, then when the determination result is no, steps S801 - S803 may not be executed, saving this processing flow. When the determination result is yes, steps S801 - S803 are executed to implement zooming under mouse and keyboard control. Assuming that it is determined whether the foreground application supports zooming after step S801 and before step S802, then when the determination result is no, steps S802 and S803 may not be executed. When the determination result is yes, steps S802 and S803 are continued to be executed to implement zooming under mouse and keyboard control. This will not be listed one by one.

[0232] In one example, the process of determining whether the foreground application supports zooming and executing step S803 when the foreground application supports zooming may include:

[0233] Determining whether the foreground application supports zooming;

[0234] When the foreground application supports zooming, in response to a zoom touch event, zooming the running interface of the foreground application; or,

[0235] When the foreground application does not support zooming, ending the processing flow of the interaction event.

[0236] In one example, an application whitelist may be set, and the applications in the whitelist are all capable of supporting the zoom function.

[0237] Optionally, the above step of determining whether the foreground application supports zooming may be executed by the InputManagerService.

[0238] Optionally, when determining whether the foreground application supports zooming, it may include:

[0239] Comparing the information of the foreground application with the pre - configured information of the whitelist applications to determine whether the foreground application is an application in the whitelist applications, and the whitelist applications include at least one application that supports the zoom function;

[0240] When the foreground application is an application in the whitelist applications, it is considered that the foreground application supports zooming; or,

[0241] When the foreground application is not an application in the whitelist, it is considered that the foreground application does not support zooming.

[0242] In another implementation, the above method further includes: after generating a zoom touch event, discarding the combination key event. That is, the combination key event can be discarded after step S802 is executed, and the discarded event does not need to be processed subsequently.

[0243] Figure 8 The method shown mainly realizes the effect of controlling the zoom function of the touch screen electronic device through the keyboard and mouse by converting the unresponsive combination key event into a responsive zoom touch event and then responding. And this solution is easy to implement and does not require too many modifications.

[0244] In the embodiments of the present application, it is mainly considered that the touch screen electronic device itself already has corresponding touch operations for the zoom function. Therefore, as long as the combination key event for zooming is converted into a zoom touch event, the electronic device can respond to the zoom touch event. Or it can be understood that a mapping relationship is established between the key event and the touch event of the zoom function. Thus, when the combination key event for zooming is input into the electronic device, the electronic device only needs to add a conversion step to first convert the zoom combination key event into a zoom touch event, and then can continue to process according to the original processing flow for touch events. Therefore, from the user's perspective, it realizes the control of the zoom function of the electronic device through the keyboard and mouse. This solution adds an additional conversion step to the existing touch event processing flow, so the transformation difficulty is relatively small and it is easy to implement. It does not require changing the hardware structure nor re-designing a complex processing flow.

[0245] Figure 9 It is a schematic flowchart of another method for processing interaction events in the embodiments of the present application. Figure 9 can be regarded as Figure 8 an example of the method shown.

[0246] S901. Obtain a combination key event.

[0247] Step S901 can be regarded as an example of step S801.

[0248] S902. Determine whether the combination key event is a ctrl+scroll event. When the determination result is yes, execute step S903; when the determination result is no, execute step S904.

[0249] S903. Determine whether the foreground application supports zooming. When the determination result is yes, execute step S905; when the determination result is no, execute step S904.

[0250] S904. Report the combination key event to the foreground application.

[0251] Step S904 may be that the InputDispatcher reports the combined key event to the view, and then the view sends it to the foreground application. Assume that only the control key event, that is, the ctrl event, is obtained in step S901, then the ctrl event is directly reported to the foreground application, and the subsequent steps to implement the zoom function will not be executed. Assume that only the scroll wheel event is obtained in step S901, then the scroll wheel event is directly reported to the foreground application, and the subsequent steps to implement the zoom function will not be executed. The foreground application may respond to the scroll wheel event and perform page flipping.

[0252] Steps S902 and S904 mainly make a preliminary judgment on the combined key event. Only when the combined key event is indeed the combined key event for the zoom function will the subsequent steps be continued; otherwise, the combined key event is directly reported to the foreground application.

[0253] Steps S903 and S904 mainly make a preliminary judgment on the foreground application. Only when the foreground application supports the zoom function will the subsequent steps be continued; otherwise, the combined key event is directly reported to the foreground application.

[0254] It should be understood that there is no limitation on the sequence of steps S902 and step S903. Step S903 can also be executed before, after, or during step S901, without limitation.

[0255] S905. Judge whether the injection of the previous round is still in progress. When the judgment result is yes, execute step S906; when the judgment result is no, execute step S907.

[0256] S906. Discard the combined key event.

[0257] S907. Inject for a period of time to generate a zoom touch event.

[0258] Step S907 can be regarded as an example of step S802.

[0259] Steps S905 - S907 can mainly achieve that for continuous combined key operations, through the injection of each round, the generated zoom touch event is equivalent to the actual two-finger pinch event.

[0260] S908. Report the zoom touch event to the foreground application.

[0261] Optionally, when step S907 is completed, in addition to executing step S908, the step of discarding the combined key event can also be executed.

[0262] Figure 10 It is a schematic flowchart of another method for processing interaction events in an embodiment of the present application.Figure 10 can be regarded as Figure 8 or Figure 9 an example of the method shown

[0263] S1001. The hardware driver reports the combined key event to the InputReader.

[0264] Step S1001 can be regarded as an example of step S801 or S901.

[0265] S1002. The InputReader reports the combined key event to the InputDispatcher.

[0266] S1003. The InputDispatcher reports the combined key event to the InputFilter in the InputManagerService, and the InputFilter filters before event distribution.

[0267] S1004. The InputFilter determines whether the combined key event is a zoom combined key event.

[0268] Step S1004 can be regarded as an example of S902.

[0269] S1005. The InputFilter determines whether the foreground application supports zooming.

[0270] Step S1005 can be regarded as an example of S903.

[0271] S1006. The InputFilter injects a zoom touch event into the InputDispatcher.

[0272] Step S1006 can be regarded as an example of step S802 or S907.

[0273] S1007. The InputDispatcher sends a zoom touch event to the foreground application.

[0274] Step S1007 can be regarded as an example of step S908.

[0275] Step S1007 can be that the InputDispatcher reports the zoom touch event to the view, and then the view sends it to the foreground application.

[0276] It can be seen that Figure 10 the method shown is mainly introduced by taking the first type of interaction scenario as an example, and Figure 10The method shown is a modification based on the existing interactive event processing framework and processing flow, adding steps S1003 - S1006. As a result, what step S1007 sends to the foreground is a zoom touch event instead of a combination key event, enabling the view to respond to the zoom touch event, thus realizing the zoom function triggered by the combination key event as a whole. It should be understood that those skilled in the art can also implement the conversion from the combination key event to the zoom touch event in other suitable processing modules or by designing a separate processing module, but the modifications required will be Figure 10 more. Figure 10 Based on the modification made on the basis of the existing processing framework and processing flow, the modification is small, the development cost is low, and it is more convenient.

[0277] The method of the embodiment of the present application has been mainly introduced above in combination with the drawings. It should be understood that although the steps in the flowcharts involved in the above - described embodiments are displayed in sequence, these steps are not necessarily executed in the order shown in the figures. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above - described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps. The device of the embodiment of the present application will be introduced below in combination with the drawings.

[0278] Figure 11 is a schematic diagram of a device for processing interactive events according to an embodiment of the present application. As Figure 11 shown, the device 2000 includes an acquisition unit 2001 and a processing unit 2002. The device 2000 can be any of the above - mentioned touch - screen electronic devices, such as electronic device D or electronic device 400.

[0279] The device 2000 can be used to execute any of the above - mentioned methods for processing interactive events. For example, the acquisition unit 2001 can be used to execute step S801, and the processing unit 2002 can be used to execute steps S802 and S803. Also, for example, the processing unit 2002 can also be used to execute the step of determining whether the combination key event is a combination key event corresponding to the zoom function, and / or used to execute the step of determining whether the foreground application supports the zoom function. Also, for example, the acquisition unit 2001 can also be used to execute step S901, and the processing unit 2002 can also be used to execute steps S902 to S908. Also, for example, the acquisition unit 2001 can also be used to execute step S1001, and the processing unit 2002 can also be used to execute steps S1002 to S1007.

[0280] In one implementation, the apparatus 2000 may further include a storage unit for storing data such as interaction events and configuration information of applications that support the zoom function. The storage unit may be integrated in the processing unit 2002 or may be a unit independent of the acquisition unit 2001 and the processing unit 2002.

[0281] It should be noted that for the information interaction, execution process, etc. between the above-mentioned apparatus / units, since they are based on the same concept as the method embodiments of the present application, their specific functions and the technical effects brought about can be specifically referred to in the method embodiment part, and will not be elaborated here.

[0282] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here.

[0283] The embodiment of the present application further provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, and when the processor executes the computer program, the steps in any of the above methods are implemented.

[0284] The embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each of the above method embodiments can be implemented.

[0285] The embodiment of the present application provides a computer program product, and when the computer program product runs on a mobile terminal, the mobile terminal is enabled to implement the steps in each of the above method embodiments when executed.

[0286] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0287] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0288] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0289] In the embodiments provided in this application, it should be understood that the disclosed device / network device and method can be implemented in other ways. For example, the device / network device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0290] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0291] It should be understood that when used in the specification of this application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0292] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0293] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining" or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" according to the context.

[0294] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0295] The reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure or characteristic described in combination with the embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0296] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A processing system for interaction events, characterized in that, the system includes a first electronic device and a second electronic device, the systems of the first electronic device and the second electronic device are different, the second electronic device is connected to a keyboard and a mouse, and the first electronic device is not connected to the keyboard and the mouse; the first electronic device is configured to project the screen of the first interface of the first application to the second electronic device when the first interface of the first application is displayed; the second electronic device is configured to display a projection window after the first electronic device projects the screen to the second electronic device, and the display content in the projection window is the first interface; the second electronic device is further configured to receive keyboard and mouse operations acting on a first display element, and send an interaction event corresponding to the keyboard and mouse operations to the first electronic device, where the first display element is a display element in the projection window; the first electronic device is further configured to filter the interaction event to obtain a combined event, where the combined key event is a combined event of a control key event and a mouse wheel event, and the combined key event is used to trigger a zoom function; the first electronic device is further configured to convert the combined event into a zoom touch event, and distribute the zoom touch event to the first application, and the first application zooms the first display element in the first interface.

2. The processing system according to claim 1, characterized in that, the first electronic device is further configured to convert the parameters of the combined key event into corresponding parameters in the zoom touch event.

3. The processing system according to claim 2, characterized in that, the combined key event is a combined event of a control key event and a mouse wheel event; the parameters of the combined key event include the rolling direction and rolling amount of the wheel; the parameters of the zoom touch event include the pinch direction and pinch distance; the first electronic device is further configured to convert the rolling amount into the pinch distance and convert the rolling direction into the pinch direction.

4. The processing system according to any one of claims 1 to 3, characterized in that, the first electronic device is further configured to determine whether the conversion of the combined key event in the previous round is still in progress; when the conversion of the combined key event in the previous round is still in progress, discard the combined key event; or, when the conversion of the combined key event in the previous round is not in progress, convert the combined key event into the zoom touch event.

5. The processing system according to any one of claims 1 to 3, characterized in that, the first electronic device is further configured to determine whether the first application supports zooming; when the first application supports zooming, in response to the zoom touch event, zoom the first interface of the first application; or, when the first application does not support zooming, end the processing flow of the interaction event.

6. The processing system according to claim 5, characterized in that, The first electronic device is further configured to compare the information of the first application with the pre-configured information of the whitelist applications to determine whether the first application is an application in the whitelist applications, where the whitelist applications include at least one application that supports a zoom function; When the first application is an application in the whitelist applications, it is considered that the first application supports zooming; Or, When the first application is not an application in the whitelist applications, it is considered that the first application does not support zooming.

7. The processing system according to any one of claims 1 to 3, wherein, The first electronic device is further configured to discard the combined key event after the zoom touch event is generated.

8. The processing system according to any one of claims 1 to 3, wherein, The first electronic device is further configured to filter the interaction event through the input dispatcher to obtain a combined event.

9. A method for processing an interaction event, wherein, A first electronic device displays a first interface of a first application; The first electronic device is projected onto a second electronic device. Wherein, the second electronic device displays a projection window, and the content displayed in the projection window is the first interface. The second electronic device is connected to a keyboard and a mouse, and the first electronic device is not connected to the keyboard and the mouse; In response to a keyboard and mouse operation on a first display element on the second electronic device, the first electronic device obtains an interaction event sent by the second electronic device to the first electronic device after obtaining the keyboard and mouse operation. The first display element is a display element in the projection window; The first electronic device filters the interaction event to obtain a combined key event, where the combined key event is a combined event of a control key event and a mouse wheel event, and the combined key event is used to trigger a zoom function; The first electronic device converts the combined key event into a zoom touch event; The first electronic device distributes the zoom touch event to the first application, and the first application zooms the first display element in the first interface.

10. An electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, the method according to claim 9 is implemented.

11. A computer-readable storage medium, the computer-readable storage medium stores a computer program, wherein, When the computer program is executed by a processor, the method according to claim 9 is implemented.

Images