Operation processing method and first equipment

Abstract

The invention discloses an operation processing method and first equipment, and relates to the technical field of equipment application, and the method comprises the steps: building communication connection between the first equipment and second equipment; in response to the detected target operation of the target application, controlling the first equipment to send display data of the target application processed by the display card of the first equipment to a display of the second equipment for display based on the communication connection, and controlling the first equipment to display a target operation interface of the target application; in response to an application operation executed on the target operation interface, determining an application event triggered by the application operation, and processing the application event to enable a target application to respond to the application event so as to update the display data; the application event comprises a first type of event or a second type of event, the first type of event can be responded by the target application after being processed based on a public service corresponding to a plurality of applications provided by the first equipment, and the second type of event can be responded by the target application after being processed based on a private service corresponding to the target application provided by the first equipment.

Description

Technical Field

[0001] The present application relates to the field of device application technology, and in particular to an operation processing method and a first device. Background Art

[0002] In screen projection scenarios such as projecting an application from a mobile terminal to a large-screen device, the projected application is usually controlled by operating a remote control or a remote keyboard. However, this method has defects such as inconvenient operation, limited operation control functions, and difficulty in achieving comprehensive operation control. Summary of the invention

[0003] To this end, this application discloses the following technical solutions:

[0004] An operation processing method, applied to a first device, comprising:

[0005] Establishing a communication connection between the first device and the second device;

[0006] In response to detecting a target operation of a target application, controlling the first device to send display data of the target application processed by a graphics card of the first device to a display of a second device for display based on the communication connection, and controlling the first device to display a target operation interface of the target application;

[0007] In response to an application operation performed on the target operation interface, determining an application event triggered by the application operation, and processing the application event so that the target application responds to the application event to update the display data;

[0008] Among them, the application events include first-type events or second-type events. The first-type events can be responded to by the target application after being processed based on the public services corresponding to multiple applications provided by the first device, and the second-type events can be responded to by the target application after being processed based on the private services corresponding to the target application provided by the first device.

[0009] Optionally, in response to the application operation performed on the target operation interface, determining an application event triggered by the application operation includes:

[0010] In response to a first application operation corresponding to a first type of control or a first gesture of the target operation interface, determining a first type of event triggered by the first application operation;

[0011] In response to a second application operation corresponding to a second type of control or a second gesture of the target operation interface, a second type of event triggered by the second application operation is determined, the first type of control is different from the second type of control, and the first gesture is different from the second gesture.

[0012] Optionally, the target application includes a first application and a second application different from the first application;

[0013] The target operation interface includes a first operation interface for operating the first application and a second operation interface for operating the second application;

[0014] The first operation interface includes a first second-category control, and the second operation interface includes a second second-category control. When the first second-category control is operated, it can be responded to by the first application to provide a first function based on the private service processing corresponding to the first application provided by the first device. When the second second-category control is operated, it can be responded to by the second application to provide a second function based on the private service processing corresponding to the second application provided by the first device. The first function is different from the second function.

[0015] Optionally, establishing a communication connection between the first device and the second device includes:

[0016] In response to the specific application being started, establishing the communication connection;

[0017] The specific application can be used to provide a target operation interface of the target application based on detecting a target operation of the target application, wherein the target operation interfaces of different target applications are different.

[0018] Optionally, the public service maps the first type of event to a display interface of a target application based on the location relationship so as to be responded to by the target application;

[0019] The private service does not perform location mapping for the second type of events. The private service transmits the event information to the target application through the application program interface between the target operation interface and the target application so that the target application can respond to it. For the second type of events, different target applications are connected to different application program interfaces to implement the private service.

[0020] Optionally, the first type of event can be responded to by the target application after being processed based on public services corresponding to multiple applications provided by the first device, including:

[0021] A public service event corresponding to the first type of event is constructed, and the public service event is transmitted to the virtual display interface of the target application through an event distribution module to be responded to by the target application, wherein the event distribution module is capable of responding to user input operations.

[0022] Optionally, the public service maps the first type of event to a display interface of a target application based on a first correspondence to generate response event information of the target application, and the first correspondence has a mapping relationship with a first precision with the display interface;

[0023] The private service maps the second type of event to the display interface of the target application based on a second correspondence to generate response event information of the target application. The second correspondence has a mapping relationship with the display interface with a second precision, and the second precision is higher than the first precision.

[0024] Optionally, the public service maps the first type of event to a display interface of a target application based on the first corresponding relationship to generate response event information of the target application, including:

[0025] The first type of event information of the first type of event is passed to the virtual device node that matches the target operation interface on the first device, and the virtual device node is a virtual display interface corresponding to the target application display interface. The first type of event is mapped to a target area of ​​a first size corresponding to the target control of the virtual display interface based on a first corresponding relationship so as to be responded to by the target program.

[0026] Optionally, the private service maps the second type of event to a display interface of a target application based on a second corresponding relationship to generate response event information of the target application, including:

[0027] The second type of event information of the second type of event is passed to the virtual device node that matches the target operation interface on the first device, and the virtual device node is a virtual display interface corresponding to the target application display interface. The second type of event is mapped to a target area of ​​a second area size corresponding to the target control of the virtual display interface based on a second corresponding relationship so as to be responded to by the target program, and the second area size is smaller than the first area size.

[0028] An operation processing device, comprising:

[0029] A connection establishing module, used to establish a communication connection between the first device and the second device;

[0030] A first response module, configured to, in response to detecting a target operation of a target application, control the first device to send display data of the target application processed by a graphics card of the first device to a display of a second device for display based on the communication connection, and control the first device to display a target operation interface of the target application;

[0031] a second response module, configured to respond to an application operation performed on the target operation interface, determine an application event triggered by the application operation, and process the application event so that the target application responds to the application event to update the display data;

[0032] Among them, the application events include first-type events or second-type events. The first-type events can be responded to by the target application after being processed based on the public services corresponding to multiple applications provided by the first device, and the second-type events can be responded to by the target application after being processed based on the private services corresponding to the target application provided by the first device.

[0033] A first device includes:

[0034] A specific application, capable of providing a target operation interface of the target application based on detecting a target operation of the target application;

[0035] Processor for:

[0036] Establishing a communication connection between the first device and the second device;

[0037] In response to detecting a target operation of a target application, controlling the first device to send display data of the target application processed by a graphics card of the first device to a display of a second device for display based on the communication connection, and controlling the first device to display a target operation interface of the target application;

[0038] In response to an application operation performed on the target operation interface, determining an application event triggered by the application operation, and processing the application event so that the target application responds to the application event to update the display data;

[0039] Among them, the application events include first-type events or second-type events. The first-type events can be responded to by the target application after being processed based on the public services provided by the first device, and the second-type events can be responded to by the target application after being processed based on the private services of the corresponding target application provided by the first device.

[0040] A storage medium carries one or more computer instruction sets, which, when executed by an electronic device, can enable the electronic device to implement any of the operation processing methods described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 It is a flow chart of the operation processing method provided by this application;

[0043] Figure 2 and Figure 3These are examples of different target operation interfaces corresponding to different target applications provided by this application;

[0044] Figure 4 is an exemplary implementation flow chart of the operation processing method provided by this application;

[0045] Figure 5 It is a structural diagram of the operation processing device provided by the present application;

[0046] Figure 6 This is a composition structure diagram of the first device provided in this application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The embodiment of the present application provides an operation processing method and a first device, which are used to solve the defects of inconvenience in operation control of the projected application in the application screen projection scenario, limited operation control functions, difficulty in achieving comprehensive operation control, etc. The operation processing method provided in the embodiment of the present application can be applied to the first device, which can be an electronic device in many general or special computing device environments or configurations, such as: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor devices, etc.

[0049] See also Figure 1 As shown in the method flow chart, the operation processing method provided in the embodiment of the present application may include the following steps 101 to 103, and these steps are described in detail below.

[0050] Step 101: Establish a communication connection between a first device and a second device.

[0051] The second device and the first device are independent of each other.

[0052] The communication connection can be a wireless connection or a wired connection, and can be, but is not limited to, a wireless connection such as a Bluetooth connection, a Wi-Fi connection, or a wired connection established through cables such as HDMI (High Definition Multimedia Interface) and VGA (Video Graphic Array), and can be used to communicate between the first device and the second device to provide support for related applications based on communication, such as screen projection applications.

[0053] In actual applications, optionally, the communication connection may be established in response to a specific application being started. The specific application may be, but is not limited to, a customized device interconnection application.

[0054] The specific application can be used to provide a target operation interface of the target application based on detecting the target operation of the target application.

[0055] The target operation is used to trigger screen projection of the target application, and may specifically be, but is not limited to, an operation performed by the user on the specific application of the first device through any one or more methods such as mouse, keyboard, gesture or voice to trigger screen projection of the target application.

[0056] Among them, the target operation interfaces of different target applications are different. That is to say, in the embodiment of the present application, for each different target application on the first device that is projected to other devices, the specific application can provide independent different target operation interfaces for these different target applications, so as to provide matching operation control functions for each target application as needed through the independent target operation interface corresponding to each target application.

[0057] The target operation interface may be, but is not limited to, implemented in the form of a virtual function keyboard, for example, specifically implemented in the form of a virtual "remote function keyboard".

[0058] See also Figure 2 and Figure 3 ,in Figure 2 Provides an example of the target operation interface for the projected "WPS" such as PPT, an office application. Figure 3 An example of a target operation interface for the social application "TikTok" being projected is provided. The two different target applications correspond to different target operation interfaces, which are specifically different remote function keyboards that match their respective target applications, and are used to provide matching operation control functions for their respective corresponding target applications on demand.

[0059] Step 102: In response to detecting a target operation of a target application, controlling the first device to send display data of the target application processed by the graphics card of the first device to a display of the second device for display based on the communication connection, and controlling the first device to display a target operation interface of the target application.

[0060] As mentioned above, the target operation is used to trigger screen projection of the target application. In response to detecting the target operation of the target application, the embodiment of the present application performs two aspects of control processing. On the one hand, the target application is projected, specifically controlling the first device to send the display data of the target application processed by the graphics card of the first device to the display of the second device based on the communication connection, so as to realize the projection of the target application of the first device to the second device for application interface display; on the other hand, the first device is controlled to display the target operation interface of the target application, which is an independent target operation interface corresponding to the target application, so as to provide efficient, convenient and comprehensive operation control functions for the target application through the independent target operation interface corresponding to the target application.

[0061] In practice, the specific application can be pre-developed and deployed on the first device to provide an independent target operation interface for each target application on the first device. In addition, the present application specifically provides general operation control functions and / or customized operation control functions for each target application through an independent target operation interface corresponding to the target application.

[0062] Among them, the general control function can also be called a general event function. The general control function / general event function is used to provide control functions for general events. The general events are operation events that can be used in different applications and are defined by the device input system (Input System) such as the Android input system, including but not limited to various touch events (click, slide, etc.), keyboard events (different keys or combinations), and mouse events (click, hover, etc.).

[0063] The customized operation control function corresponding to the target application is used to provide a control function for customized events for the target application. The customized event refers to a unique operation control event corresponding to the target application that cannot be implemented through general events. More specifically, it may refer to an event that has not been defined by a device input system such as the Android input system and is other than the above-mentioned general operation events such as touch, keyboard, mouse, etc. It may include but is not limited to sensor events, remote sensing events, voice input, camera-based gesture recognition, and complex combination events implemented in other ways (such as posting a comment: click the comment button, enter text, click the post button), etc.

[0064] At least one of an operation interface corresponding to the general operation control function and an operation interface corresponding to the customized operation control function is provided on the target operation interface of the target application.

[0065] The operation interface corresponding to the general operation control function may include but is not limited to a first type of control and / or a first gesture operation area to support users to implement relevant general operation control of the target application through a first gesture and / or a first type of control operation (such as clicking a button) performed based on a mouse, keyboard, touch, etc.

[0066] The operation interface corresponding to the customized operation control function may include but is not limited to a second type of control, a second gesture operation area, a voice interface, a remote sensing interface, etc., to support users to implement relevant customized operation control of the target application through control operation methods such as voice interface controls, remote sensing controls, complex combination controls (such as complex combination control operations such as clicking a comment button, entering text, clicking a publish button, etc. corresponding to a "comment" event), or a second gesture.

[0067] The first type of control or the second type of control can be implemented in various forms such as, but not limited to, buttons, date / time controls, text boxes, selection boxes / list boxes, control bars, etc., depending on actual needs. Figure 2 and Figure 3 , Figure 2 The "Previous Page", "Next Page" and "Play / Exit" buttons are the operation interfaces for the general operation control functions of "WPS", the screen-projected office application. Figure 3 The "swipe up", "swipe down" and "play / pause" buttons are operation interfaces for the general operation control functions of "TikTok", the projected social application. The operation events corresponding to these buttons are operation events that have been defined by the device input system, such as the Android input system, and can be used in different applications. For example, "previous page" / "swipe up" can be used in different target applications to switch objects in a first direction (such as left or up), "next page" / "swipe down" can be used in different target applications to switch objects in a second direction (such as right or down), and "play / exit" or "play / pause" can be used in different target applications to control content playback, and so on. Figure 3 It also provides an operation interface for customized operation control functions of "TikTok", the projected social application, such as a comment control. The operation event corresponding to the control is an operation event that has not been defined by the device input system, such as the Android input system.

[0068] Step 103: In response to the application operation performed on the target operation interface, determine an application event triggered by the application operation, and process the application event so that the target application responds to the application event to update the display data.

[0069] Among them, the application events include first-type events or second-type events. The first-type events can be responded to by the target application after being processed based on the public services corresponding to multiple applications provided by the first device, and the second-type events can be responded to by the target application after being processed based on the private services corresponding to the target application provided by the first device.

[0070] The first type of event, that is, the event corresponding to the general operation control function provided by the target operation interface of the target application, can be triggered by performing an application operation on the operation interface corresponding to the general operation control function on the target operation interface (such as the first type of control and / or the first gesture operation area); the second type of event, that is, the event corresponding to the customized operation control function provided by the target operation interface of the target application, can be triggered by performing an application operation on the operation interface corresponding to the customized operation control function on the target operation interface (such as the second type of control and / or the second gesture operation area, etc.).

[0071] Accordingly, in this step, in response to the application operation performed on the target operation interface, determining the application event triggered by the application operation may be implemented, but not limited to, as follows:

[0072] 11) In response to a first application operation corresponding to a first type of control or a first gesture of the target operation interface, determining a first type of event triggered by the first application operation.

[0073] The first type of control and the first gesture operation area corresponding to the first gesture are used to provide an operation interface corresponding to a general operation control function for the target application on the target operation interface.

[0074] The first type of controls may specifically include but are not limited to buttons, control bars (such as volume control bars, brightness control bars, play progress control bars, etc.) and other quick interface controls. The first gesture may include but are not limited to quick gesture operations such as clicking and sliding.

[0075] 12) In response to a second application operation corresponding to a second type of control or a second gesture of the target operation interface, determining a second type of event triggered by the second application operation, the first type of control being different from the second type of control, and the first gesture being different from the second gesture.

[0076] The second type of control and the second gesture operation area corresponding to the second gesture are used to provide an operation interface corresponding to the customized operation control function for the target application on the target operation interface.

[0077] The second category of controls may specifically include, but are not limited to, voice interface controls, remote sensing controls, and cascade controls for complex / combined operations (such as comment interface controls that trigger information input controls and comment publishing controls when clicked; these cascaded controls may be considered cascade controls).

[0078] The first gesture is different from the second gesture, and the first gesture may be a gesture defined by an input system such as Android and applicable to different applications, including but not limited to gestures such as click and slide. The second gesture may include some application customized gestures that are not defined by an input system such as Android.

[0079] After determining the first type of event triggered by the first application operation, or determining the second type of event triggered by the second application operation, the first application event or the second application event may be further processed so that the target application responds to the first application event or the second application event to update the display data.

[0080] Among them, processing the first application event or the second application event so that the target application responds to the first application event or the second application event to update the display data may include but is not limited to parsing the first application event or the second application event to obtain corresponding event information, such as the operation type (click / slide / button), the identification of the operated control (such as "previous page", "next page", "play / exit" button), the projection virtual screen (virtual display interface) ID, the operation position coordinates, etc., and mapping the first application event or the second application event to the display interface of the target application based on the corresponding event information, so that the target application responds to the first application event or the second application event, and then updates the display data of the target application by responding to the first application event or the second application event, and correspondingly realizes the synchronous update of the corresponding projection interface of the target application on the second device at the projection destination.

[0081] To summarize, the operation processing method provided in the embodiment of the present application, for the screen projection application scenario, after the first device establishes a communication connection with the second device and detects the target operation for projecting the target application, in addition to controlling the first device to send the display data of the target application processed by the graphics card of the first device to the display of the second device for display based on the communication connection, the first device is also controlled to display the target operation interface of the target application, so that an independent target operation interface for screen projection operation control can be provided on the first device for each projected target application, and support is provided on-demand design of general and / or customized operation control interfaces for the corresponding target application on the independent target operation interface of each target application, thereby effectively improving the operational convenience of each projected target application, while overcoming the defects of limited operation control functions of the target application and difficulty in achieving comprehensive operation control.

[0082] In an optional embodiment, the target application includes a first application and a second application different from the first application.

[0083] There is no limit to the number of target applications that are projected, and it can be one or more. The term "multiple" refers to that the number of target applications is two or more. For example, in addition to the first application and the second application mentioned above, the target application may also include other applications such as the third application and the fourth application. There is no limit to this, and it depends on the actual application situation. This embodiment mainly takes the example that the target application includes the first application and the second application, that is, the number of target applications is 2, to illustrate the situation where the number of target applications is multiple.

[0084] In the case where the target application includes a first application and a second application, the target operation interface correspondingly includes a first operation interface for operating the first application and a second operation interface for operating the second application.

[0085] The first operation interface is used to provide an operation interface for general operation control functions and / or customized operation control functions for the first application; the second operation interface is used to provide an operation interface for general operation control functions and / or customized operation control functions for the second application.

[0086] Among them, for the customized operation control function of the target application, the first operation interface includes first and second category controls, and the first and second category controls are specifically operation interfaces in the first operation interface corresponding to the corresponding customized operation control function of the first application, and are used to support the corresponding customized operation control of the first application. When the first and second category controls are operated, they can be responded by the first application to provide the first function based on the private service processing corresponding to the first application provided by the first device.

[0087] It is easy to understand that the first function is the corresponding customized control function of the first application triggered by operating the first and second type controls, and the application events generated by the operation of the first and second type controls are the second type events (customized events) of the first application. For example, by operating the corresponding first and second type controls, a customized event of adding a hyperlink / jumping based on a hyperlink to an office application (first application) such as WPS is triggered to implement a hyperlink adding function / jumping based on a hyperlink, etc.

[0088] Similarly, the second operation interface includes a second second-category control, which is an operation interface in the second operation interface corresponding to a corresponding customized operation control function for the second application, and is used to support corresponding customized operation control for the second application. When the second second-category control is operated, it can be responded to by the second application to provide a second function based on the private service processing corresponding to the second application provided by the first device.

[0089] The first function is different from the second function, and they are different customized operation control functions for the first application and the second application, respectively. That is, different from the first function, the second function is a corresponding customized control function for the second application triggered by operating the second second-class control, and the application event generated by the operation of the first second-class control is a second-class event (customized event) for the second application, for example, by operating the corresponding second second-class control, a comment event (customized event of the social application) for a social application such as Tik Tok is triggered to implement the corresponding comment function, etc.

[0090] For the general operation control function of the target application, the first operation interface includes a first first-category control, and the first first-category control is specifically an operation interface in the first operation interface corresponding to the corresponding general operation control function of the first application, and is used to support the corresponding general operation control of the first application. The application event generated by the operation of the first first-category control is a first-category event (general event) for the first application, and the public service maps the first-category event generated by the operation of the first first-category control to the display interface of the first application based on the first corresponding relationship to generate the first response event information of the first application.

[0091] Similarly, the second operation interface includes a second first-category control, and the second first-category control is specifically an operation interface in the second operation interface corresponding to the corresponding general operation control function of the second application, and is used to support the corresponding general operation control of the second application. The application event generated by the operation of the second first-category control is a first-category event (general event) for the second application, and the public service maps the first-category event generated by the operation of the second first-category control to the display interface of the second application based on the first corresponding relationship to generate second response event information of the second application.

[0092] The second response event information has different functions from the first response event information, and is response event information that matches the respective applications when the corresponding general event is responded to in the second application and the first application. Figure 2 and Figure 3 , assuming that the first application and the second application are office applications such as WPS and social applications such as Douyin, respectively, the first first-category control may include object switching controls such as "previous page" and "next page" and playback control controls such as "play / exit" on the remote operation panel of office applications such as WPS, and the second first-category control may include object switching controls such as "swipe up" and "swipe down" and playback control controls such as "play / pause" on the remote operation panel of social applications such as Douyin. If the object switching controls such as "previous page / next page" are operated on the remote operation panel of office applications such as WPS, the first response event information is specifically response event information for up / down page switching of office applications such as WPS, such as PPT. If the object switching controls such as "swipe up / swipe down" are operated on the remote operation panel of social applications such as Douyin, the second response event information is specifically response event information for up / down switching videos of social applications such as Douyin. The first response event information and the second response event information are respectively matched with the applications in which the corresponding general events are located.

[0093] Optionally, the first correspondence includes the correspondence between the application operation information corresponding to the first type of event and the corresponding general event information, for example, the correspondence between the operation information of clicking the upward switching button such as "previous page" or "swipe up" and the event information such as the upward switching object (turning the page up / switching video), the correspondence between the operation information of clicking the downward switching button such as "next page" or "swipe down" and the event information such as the downward switching object (turning the page, switching video), etc. The first application relationship is used to support the public service to parse the first type of event triggered by operating the first type of control / first gesture operation area to obtain the event information of the first type of event, such as the operation type (click / slide / key), the identifier of the operated control (such as "previous page / swipe up", "next page / swipe down", "play / exit / pause" button), the projection virtual screen (virtual display interface) ID, the operation position coordinates, etc., so as to facilitate the public service to map the first type of event to the display interface of the corresponding target application (such as the first application, the second application) based on the event information of the first type of event so that it can be responded by the target application. For example, the public service will map the upward switching object event triggered by clicking the "previous page" button on the remote function keyboard of PPT to the display interface of PPT so that PPT can switch pages upward in response to the event.

[0094] During implementation, the public service may specifically parse the event information of the first type of event based on the first corresponding relationship and in combination with the actual corresponding target application (such as office application, social application).

[0095] This embodiment provides different target operation interfaces such as independent first operation interfaces and second operation interfaces for different target applications such as the projected first application and the second application on the first device, and provides a first first-category control and a first second-category control in the first operation interface for the first application, and provides a second first-category control and a second second-category control in the second operation interface for the second application. These can be used to support the required general operation control and / or customized operation control for different target applications such as the first application and the second application, thereby enabling convenient operation control of each projected target application, while overcoming the defects of limited operation control functions for the target application and difficulty in achieving comprehensive operation control.

[0096] In an optional embodiment, the public service maps the first type of event to a display interface of a target application based on a location relationship so that the target application can respond to it.

[0097] The position relationship is used to characterize the correspondence between the operation position of the operation performed on the target operation interface corresponding to the target application and the corresponding interface position on the display interface of the target application.

[0098] In implementation, optionally, the control position of each first-category control (such as a button) on the target operation interface may be pre-mapped to the corresponding interface position on the target application display interface, and / or the different area positions of the first gesture operation area may be mapped to the corresponding interface position on the target application display interface, so that there is a certain correspondence between the control position of each first-category control and the corresponding interface position on the target application display interface, and this embodiment refers to this correspondence as a first position relationship, and / or there is a certain correspondence between the different area positions of the first gesture operation area and the corresponding interface position on the target application display interface, and this embodiment refers to this correspondence as a second position relationship. The position relationship may include the first position relationship and / or the second position relationship.

[0099] On this basis, subsequent public services can map the first type of event triggered by operating the first type of control on the target operation interface to the display interface of the target application according to the first position relationship, so as to be responded to by the target application.

[0100] Similarly, the public service can map the first type of event triggered by performing the first gesture in the first gesture operation area on the target operation interface to the display interface of the target application according to the second position relationship, so as to be responded to by the target application.

[0101] Taking the position mapping of the first type of control as an example, illustratively, the control position of the object switching control (first type of control) in the first direction on the target operation interface, such as the button position of the "previous page" / "swipe up" button, etc., can be mapped to the left half / upper half interface area of ​​the target application display interface, and the control position of the object switching control (first type of control) in the second direction on the target operation interface, such as the button position of the "next page" / "swipe down" button, etc., can be mapped to the right half / lower half interface area of ​​the target application display interface, thereby obtaining the corresponding position relationship. Subsequently, based on the position relationship, the first type of event triggered by the user operating the object switching control in the corresponding direction can be mapped to the display interface of the target application, for example, the upward page change event triggered by the user operating the "previous page" button is mapped to the upper half interface area of ​​the target application display interface, indicating that the target application needs to be paged upward in order to be responded to by the target application.

[0102] The mapping between the positions represented by the positional relationship in this embodiment is a fuzzy mapping, which specifically represents the correspondence between the operation position on the target operation interface and the corresponding interface area on the display interface of the target application rather than a precise position point. In actual applications, based on the positional relationship, the operation position on the target operation interface can be mapped to any position in the area corresponding to the operation position on the target application display interface, so as to realize the mapping of the first type of event to the target application display interface based on the positional relationship.

[0103] The private service does not perform location mapping for the second type of event, and the private service transmits the event information to the target application through the application program interface between the target operation interface and the target application to be responded to by the target application.

[0104] Among them, for the second type of events, different target applications are connected to different application programming interfaces to realize independent private services for different services. Specifically, in this embodiment, for each target operation interface with customized operation control function (such as having the second type of controls and / or the second gesture operation area), an independent application programming interface is customized and developed for the target operation interface, different target operation interfaces correspond to different application programming interfaces, and the independent application programming interface corresponding to the target operation interface is used to connect the target operation interface and the target application corresponding to the target operation interface, so as to realize the provision of corresponding private services between the target operation interface and its corresponding target application by connecting the independent application programming interface between the target operation interface and its corresponding target application.

[0105] On this basis, for the second type of event corresponding to the customized operation control function triggered by operating the target operation interface, the private service can pass the event information of the second type of event to the target application through the application program interface between the target operation interface and the target application so that the target application can respond.

[0106] Take the example of a remote function keyboard provided by a specific application such as a social application such as TikTok, where the target application is a device interconnection application, etc., and where the target operation interface is a device interconnection application, etc., and assuming that the remote function keyboard provides customized operation control functions for the social application, such as a comment function, a customized application program interface A can be developed for the social application, and the remote function keyboard of the social application is connected to the social service based on the application program interface A to provide a corresponding private service between the remote function keyboard of the social application and the social service. Subsequently, for the second type of event of the social application triggered by operating the remote function keyboard, the private service can pass the event information of the second type of event to the social application through the application program interface A so that the social application can respond. For example, after the user clicks on the comment on the remote function keyboard, the comment information sent by the user is directly passed to the comment input interface of the corresponding social application through the application program interface A so that the social application can respond.

[0107] Among them, on the first device, each projected target application corresponds to a virtual display interface, and different target applications correspond to different virtual display interfaces. Mapping the corresponding application event (such as the first application event) triggered by the application operation to the display interface of the target application can refer to mapping the corresponding application event triggered by the application operation to the virtual display interface of the target application.

[0108] The following is an application example of the present embodiment. In this example, the first type of event is responded to by the target application after being processed based on the public service corresponding to multiple applications provided by the first device. It can be further implemented as: constructing a public service event corresponding to the first type of event, and delivering the public service event to the virtual display interface of the target application through an event distribution module to be responded to by the target application, wherein the event distribution module can respond to user input operations.

[0109] In this implementation, after determining the first type of event triggered by the application operation on the target operation interface, a public service event corresponding to the first type of event can be constructed. Figure 4 The operation processing implementation flow chart shown in the figure can use the event processing module set in the application layer to construct a public service event corresponding to the first type of event. The event information of the constructed public service event may include but is not limited to event parameters such as operation type (click / slide / button), projection virtual screen (virtual display interface) ID, operation position coordinates, etc. After obtaining the public service event corresponding to the first type of event, the public service event can be further transmitted to the virtual display interface of the target application through the event distribution module to be responded by the target application. Optionally, as Figure 4 As shown, in implementation, an input manager for input management and an event distributor (event distribution module) for event distribution can be set at the system framework layer. For public service events corresponding to the first type of events, the event information of the public service event, such as the various event parameters mentioned above, can first be sent to the input manager of the system framework layer, and the input manager passes the event information of the public service event to the event distributor for distribution. The event distributor distributes the event information of the public service event to the virtual display interface of the corresponding target application according to the projection virtual screen (virtual display interface) ID so that the target application can respond to it.

[0110] In this example, Figure 4 As shown, for the second type of event (corresponding to the customized operation control function of the target application), the event processing module can call the customized application program interface corresponding to the target application at the application layer, and based on the called customized application program interface, the event information of the second type of event, such as comment information, is delivered to the virtual display interface of the target application through cross-process communication to be responded by the target application.

[0111] This embodiment maps the first type of event to the display interface of the target application based on the location relationship in the public service so that the target application can respond to it, and transmits the event information of the second type of event to the target application based on the application program interface between the target operation interface and the target application in the private service so that the target application can respond to it. This enables general operation control of the target application corresponding to the first type of event based on the target operation interface, and / or customized operation control corresponding to the second type of event, thereby improving the operating convenience of the projected target application and overcoming the defects of limited operation control functions of the target application and difficulty in achieving comprehensive operation control.

[0112] In an optional embodiment, the public service maps the first type of event to a display interface of a target application based on a first corresponding relationship to generate response event information of the target application.

[0113] As described above, the first correspondence includes the correspondence between the application operation information corresponding to the first type of event and the corresponding general event information, for example, the correspondence between the operation information of clicking the upward switching button such as "previous page" or "swipe up" and the event information such as the upward switching object (turning the page up / switching video), the correspondence between the operation information of clicking the downward switching button such as "next page" or "swipe down" and the event information such as the downward switching object (turning the page, switching video), etc. The first application relationship is used to support the public service to parse the first type of event triggered by operating the first type of control to obtain the event information corresponding to the first type of event, thereby facilitating the public service to map the first type of event to the display interface of the corresponding target application (such as the first application, the second application) based on the event information of the first type of event so that it can be responded to by the target application.

[0114] The first corresponding relationship is common to all target applications that are projected.

[0115] The first correspondence has a mapping relationship of first precision with the display interface of the target application. The public service can map the first type of event to a target area of ​​a first area size corresponding to a target control on a virtual display interface corresponding to the target application based on the first correspondence. The first area size matches and is negatively correlated with the first precision, and the lower the first precision, the larger the first area.

[0116] Among them, the mapping of the first type of event performed based on the first corresponding relationship is a fuzzy mapping, which is specifically used to map the first type of event triggered by the corresponding control on the target operation interface (such as buttons for switching objects upward such as previous page / swipe up) performing application operations to a target area of ​​a first area size corresponding to the target control on the virtual display interface corresponding to the target application (mapping to any position within the area), rather than mapping to a precise location point. For example, the upward switching object event triggered by clicking the previous page / swipe up button is mapped to the upper interface area of ​​the virtual display interface of the office application or social application, so that the office application or social application can respond to switch objects upward (such as changing pages or switching videos).

[0117] In this embodiment, the public service maps the first type of event to the display interface of the target application based on a first correspondence relationship to generate response event information of the target application, which can be further implemented as: transmitting the first type of event information of the first type of event to a virtual device node matched by the target operation interface on the first device, the virtual device node being a virtual display interface corresponding to the display interface of the target application, and the first type of event is mapped to a target area of ​​a first size corresponding to the target control of the virtual display interface based on the first correspondence relationship so as to be responded to by the target program.

[0118] Specifically, the public service may transmit the first type of event information of the first type of event to the virtual device node matched by the target operation interface on the first device through the event distribution module.

[0119] The private service maps the second type of event to a display interface of a target application based on a second corresponding relationship to generate response event information of the target application.

[0120] The second correspondence includes the correspondence between the application operation information and the event information corresponding to the second type of event. For example, such as the correspondence between the operation information of "clicking the comment button and entering content" and the event information of the event "initiating a comment", the second correspondence is used to support the private service of the target application to parse the second type of event to obtain the event information of the second type of event, such as the operation type of the second type of event (such as click / slide), the identifier of the operated control, the corresponding virtual display interface ID, the operation position coordinates, etc., and then map the second type of event to the display interface of the target application based on the event information to generate the response event information of the target application.

[0121] The second corresponding relationship is specifically used for the corresponding target application, and different target applications correspond to different second corresponding relationships.

[0122] The second corresponding relationship has a mapping relationship with the display interface with a second precision. The private service can map the second type of event to a target area of ​​a second area size corresponding to a target control on the virtual display interface corresponding to the target application based on the second corresponding relationship. The second area size matches and is negatively correlated with the second precision, and the higher the second precision, the smaller the second area.

[0123] The second precision is higher than the first precision.

[0124] The mapping of the second type of event based on the second corresponding relationship is a precise mapping, which is specifically used to map the second type of event triggered by the corresponding control (such as the "Comment" button) on the target operation interface performing an application operation to a target area of ​​a second size corresponding to the target control on the virtual display interface corresponding to the target application, such as mapping to the comment area corresponding to the comment button on the virtual display interface, so that the target application can respond to generate corresponding response event information.

[0125] In this embodiment, the private service maps the second type of event to the display interface of the target application based on the second correspondence to generate response event information of the target application, which can be further implemented as: the second type of event information of the second type of event is transmitted to the virtual device node matched by the target operation interface on the first device, the virtual device node is a virtual display interface corresponding to the display interface of the target application, and the second type of event is mapped to the target area of ​​the second size corresponding to the target control of the virtual display interface based on the second correspondence to be responded to by the target program.

[0126] Specifically, the private service may transmit the second type of event information of the second type of event to the virtual device node that matches the target operation interface on the first device through an application program interface corresponding to the target operation interface.

[0127] That is to say, for the first type of events, the public service can map it to an approximate area of ​​the virtual display interface corresponding to the target application, which is determined by the trigger of each APP itself. For the second type of events, the private service will accurately map it to the matching control area of ​​the virtual display interface corresponding to the target application, such as mapping comment information to the comment area corresponding to the comment control of the virtual display interface corresponding to the target application.

[0128] This embodiment maps the first type of event to the display interface of the target application based on a first corresponding relationship in the public service so that the target application can respond to it, and transmits the event information of the second type of event to the target application based on a second corresponding relationship in the private service so that the target application can respond to it. This enables general operation control of the target application corresponding to the first type of event based on the target operation interface, and / or customized operation control corresponding to the second type of event, thereby improving the operating convenience of the projected target application and overcoming the defects of limited operation control functions of the target application and difficulty in achieving comprehensive operation control.

[0129] Corresponding to the above-mentioned operation processing method, the embodiment of the present application also provides an operation processing device, whose composition structure is as follows: Figure 5 As shown, including:

[0130] A connection establishing module 501 is used to establish a communication connection between a first device and a second device;

[0131] A first response module 502 is used for controlling the first device to send the display data of the target application processed by the graphics card of the first device to the display of the second device for display based on the communication connection in response to detecting the target operation of the target application, and controlling the first device to display the target operation interface of the target application;

[0132] A second response module 503, configured to respond to an application operation performed on the target operation interface, determine an application event triggered by the application operation, and process the application event so that the target application responds to the application event to update the display data;

[0133] Among them, the application events include first-type events or second-type events. The first-type events can be responded to by the target application after being processed based on the public services corresponding to multiple applications provided by the first device, and the second-type events can be responded to by the target application after being processed based on the private services corresponding to the target application provided by the first device.

[0134] In an optional implementation manner, the second response module 503, in response to the application operation performed on the target operation interface, determines the application event triggered by the application operation, is specifically configured to:

[0135] In response to a first application operation corresponding to a first type of control or a first gesture of the target operation interface, determining a first type of event triggered by the first application operation;

[0136] In response to a second application operation corresponding to a second type of control or a second gesture of the target operation interface, a second type of event triggered by the second application operation is determined, the first type of control is different from the second type of control, and the first gesture is different from the second gesture.

[0137] In an optional implementation manner, the target application includes a first application and a second application different from the first application;

[0138] The target operation interface includes a first operation interface for operating the first application and a second operation interface for operating the second application;

[0139] The first operation interface includes a first second-category control, and the second operation interface includes a second second-category control. When the first second-category control is operated, it can be responded to by the first application to provide a first function based on the private service processing corresponding to the first application provided by the first device. When the second second-category control is operated, it can be responded to by the second application to provide a second function based on the private service processing corresponding to the second application provided by the first device. The first function is different from the second function.

[0140] In an optional implementation manner, the connection establishing module 501 is specifically configured to:

[0141] In response to the specific application being started, establishing the communication connection;

[0142] The specific application can be used to provide a target operation interface of the target application based on detecting a target operation of the target application, wherein the target operation interfaces of different target applications are different.

[0143] In an optional implementation manner, the public service maps the first type of event to a display interface of a target application based on a location relationship so as to be responded to by the target application;

[0144] The private service does not perform location mapping for the second type of events. The private service transmits the event information to the target application through the application program interface between the target operation interface and the target application so that the target application can respond to it. For the second type of events, different target applications are connected to different application program interfaces to implement the private service.

[0145] In an optional implementation manner, the second response module 503, when processing the first type of event based on the public service corresponding to the plurality of applications provided by the first device so as to be responded to by the target application, is specifically configured to:

[0146] A public service event corresponding to the first type of event is constructed, and the public service event is transmitted to the virtual display interface of the target application through an event distribution module to be responded to by the target application, wherein the event distribution module is capable of responding to user input operations.

[0147] In an optional implementation, the public service maps the first type of event to a display interface of a target application based on a first correspondence to generate response event information of the target application, and the first correspondence has a mapping relationship with a first precision with the display interface;

[0148] The private service maps the second type of event to the display interface of the target application based on a second correspondence to generate response event information of the target application. The second correspondence has a mapping relationship with the display interface with a second precision, and the second precision is higher than the first precision.

[0149] In an optional implementation manner, the public service maps the first type of event to a display interface of a target application based on the first corresponding relationship to generate response event information of the target application, including:

[0150] The first type of event information of the first type of event is passed to the virtual device node that matches the target operation interface on the first device, and the virtual device node is a virtual display interface corresponding to the target application display interface. The first type of event is mapped to a target area of ​​a first size corresponding to the target control of the virtual display interface based on a first corresponding relationship so as to be responded to by the target program.

[0151] In an optional implementation manner, the private service maps the second type of event to a display interface of a target application based on a second corresponding relationship to generate response event information of the target application, including:

[0152] The second type of event information of the second type of event is passed to the virtual device node that matches the target operation interface on the first device, and the virtual device node is a virtual display interface corresponding to the target application display interface. The second type of event is mapped to a target area of ​​a second area size corresponding to the target control of the virtual display interface based on a second corresponding relationship so as to be responded to by the target program, and the second area size is smaller than the first area size.

[0153] The present application also provides a first device, which has a structure Figure 6 As shown, including:

[0154] The specific application can be used to provide a target operation interface of the target application based on detecting the target operation of the target application; the specific application can be deployed on the memory 10 of the first device.

[0155] The processor 20 is used to execute the operation processing method as described in any of the method embodiments above.

[0156] Optionally, the first device also includes storage resources such as memory and cache.

[0157] Optionally, the first device also includes a camera component, and / or is connected to an external camera component.

[0158] In addition, the first device may also include components such as a communication interface and a communication bus. The memory, the processor and the communication interface communicate with each other through the communication bus.

[0159] The communication interface is used for communication between the first device and other devices. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0160] An embodiment of the present application further discloses a storage medium, which carries one or more computer instruction sets. When the one or more computer instruction sets are executed by an electronic device, the electronic device can implement the operation processing method described in any of the method embodiments above.

[0161] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0162] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0163] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or the part that makes a creative contribution can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0164] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0165] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An operation processing method, applied to a first device, the method comprising: Establishing a communication connection between the first device and the second device; In response to detecting a target operation of a target application, controlling the first device to send display data of the target application processed by a graphics card of the first device to a display of a second device for display based on the communication connection, and controlling the first device to display a target operation interface of the target application; In response to an application operation performed on the target operation interface, determining an application event triggered by the application operation, and processing the application event so that the target application responds to the application event to update the display data; Among them, the application events include first-type events or second-type events. The first-type events can be responded to by the target application after being processed based on the public services corresponding to multiple applications provided by the first device, and the second-type events can be responded to by the target application after being processed based on the private services corresponding to the target application provided by the first device.

2. The operation processing method according to claim 1, wherein in response to the application operation performed on the target operation interface, determining the application event triggered by the application operation comprises: In response to a first application operation corresponding to a first type of control or a first gesture of the target operation interface, determining a first type of event triggered by the first application operation; In response to a second application operation corresponding to a second type of control or a second gesture of the target operation interface, a second type of event triggered by the second application operation is determined, the first type of control is different from the second type of control, and the first gesture is different from the second gesture.

3. The operation processing method according to claim 2, wherein the target application comprises a first application and a second application different from the first application; The target operation interface includes a first operation interface for operating the first application and a second operation interface for operating the second application; The first operation interface includes a first second-category control, and the second operation interface includes a second second-category control. When the first second-category control is operated, it can be responded to by the first application to provide a first function based on the private service processing corresponding to the first application provided by the first device. When the second second-category control is operated, it can be responded to by the second application to provide a second function based on the private service processing corresponding to the second application provided by the first device. The first function is different from the second function.

4. The operation processing method according to claim 1, wherein establishing a communication connection between the first device and the second device comprises: In response to the specific application being started, establishing the communication connection; The specific application can be used to provide a target operation interface of the target application based on detecting a target operation of the target application, wherein the target operation interfaces of different target applications are different.

5. The operation processing method according to claim 2, wherein the public service maps the first type of event to a display interface of a target application based on a position relationship so as to be responded to by the target application; The private service does not perform location mapping for the second type of event, and the private service transmits the event information to the target application through the application program interface between the target operation interface and the target application so as to be responded to by the target application; wherein, For the second type of events, different target applications are connected to different application programming interfaces to implement private services.

6. The operation processing method according to claim 5, wherein the first type of event can be responded to by the target application after being processed based on the public services corresponding to the plurality of applications provided by the first device, comprising: A public service event corresponding to the first type of event is constructed, and the public service event is transmitted to the virtual display interface of the target application through an event distribution module to be responded to by the target application, wherein the event distribution module is capable of responding to user input operations.

7. The operation processing method according to claim 2, wherein the public service maps the first type of event to a display interface of a target application based on a first correspondence relationship to generate response event information of the target application, and the first correspondence relationship has a mapping relationship with a first precision with the display interface; The private service maps the second type of event to the display interface of the target application based on a second correspondence to generate response event information of the target application. The second correspondence has a mapping relationship with the display interface with a second precision, and the second precision is higher than the first precision.

8. The operation processing method according to claim 7, wherein the public service maps the first type of event to a display interface of a target application based on a first correspondence to generate response event information of the target application, comprising: The first type of event information of the first type of event is passed to the virtual device node that matches the target operation interface on the first device, and the virtual device node is a virtual display interface corresponding to the target application display interface. The first type of event is mapped to a target area of ​​a first size corresponding to the target control of the virtual display interface based on a first corresponding relationship so as to be responded to by the target program.

9. The operation processing method according to claim 8, wherein the private service maps the second type of event to the display interface of the target application based on the second corresponding relationship to generate response event information of the target application, comprising: The second type of event information of the second type of event is passed to the virtual device node that matches the target operation interface on the first device, and the virtual device node is a virtual display interface corresponding to the target application display interface. The second type of event is mapped to a target area of ​​a second area size corresponding to the target control of the virtual display interface based on a second corresponding relationship so as to be responded to by the target program, and the second area size is smaller than the first area size.

10. A first device, comprising: A specific application, capable of providing a target operation interface of the target application based on detecting a target operation of the target application; Processor for: Establishing a communication connection between the first device and the second device; In response to detecting a target operation of a target application, controlling the first device to send display data of the target application processed by a graphics card of the first device to a display of a second device for display based on the communication connection, and controlling the first device to display a target operation interface of the target application; In response to an application operation performed on the target operation interface, determining an application event triggered by the application operation, and processing the application event so that the target application responds to the application event to update the display data; Among them, the application events include first-type events or second-type events. The first-type events can be responded to by the target application after being processed based on the public services provided by the first device, and the second-type events can be responded to by the target application after being processed based on the private services of the corresponding target application provided by the first device.

Images