Terminal application control method, terminal device and chip system

Abstract

The present application is applicable to the field of terminal technology, and provides a terminal application control method, a terminal device, and a chip system. The above method includes: the first terminal device sets a label for the page element in the target application interface, and collects the content of the page element to obtain the page layout information; the first terminal device obtains the type and sharing authority of the second terminal device; the server determines the target page template according to the type of the second terminal device, renders the page corresponding to the target page template based on the page layout information, and generates an address identifier of the rendered page; the second terminal device opens the address identifier through the browser, and displays the target page adapted to the browser interface, and the target page contains the target application interface. The above method can display the interface of the target application in the second terminal device that supports the browser. The second terminal device may not support the target application, and can display the interface of the target application by supporting the browser.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method for controlling terminal applications, a terminal device, and a chip system. Background Art

[0002] Currently, there are two solutions for collaboratively controlling the APP (Application) on the mobile source side on multiple clients: multi-client APP collaborative control and screen projection collaborative control. Among them, multi-client APP collaborative control refers to installing the same APP as the mobile source APP on multiple clients, and opening the client's APP page by sharing links and copying text. Screen projection collaborative control refers to projecting the application on the mobile source side to the client through screen projection technology, operating the application on the client, and the mobile source side will also execute the corresponding instructions.

[0003] However, the above method of controlling the APP on the mobile phone source side has high requirements on the client, and multiple clients need to download and install the same APP adapted to the screen size of the client. It is impossible to control the APP on the mobile phone source side on a client that the APP does not support. Summary of the invention

[0004] The present application provides a terminal application control method, a terminal device and a chip system, which can display the interface of a target application in a terminal device that supports a browser. The terminal device may not support the target application, but can display the interface of the target application as long as it supports a browser.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a terminal application control method, including: a first terminal device sets a first tag for a page element in a target application interface, and collects the content of the page element to obtain page layout information, wherein the page layout information includes the first tag and the content of the page element; the first terminal device obtains the type of the second terminal device, and sends the page layout information and the type of the second terminal device to a server; the server determines a target page template according to the type of the second terminal device, renders a page corresponding to the target page template based on the page layout information, and generates an address identifier of the rendered page; the first terminal device obtains the address identifier, and sends the address identifier to the second terminal device; the second terminal device opens the address identifier through a browser, and displays a target page adapted to the interface of the browser, wherein the target page includes the target application interface.

[0007] In an embodiment of the present application, a tag is set for the page element in the target application interface, and the content of the page element is collected to obtain page layout information. After that, the server determines the target page template according to the type of the second terminal device, renders the page corresponding to the target page template based on the page layout information, and generates an address identifier of the rendered page. The first terminal device sends the address identifier to the second terminal device. The second terminal device opens the address identifier through a browser and displays a target page adapted to the browser interface, and the target page contains the target application interface. In an embodiment of the present application, the interface of the target application can be displayed in a second terminal device that supports a browser, so that the interface of the target application in the first terminal device is shared to the second terminal device for display. Moreover, the second terminal device may not support the target application, and can display the interface of the target application as long as it supports a browser.

[0008] The server may be a server for building an interface as described later.

[0009] In combination with the first aspect, in some embodiments, the method also includes: the second terminal device listens to operation events acting on the target page through a browser, and sends the operation events to the server, the operation events including data, type, operation mode and window title of the operation events; the server sends the operation event to the first terminal device; the first terminal device updates the page elements of the target application according to the operation events.

[0010] In the embodiment of the present application, if the user performs a preset operation on the browser interface of the second terminal device, the second terminal device can monitor the operation event through the browser and send the operation event to the server, and then the server sends the operation event to the first terminal device. The first terminal device can update the page elements of the target application according to the operation event, so that the second terminal device can achieve reverse control of the target application of the first terminal device.

[0011] Exemplarily, when the second terminal device receives an operation on the browser interface (e.g., a click event of a control), the browser monitors the operation event, and the second terminal device sends the operation event and the label of the operation event to the server, and the server sends the operation event and the label of the operation event to the first terminal device. The first terminal device determines the control corresponding to the operation event in the target application according to the label of the operation event, and executes the operation event according to the determined control. The operation event may include the data, type, operation mode, and window title of the operation event.

[0012] Exemplarily, when the interface of the target application of the first terminal device changes, the first terminal device sends the page layout change information of the target application to the server, and the server determines the page change information of the target page template according to the page layout change information of the target application. The server sends the page change information to the second terminal device, and the second terminal device synchronously refreshes the page change information to the loaded page through the browser.

[0013] In a second aspect, an embodiment of the present application provides a terminal application control method, which is applicable to a first terminal device, and the method includes: setting a first tag for a page element in a target application interface, and collecting the content of the page element to obtain page layout information, wherein the page layout information includes the first tag and the content of the page element; obtaining the type of the second terminal device, and sending the page layout information and the type of the second terminal device to a server; receiving the address identifier sent by the server, wherein the page address identifier is obtained by the server based on a rendered page, wherein the rendered page is obtained by the server determining a target page template according to the type of the second terminal device, and rendering the target page template according to the page layout information; sending the address identifier to the second terminal device so that the second terminal device opens the address identifier through a browser and displays a target page adapted to the browser interface, wherein the target page includes the target application interface.

[0014] The above-mentioned terminal application control method sets tags for page elements in the target application interface, collects the content of the page elements, and obtains page layout information. Afterwards, the server determines the target page template according to the type of the second terminal device, renders the page corresponding to the target page template based on the page layout information, and generates an address identifier of the rendered page. The first terminal device sends the address identifier to the second terminal device. The second terminal device opens the address identifier through a browser and displays a target page adapted to the browser interface, which includes the target application interface. The embodiment of the present application can display the interface of the target application in a second terminal device that supports a browser, and realizes sharing the interface of the target application in the first terminal device to the second terminal device for display. Moreover, the second terminal device may not support the target application, but can display the interface of the target application by supporting a browser. The second terminal device can adaptively display the interface of the target application according to the browser interface.

[0015] Among them, the page elements can be controls in the target application interface. If labels are set for the above controls, the text content or image content of the above controls needs to be collected. For example, the target application interface includes controls such as "OK", "Next", and "Cancel". If corresponding labels are set for the above controls, the text content of each control, such as "OK", "Next", and "Cancel", is collected. The above-mentioned page elements with labels and the content of the page elements are recorded as page layout information.

[0016] In some embodiments, the types of the second terminal device may include PC, tablet, mobile phone, watch and other devices.

[0017] In combination with the second aspect, in some embodiments, the above method also includes: receiving operation event data sent by the server, the operation event data including the data, type, operation mode and window title of the operation event, the operation event data being the operation event received by the server and acting on the target page; generating control instructions based on the operation event data; executing the control instructions to update the page elements of the target application.

[0018] Specifically, the operation event data is an operation event sent by the second terminal device to the server, and the operation event is an operation event acting on the target page monitored by the second terminal device through a browser.

[0019] In the embodiment of the present application, if the user performs a preset operation on the browser interface of the second terminal device, the browser can monitor the operation event, and the second terminal device sends the operation event to the server, and then the server sends the operation event to the first terminal device. The first terminal device can update the page elements of the target application according to the operation event, thereby enabling the second terminal device to reversely control the target application of the first terminal device.

[0020] In combination with the second aspect, in some embodiments, the second terminal device includes multiple types of second terminal devices, the address identifier is multiple, and each page address corresponds to a type of second terminal device; sending the address identifier to the second terminal device includes: adding a unique identifier to each of the address identifiers; and sending the added unique address identifier to the corresponding second terminal device.

[0021] In the case of multiple second terminal devices, the embodiment of the present application can render a target page template for each type of second terminal device, obtain an address identifier, and then add a unique identifier to each address identifier, so that each address identifier can only be used once. After that, the address identifier with the unique identifier is sent to the corresponding second terminal device, so that each second terminal device opens its own address identifier through the browser and displays the interface of the target application in the browser interface.

[0022] Exemplarily, the sharing authority of each second terminal device may be different, and the sharing authority may include displaying the interface of the target application, displaying the interface of the target application and controlling the target application, and controlling the target application. If the sharing authority is to display the interface of the target application, the server does not receive operation events acting on the browser interface of the second terminal device. If the sharing authority is to display the interface of the target application and control the target application, the server receives operation events acting on the browser interface of the second terminal device.

[0023] In some embodiments, the second terminal device may include a second terminal device A and a second terminal device B. The sharing permission of the second terminal device A is to display the interface of the target application, and the sharing permission of the second terminal device B is to control the target application. Then, the first terminal device may send the address identifier to the second terminal device A, and send the control information of the target application to the second terminal device B, so that the second terminal device B can control the display of the target page by the second terminal device A.

[0024] The target application may be an application capable of playing videos, and the control information of the target application may be control time information. The second terminal device B receives a touch operation on the interface of the second terminal device B, determines a target time at which the touch operation is acquired, generates a control instruction according to the touch operation and the target time, and sends the control instruction to the second terminal device A. The second terminal device A responds to the control instruction to stop playing the video or continue playing the video.

[0025] In a third aspect, an embodiment of the present application provides a terminal application control method, which is applicable to a server, and the method includes: receiving page layout information sent by a first terminal device and the type of a second terminal device, the page layout information including a first tag and the content of a page element in a target application interface in the first terminal device, the first tag being a tag set for the page element; determining a target page template according to the type of the second terminal device, and rendering the target page template according to the page layout information; after rendering the target page template, generating an address identifier of the rendered page; sending the address identifier to the first terminal device so that the first terminal device sends the address identifier to the second terminal device, and the second terminal device opens the address identifier through a browser to display a target page adapted to the interface of the browser, wherein the target page includes the target application interface.

[0026] The above-mentioned terminal control method sets tags for page elements in the target application interface, collects the content of the page elements, and obtains page layout information. Afterwards, the server determines the target page template according to the type of the second terminal device, renders the page corresponding to the target page template based on the page layout information, and generates an address identifier of the rendered page. The first terminal device sends the address identifier to the second terminal device. The second terminal device opens the address identifier through a browser and displays a target page adapted to the browser interface, which includes the target application interface. The embodiment of the present application can display the interface of the target application in a second terminal device that supports a browser, and realizes sharing the interface of the target application in the first terminal device to the second terminal device for display. Moreover, the second terminal device may not support the target application, but can display the interface of the target application by supporting a browser. The second terminal device can adaptively display the interface of the target application according to the browser interface.

[0027] In combination with the third aspect, in some embodiments, the page element is the first control in the target application interface, and one first control corresponds to one first label; there are multiple types of the second terminal device, each type corresponds to at least one target page template, each of the target page template includes multiple second controls, and each second control corresponds to a second label; rendering the target page template according to the page layout information includes: determining the second control corresponding to each first control according to a preset label correspondence relationship between the first label and the second label; associating the content of the first control with the corresponding second control, and rendering the target page template.

[0028] Exemplarily, different types of second terminal devices correspond to different target page templates. For example, the types of second terminal devices may include PC, tablet, mobile phone, watch and other devices. Among them, PC corresponds to target page template 1, tablet corresponds to target page template 2, mobile phone corresponds to target page template 3, and watch corresponds to target page template 4.

[0029] In one scenario, the page corresponding to the target page template 1 is page 1. There may be multiple second controls in page 1, and each second control corresponds to a second label. The correspondence between the first label and the second label can be preset. The first label includes the first labels corresponding to N first controls, and the second label includes the second labels corresponding to M second controls. The label correspondence between each first label and each second label can be preset according to the correspondence between each first control and each second control. For example, the first label corresponding to the ImageView control corresponds to the second label corresponding to the img control; the first label corresponding to the TextView control corresponds to the second label corresponding to the p control; the first label corresponding to the Button control corresponds to the second label corresponding to the Input:button control.

[0030] In combination with the third aspect, in some embodiments, the above method also includes: receiving an operation event acting on the browser interface of the second terminal device, the operation event including the data, type, operation mode and window title of the operation event; sending the operation event to the first terminal device so that the first terminal device updates the page elements of the target application according to the operation event.

[0031] For example, the operation event data is an operation event sent by the second terminal device to the server, and the operation event is an operation event acting on the browser interface monitored by the second terminal device through the browser.

[0032] In combination with the third aspect, in some embodiments, the method also includes: receiving sharing permissions of a second terminal device sent by the first terminal device; the sharing permissions include displaying the interface of the target application, and displaying the interface of the target application and controlling the target application; if the sharing permission is to display the interface of the target application, the step of receiving an operation event on the browser interface of the second terminal device is not performed; if the sharing permission is to display the interface of the target application and control the target application, the step of receiving an operation event on the browser interface of the second terminal device is performed.

[0033] In combination with the third aspect, in some embodiments, the second terminal device includes a second terminal device A and a second terminal device B, the sharing permission of the second terminal device A is to display the interface of the target application, and the sharing permission of the second terminal device B is to control the target application; sending the address identifier to the first terminal device includes: sending the address identifier and the control information of the target application to the first terminal device, so that the first terminal device sends the address identifier to the second terminal device A, and sends the control information to the second terminal device B, so that the second terminal device B can control the display of the target page by the second terminal device A.

[0034] In a fourth aspect, an embodiment of the present application provides a terminal device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, and the computer program code includes computer instructions; when the one or more processors execute the computer instructions, the terminal device executes a method as described in any one of the first aspect, or a method as described in any one of the second aspect, or a method as described in any one of the third aspect.

[0035] In a fifth aspect, an embodiment of the present application provides a server, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, and the computer program code includes computer instructions; when the one or more processors execute the computer instructions, the terminal device executes the method as described in any one of the first aspect, or the method as described in any one of the second aspect, or the method as described in any one of the third aspect.

[0036] In a sixth aspect, an embodiment of the present application provides a chip system, the chip system comprising a processor, the processor coupled to a memory, the processor executing a computer program stored in the memory to implement a method as described in any one of the first aspect, or a method as described in any one of the second aspect, or a method as described in any one of the third aspect. The chip system may be a single chip, or a chip module composed of multiple chips.

[0037] In a seventh aspect, an embodiment of the present application provides a chip system, the chip system comprising a memory and a processor, the processor executing a computer program stored in the memory to implement a method as described in any one of the first aspect, or a method as described in any one of the second aspect, or a method as described in any one of the third aspect. The chip system may be a single chip or a chip module composed of multiple chips.

[0038] In an eighth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any method in the first aspect, or any method in the second aspect, or any method in the third aspect.

[0039] In the ninth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method as described in any one of the first aspect, or the method as described in any one of the second aspect, or the method as described in any one of the third aspect is implemented.

[0040] It can be understood that the terminal device described in the fourth aspect, the server described in the fifth aspect, the chip system described in the sixth and seventh aspects, the computer program product described in the eighth aspect, and the computer-readable storage medium described in the ninth aspect are used to execute the method provided in the first aspect, or execute the method described in the second aspect, or execute the method described in the third aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a scenario used by the terminal application control method provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of a scenario provided for an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a process for sharing an interface of a target application of a first terminal device to be displayed on an interface of a second terminal device, provided in an embodiment of the present application;

[0044] Figure 4 A schematic diagram of a process in which a second terminal device performs reverse control on a target application provided in an embodiment of the present application;

[0045] Figure 5 A schematic diagram of a process flow of a terminal application control method provided in an embodiment of the present application;

[0046] Figure 6 A schematic diagram of the "Buy" APP interface in mobile phone A provided in an embodiment of the present application;

[0047] Figure 7 A schematic diagram showing a "Buy" APP interface in a tablet computer provided in an embodiment of the present application;

[0048] Figure 8 A schematic diagram showing a "Buy" APP interface in a PC provided in an embodiment of the present application;

[0049] Fig. 9 A schematic diagram showing the "Buy" APP interface in mobile phone B provided in an embodiment of the present application;

[0050] Fig.10 A schematic diagram of a scenario provided for an embodiment of the present application;

[0051] FIG. 11( a ) and FIG. 11( b ) are schematic diagrams of scenarios provided in an embodiment of the present application;

[0052] Fig.12 A schematic diagram of the hardware architecture of a terminal device provided in an embodiment of the present application;

[0053] Fig.13 A software structure block diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0060] In addition, the “plurality” mentioned in the embodiments of the present application should be interpreted as two or more.

[0061] The steps involved in the terminal application control method provided in the embodiment of the present application are only examples. Not all steps must be executed, or not all information or content in a message is mandatory. They can be increased or decreased as needed during use.

[0062] In the embodiments of the present application, the same step or steps or messages with the same function in different embodiments can be referenced to each other.

[0063] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art will appreciate that, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0064] Related technology 1 provides a method for collaborative control of multiple client APPs. For example, client A responds to a page sharing operation performed by a user in the APP, generates a sharing text and a target page corresponding to the sharing text. Client A sends the sharing text and the target page to client B through instant messaging technology. After client B obtains the sharing text, it copies the sharing text. The user can perform a sharing text paste operation in the APP of client B, and client B responds to the sharing text paste operation, detects whether the text in the sharing text paste operation is consistent with the text in the target page, and if they are consistent, opens the target page to be shared by client A.

[0065] However, related technology 1 has high requirements for the client, requiring multiple clients to download and install the same APP that adapts to the screen size of the client, and for clients that cannot install such APP, multi-client APP collaborative control cannot be achieved. Moreover, related technology 1 cannot achieve reverse control. In essence, it just opens the same page for sharing, and any operation on the page cannot be passed to other clients. Moreover, the way in which client A shares the target page with other clients in related technology 1 is relatively complicated. It is necessary to click Share on a page of client A's APP to generate a sharing text, and pass the sharing text to other clients through the instant messaging application. Other clients copy the sharing text and open the APP to jump to the target page.

[0066] Related technology 2 provides a method for collaborative control of screen projection. Client A tags the pages and instructions in the APP and sends them to client B. Client B restores the tagged instructions, converts the instructions on client A into instructions on client B, and builds the screen of client A. The user can use the mouse to click, slide, and other operations on the screen of client A built on client B to simulate the user's touch events on client A. Client B listens to mouse events and transmits the events to client A in the form of an instruction stream. Client A restores the instruction stream sent by client B and executes the restored instructions, thereby realizing the control of client A on client B.

[0067] However, in the related technology 2, a client A can only project a page to a client B, and a client B can reversely control client A, but it is impossible to realize the scenario where a client A controls multiple clients B. Moreover, the instruction stream sent by client B to client A is a touch event generated by the user's operation on client B, and the formation of a touch event requires the user to actually operate on client B. Therefore, after the user performs an operation on client B, it may take a long time for client A to respond to the operation.

[0068] Based on the above problems, an embodiment of the present application provides a terminal application control method. The first terminal device sets a label for the page elements that need to be shared in the target application, and obtains the type and sharing authority of the second terminal device, where the second terminal device is a terminal device that shares the target application. The first terminal device sends the type and page layout information of the second terminal device to the server, where the page layout information is obtained based on the page elements. The server determines the target page template according to the type of the second terminal device, renders the Html page of the target page template based on the page layout information, and then sends the URL (Uniform Resource Locator) address of the rendered target page template to the first terminal device. The first terminal device sends the URL address to the second terminal device, and the second terminal device opens the URL address through a browser or WebView, where the page controls in the browser or WebView correspond one-to-one to the controls in the target application through labels.

[0069] When the second terminal device receives an operation on the browser or WebView (such as a click event of a control), the second terminal device responds to the operation and sends the event corresponding to the operation and the event label to the first terminal device. The first terminal device determines the control corresponding to the event in the target application according to the event label, and executes the event according to the determined control.

[0070] When the page of the target application of the first terminal device changes, the first terminal device sends the page layout change information of the target application to the server, and the server re-renders the constructed Html DOM tree according to the page layout change information of the target application and generates a new URL address. The first terminal device sends the new URL address to the second terminal device. The second terminal device reloads the rendered page according to the new URL address.

[0071] Exemplarily, the first terminal device and the second terminal device in the embodiments of the present application can be personal computers (PCs), mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other terminal devices. The embodiments of the present application do not impose any restrictions on the specific types of terminal devices.

[0072] Figure 1A schematic diagram of a scenario in which the terminal application control method provided in the embodiment of the present application is applicable. Figure 1 , the scenario includes a first terminal device, a second terminal device and a server.

[0073] The target APP is installed on the first terminal device, and the target APP includes an interface UI. The first terminal device is also provided with a Dues server, which includes a content acquisition module, a content scheduling module and a content preprocessing module. The content acquisition module is used to set labels for page elements in the target APP, such as setting labels for page elements that need to be shared and page elements for all event executions. After the content acquisition module sets labels for the page elements that need to be shared by the target APP, multiple marked page elements are obtained. The content scheduling module is used to schedule multiple marked page elements to the content preprocessing module. The content preprocessing module packages multiple marked page elements in a specific format so that multiple marked page elements can be passed to subsequent modules in a fixed data structure. The content preprocessing module may include Rview content processing, converting Rview into JSON language, and Light content processing, and updating the behavior list according to the results of the Light content processing.

[0074] The Dues server is also provided with a data conversion module and a Web server. The Web server is used to provide users with the functions of selecting shared terminal devices and setting permissions for shared terminal devices. Among them, shared terminal devices may include: PCs, tablets, mobile phones, watches and other devices. The first four types of terminal devices are each provided with a specific Html template, and the above Html template can set the UI corresponding to the terminal device. When the user selects other devices, the interface of the target application in the first terminal device can be proportionally scaled and adapted according to the browser resolution of the other devices.

[0075] The permissions of shared terminal devices include: sharing only, control only, and sharing and control. Among them, the permission of the shared terminal device is sharing only, which means that the current terminal device can only browse the pages shared by the target application in the first terminal device, and cannot reverse control the target application. This permission is usually used for display devices with large screens. The permission of the shared terminal device is control only, which means that the current terminal device can only be used for some simple control functions, such as being used as a remote control, and cannot display the pages shared by the target application. This permission is usually used for lightweight devices such as watches. Sharing and control is a combination of the above two permissions, which means that the page shared by the target application can be displayed, and the target application can also be reversed. This permission is usually used for mobile phones and tablet devices.

[0076] The data conversion module is used to convert the data output by the content preprocessing module and send the converted data to the Web server. For example, the data output by the content preprocessing module is a language describing the UI resource tree ViewTree. The data conversion module converts the language describing the UI resource tree ViewTree into the JSON language that the interface construction server can understand and process.

[0077] The Web server processes the converted data according to the selected shared terminal device (i.e., the second terminal device) to obtain drawing data with a preset data structure. For example, the Web server adds the type, permission information, and other related field information of the second terminal device to the converted data to obtain drawing data with a preset data structure. The Web server sends the selected shared terminal device, device permissions, and drawing data to the interface construction server via Socket. The drawing data may include tag information, type information, and content. For example, the data structure of the drawing data may be as shown in the following code:

[0078]

[0079]

[0080] The interface construction server includes a data parsing module, a layout drawing module, a page control drawing module, an interface synthesis module and a page rendering module.

[0081] The data analysis module is used to analyze the drawing data to obtain layout information and page control information.

[0082] The layout drawing module is used to adopt a unified flex layout as the CSS (Cascading Style Sheets) style of the Html page layout, and to set corresponding interface layout parameters according to the layout information (eg including layout container attributes) obtained by parsing.

[0083] The page control drawing module is used to determine the control correspondence between the label of the control in the target application and the label of the control in Html. For example, if the first terminal device is a mobile phone, the page control drawing module determines that the label of the ImageView control in the target application corresponds to the label of the img control in Html, the label of the TextView control in the target application corresponds to the label of the p control in Html, the label of the Button control in the target application corresponds to the label of the Input:button control in Html, etc.

[0084] The interface synthesis module is used to associate and synthesize the interface layout parameters with the corresponding relationships of the controls.

[0085] The page rendering module is used to determine the corresponding Html page according to the shared terminal device and device permissions, and render the Html page based on the corresponding relationship between the interface layout parameters and the controls, such as rendering the Html control tags generated after parsing and the corresponding events in the Html control tags.

[0086] The interface construction server is used to send the URL address of the rendered Html page to the first terminal device.

[0087] In addition, the interface construction server also includes a Socket monitoring module. After being activated, the Socket monitoring module is used to monitor new instructions and layout data sent by the first terminal device.

[0088] After the first terminal device obtains the URL address of the rendered Html page, it encrypts the URL address. For example, a one-way salt value can be used to encrypt the URL address. In addition, the first terminal device generates a unique suffix for the URL address through a distributed ID, and the unique suffix is ​​used to ensure that each URL address can only be used once. If there are multiple second terminal devices, after the first terminal device obtains the URL address of the rendered Html page, it generates multiple suffixes for the URL address through a distributed ID, and URL addresses with different suffixes are sent to different second terminal devices. The second terminal device opens the sharing page according to the received URL address, and the web browser of the second terminal device sends a get request to obtain the corresponding Html page, completing the mapping of the source Activity.

[0089] The reverse control process of the second terminal device on the first terminal device is as follows: the browser page of the second terminal device receives the interactive event and transmits it to the interface construction server through WebSocket. The Socket monitoring module of the interface construction server monitors all Html tags with interactive event tags. When the user performs interactive operations on the Web browser of the second terminal device, the user's interactive events are serialized into JSON data. The JSON data includes data changes, event types, event operation methods, and windowTitle corresponding to the first terminal device. The data structure of the data is shown in the following code:

[0090]

[0091]

[0092] For example, the interface building server transmits JSON data to the behavior receiving module of the Dues server through Socket, and the behavior receiving module sends the JSON data to the behavior scheduling module. The behavior scheduling module parses and converts the JSON data, repackages it into the instruction sequence required for the target APP process to execute, and injects it into the target APP process. The target APP process executes the instruction operation according to the page element root and tag to complete the reverse control of the target APP.

[0093] See also Figure 2 , the sending end can be a first terminal device, and the receiving end can be an interface construction server. During the life cycle of the page element, the sending end marks the page element and sets a tag. After that, the sending end collects the content of each page element, packages the collected content of each page element, and sends the packaged content of each page element to the receiving end. The receiving end parses the received data to obtain the content of each page element, and updates the content of the corresponding page element according to the tag.

[0094] See also Figure 3 , the process of sharing the interface of the target application of the first terminal device to the interface of the second terminal device for display may include: after starting to collaboratively control the target APP in the first terminal device, the Dues server sets tags for the page elements that need to be shared in the target APP and the page elements of all event executions, obtains multiple marked page elements, and packages the multiple marked page elements to obtain the UI data of the target APP. The Dues server sends the UI data of the target APP to the interface construction server. The interface construction server draws an Html page based on the UI data of the target APP and generates the URL address of the Html page. The process of how to draw an Html page based on the UI data of the target APP is not repeated here. The target APP sends the URL address of the acquired Html page to the second terminal device. The second terminal device opens the URL address through a web browser, and displays the interface of the target APP that matches the second terminal device based on the correspondence between the controls of the Html page and the controls of the first terminal device.

[0095] After the interface of the target APP of the first terminal device changes, the Dues server obtains the change information of the target APP interface and sends the change information of the target APP interface to the interface construction server. The interface construction server updates the Html page according to the change information of the target APP interface. After that, the second terminal device updates the interface displayed in the web browser according to the updated Html page.

[0096] See also Figure 4, the process of the second terminal device counter-controlling the target APP may include: when the second terminal device receives an operation acting on the browser interface (such as a click event of a control), the browser monitors the operation event, and the second terminal device sends the operation event and the label of the operation event to the interface construction server, and the interface construction server receives the operation event acting on the Web browser interface of the second terminal device and obtains an operation event sequence. The interface construction server converts the operation event sequence into JSON data, and the JSON data includes data changes, event types, event operation methods, window title WindowTitle of the first terminal device, etc. The interface construction server sends the JSON data to the Dues server, and the Dues server parses and converts the JSON data, repackages it into the instruction sequence required for the execution of the target APP process, and injects the instruction sequence into the process of the target APP. For example, the interface construction server can send the JSON data to the behavior receiving module of the Dues server, and the behavior receiving module sends the JSON data to the behavior scheduling module. The behavior scheduling module parses and converts the JSON data, repackages it into the instruction sequence required for the execution of the target APP process, and injects the instruction sequence into the process of the target APP. The process of the target APP executes the instruction operation according to the page element root and tag, and updates the page elements of the target APP, thereby realizing the reverse control of the target APP.

[0097] Figure 5 This is a flow chart of the terminal application control method provided in the embodiment of the present application. Figure 5 , the terminal application control method may include the following steps:

[0098] Step 101: The first terminal device sets a first tag for a page element that needs to be shared in a target application interface, and collects content of the page element to obtain page layout information.

[0099] The page elements may be controls in the target application interface. If labels are set for the controls, the text content or image content of the controls needs to be collected. For example, the target application interface includes controls such as "OK", "Next", and "Cancel". If corresponding labels are set for the controls, the text content of each control such as "OK", "Next", and "Cancel" is collected. The page layout includes a first label, a page element with a label, and the content of the page element.

[0100] Step 102: The first terminal device obtains the type and sharing authority of the second terminal device.

[0101] Among them, the types of the second terminal device may include PC, tablet, mobile phone, watch and other devices.

[0102] The permissions of the second terminal device may include: displaying the target application interface, controlling the target application, and displaying the target application and the interface and controlling the target application. Among them, the permission of the second terminal device is to display the target application interface, which means that the second terminal device can only display the page shared by the target application, and cannot reverse control the target application. This permission can usually be used for display devices with large screens. The permission of the shared terminal device is to control the target application, which means that the second terminal device can only be used for some simple control functions, such as being used as a remote control, and cannot display the page shared by the target application. This permission can usually be used for lightweight devices such as watches. The permission of the second terminal device is to display the target application and the interface and control the target application, which means that the second terminal device can both display the page shared by the target application and reverse control the target application. This permission can usually be used for devices such as mobile phones and tablet devices.

[0103] Step 103: The first terminal device sends the type of the second terminal device and page layout information to the interface construction server.

[0104] Among them, the interface construction server may be provided with a plurality of page templates. Each page template corresponds to a type of the second terminal device, and each page template has a corresponding UI. For example, the page template may be an Html template.

[0105] For example, if the type of the second terminal device is PC, the Html template has a UI corresponding to the PC; if the type of the second terminal device is a tablet, the Html template has a UI corresponding to the tablet; if the type of the second terminal device is a mobile phone, the Html template has a UI corresponding to the mobile phone; if the type of the second terminal device is a watch, the Html template has a UI corresponding to the watch.

[0106] If the type of the second terminal device is other devices, the interface of the target application in the first terminal device can be proportionally scaled and adapted according to the browser resolution of the other devices, without setting an Html template corresponding to the other devices.

[0107] Step 104: the interface construction server determines a target page template according to the type of the second terminal device, renders a page corresponding to the target page template based on the page layout information, and obtains a URL address of the rendered page.

[0108] Among them, a plurality of second controls may be set in the page corresponding to the target page template, each second control corresponds to a second label, and the position of each second control on the page may be pre-set. For example, the page module may be a JS template, and different page modules may correspond to different second controls.

[0109] Exemplarily, the interface construction server renders the page of the target page template based on the page layout information, which may include: obtaining the first tag of the first control of the target APP in the page layout information, where the first control is any control of the target APP; determining the second tag corresponding to the first tag; determining the second control corresponding to the determined second tag in the page corresponding to the target page template; associating the content of the first control with the second control, and rendering the target page template.

[0110] In this embodiment, different types of second terminal devices correspond to different target page templates. For example, the types of second terminal devices may include PC, tablet, mobile phone, watch and other devices. Among them, PC corresponds to target page template 1, tablet corresponds to target page template 2, mobile phone corresponds to target page template 3, and watch corresponds to target page template 4.

[0111] In one scenario, the page corresponding to the target page template 1 is page 1. There may be multiple second controls in page 1, and each second control corresponds to a second label. The correspondence between the first label and the second label can be preset. The first label includes the first labels corresponding to N first controls, and the second label includes the second labels corresponding to M second controls. The label correspondence between each first label and each second label can be preset according to the correspondence between each first control and each second control, and then the label correspondence between each first label and each second label can be directly used in step 104. For example, the first label corresponding to the ImageView control corresponds to the second label corresponding to the img control; the first label corresponding to the TextView control corresponds to the second label corresponding to the p control; the first label corresponding to the Button control corresponds to the second label corresponding to the Input:button control.

[0112] For target page templates 2 to 4, how to establish the above-mentioned label correspondence relationship, please refer to the scenario of target page template 1, which will not be repeated here.

[0113] After the target page template is rendered, a rendered page is obtained, which is the interface of the target APP in the first terminal device. Afterwards, the interface construction server generates a URL address of the rendered page and sends the URL address to the first terminal device.

[0114] Step 105: The first terminal device sends the URL address to the second terminal device.

[0115] In some embodiments, the first terminal device may encrypt the URL address. For example, the first terminal device may encrypt the URL address using a one-way salt value encryption method.

[0116] In some embodiments, the first terminal device may generate a unique suffix for the URL address and add the unique suffix to the end of the URL address, so that the URL address can only be used once by the second terminal device.

[0117] In the case where there are multiple second terminal devices, each second terminal device corresponds to a URL address, and the first terminal device can generate a unique suffix for each URL address and add the unique suffix to the end of the corresponding URL address.

[0118] Step 106: The second terminal device opens the URL address and displays a target page adapted to the browser interface, wherein the target page includes a target application interface.

[0119] The second terminal device can open the URL address through a web browser or WebView, and display a page containing the target APP interface in the web browser window. Moreover, the page can be adaptively adjusted according to the browser window.

[0120] The above-mentioned terminal application control method sets tags for page elements in the target application interface, collects the content of the page elements, and obtains page layout information. Afterwards, the server determines the target page template according to the type of the second terminal device, renders the page corresponding to the target page template based on the page layout information, and generates an address identifier of the rendered page. The first terminal device sends the address identifier to the second terminal device. The second terminal device opens the address identifier through a browser and displays a target page adapted to the browser interface, which includes the target application interface. The embodiment of the present application can display the interface of the target application in a second terminal device that supports a browser, and realizes sharing the interface of the target application in the first terminal device to the second terminal device for display. Moreover, the second terminal device may not support the target application, but can display the interface of the target application by supporting a browser. The second terminal device can adaptively display the interface of the target application according to the browser interface.

[0121] Embodiment 1

[0122] The following example describes the embodiment of the present application, taking the target application as a "buy" APP, the first terminal device as a mobile phone A, and the second terminal device as a PC, a tablet, and a mobile phone B. The second terminal device may be one or more, and the types of the multiple second terminal devices may be different or the same.

[0123] Figure 6 This is a diagram of the "Buy" APP interface in mobile phone A. Figure 6The "Buy" APP interface includes multiple first controls, such as first controls 11 to 17. Among them, the first control 11 can be an identification control of the "Buy" APP, and the control content can be a picture containing the word "Buy". The first control 12 can be a search control for entering search content. The first control 13 can be a category control for classifying goods, for example, it can include categories such as "daily necessities", "electronic products", and "food". The first control 14, the first control 15, and the first control 16 can be display controls for displaying goods. The first control 17 can be a display control for displaying goods, and there can be multiple first controls 17. Each first control from the first control 11 to the first control 17 corresponds to a first label.

[0124] Figure 7 This is a diagram showing the "Buy" APP interface on a tablet. Figure 7 , the "Buy" APP interface displayed by the tablet computer through the browser includes multiple second controls, such as second controls 21 to 27. Among them, the second control 21 can be an identification control of the "Buy" APP, and the content of the control can be a picture containing the word "Buy". The second control 22 can be a search control for entering search content. The second control 23 can be a category control for classifying goods, for example, it can include categories such as "daily necessities", "electronic products", and "food". The second control 24, the second control 25, and the second control 26 can be display controls for displaying goods. The second control 27 can be a display control for displaying goods, and there can be multiple second controls 27. Each second control in the second control 21 to the second control 27 corresponds to a second label. There can be a preset label correspondence relationship between the first label and the second label.

[0125] The process of displaying the interface of the "Buy" APP in mobile phone A to the tablet computer may include: setting a first tag for each first control in the first control 11 to the first control 17 in the "Buy" APP interface in mobile phone A, collecting the content of each first control, obtaining the UI data of the "Buy" APP in mobile phone A, and sending the UI data and the type of the second terminal device as a tablet computer to the interface construction server. The interface construction server determines that the target page template is the target page template corresponding to the tablet computer according to the type of the second terminal device. The target page template includes multiple second controls, each second control corresponds to a second tag, and the position of each second tag in the target page template can be pre-set. The interface construction server associates the content of the first control corresponding to the first tag with the corresponding second control according to the tag correspondence between the first tag and the second tag, and renders the target page template to obtain the target page. After that, the interface construction server sends the URL address of the target page to mobile phone A, and mobile phone A encrypts the URL address and adds a unique suffix to the URL address, and then sends the URL address to the tablet computer. The tablet computer opens the URL address through a web browser to display the interface of the "Buy" APP.

[0126] The user operates the "Buy" APP in the tablet computer, and the process in which the tablet computer reversely controls the "Buy" APP in mobile phone A may include: the user applies a preset operation to a certain commodity in the browser interface of the tablet computer, the browser of the tablet computer listens to the preset operation event, and sends the preset operation event to the interface construction server, and the interface construction server converts the preset operation event into JSON data, and the JSON data includes data changes, event types, event operation methods, WindowTitle in the mobile phone, etc. The interface construction server sends the JSON data to the Dues server of the mobile phone, and the Dues server parses and converts the JSON data, repackages it into the instruction sequence required for the execution of the "Buy" APP process, and enters the instruction sequence into the "Buy" APP process. The "Buy" APP process executes the instruction operation according to the page element root and tag, and updates the page elements of the "Buy" APP, thereby realizing reverse control of the "Buy" APP.

[0127] In some embodiments, the aspect ratio of the target page template corresponding to the tablet computer is significantly different from that of the interface of the "Buy" APP of mobile phone A, so the arrangement of the first controls 14, 15 and 17 in the interface of the "Buy" APP of mobile phone A is different from the arrangement of the second controls 24, 25 and 27 in the target page template. Figure 6 and Figure 7As shown, the first controls 14 and 15 are arranged vertically, and the second controls 24 and 25 are arranged horizontally side by side. The first controls 17 are arranged in multiple rows, with two first controls 17 in each row, and the second controls 27 are arranged in multiple rows, with five second controls 27 in each row.

[0128] In one scenario, mobile phone A can share the "Buy" APP interface with multiple second terminal devices, such as a tablet, PC, and mobile phone B. For example, when mobile phone A shares the currently browsed clothes with multiple second terminal devices, when user A applies operations such as sliding or clicking the page in mobile phone A, the browser interface of each second terminal device will also change accordingly. Similarly, user B can also apply operations such as sliding or clicking the current page in the browser interface of the second terminal device to tell user A that this piece of clothing is more beautiful, and the "Buy" APP interface in mobile phone A will also change accordingly.

[0129] Figure 8 This is a diagram showing the "Buy" APP interface on a PC. Figure 8 , the "Buy" APP interface displayed by the PC through the browser includes multiple second controls, such as second controls 31 to 37. Among them, the second control 31 can be an identification control of the "Buy" APP, and the control content can be a picture containing the word "Buy". The second control 32 can be a search control for entering search content. The second control 33 can be a category control for classifying goods, for example, it can include categories such as "daily necessities", "electronic products", and "food". The second control 34, the second control 35, and the second control 36 can be display controls for displaying goods. The second control 37 can be a display control for displaying goods, and there can be multiple second controls 37. Each second control in the second control 31 to the second control 37 corresponds to a second label.

[0130] Fig. 9 This is a diagram showing the "Buy" APP interface on mobile phone B. Fig. 9 , the "Buy" APP interface displayed by mobile phone B through the browser includes multiple second controls, such as second controls 41 to 47. Among them, the second control 41 can be an identification control of the "Buy" APP, and the content of the control can be a picture containing the word "Buy". The second control 42 can be a search control for entering search content. The second control 43 can be a category control for classifying goods, for example, it can include categories such as "daily necessities", "electronic products", and "food". The second control 44, the second control 45, and the second control 46 can be display controls for displaying goods. The second control 47 can be a display control for displaying goods, and there can be multiple second controls 47. Each second control in the second control 41 to the second control 47 corresponds to a second label.

[0131] Embodiment 2

[0132] The following describes an embodiment of the present application by taking the target application as a video application, the first terminal device as a mobile phone, and the second terminal device as a large-screen device (such as a tablet or TV) and a lightweight device (such as a watch) as an example.

[0133] See also Fig.10 When playing video A through the video application of the mobile phone, the mobile phone sends the video information to the tablet, and the tablet displays the video. For the specific process of the mobile phone sending the video information to the tablet and the tablet displaying the video through the browser, please refer to Example 1, which will not be repeated here. The mobile phone sends the control time corresponding to the video to the watch, and the watch controls the video played on the tablet in response to the user operation acting on the watch interface, such as pausing the playback, continuing the playback, etc.

[0134] In this embodiment, the first terminal device and the second terminal device do not need to be in the same physical space or in the same local area network.

[0135] Embodiment 3

[0136] The following is an example of describing the embodiment of the present application, taking the target application as a collaborative game application, the first terminal device as a mobile phone, and the second terminal device as a touch screen device (such as a tablet), wherein the second terminal device does not support the collaborative game application.

[0137] Referring to FIG. 11( a ), when user A plays a game through a collaborative game application on a mobile phone, the mobile phone sends the game screen to the tablet, and the tablet displays the video through a browser. For the specific process of the mobile phone sending the game screen to the tablet and the tablet displaying the video through the browser, please refer to Example 1, which will not be described in detail here.

[0138] Referring to Figure 11(b), user B performs an operation in the browser interface of the tablet, and the tablet sends the operation to the mobile phone, which displays the operation in the collaborative game application. The specific process of the tablet sending the operation to the mobile phone and the mobile phone displaying the operation in the collaborative game application is shown in Example 1, which will not be described in detail here.

[0139] In this embodiment, for a second terminal device that does not support the target application, the first terminal device can share the application interface with the second terminal device, and the second terminal device can control the target application in the first terminal device.

[0140] For example, Fig.12 1 is a schematic diagram of the hardware architecture of the terminal device 100 provided in an embodiment of the present application. The terminal device 100 may be the first terminal device mentioned above, or may be the second terminal device mentioned above.

[0141] The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0142] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0143] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0144] The controller may be the nerve center and command center of the terminal device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0145] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0146] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0147] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0148] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0149] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0150] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0151] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0152] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0153] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal device 100, and can also be used to transmit data between the terminal device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other terminal devices, such as AR devices, etc.

[0154] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0155] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the terminal device 100. While the charging management module 140 is charging the battery 142, it may also power the terminal device through the power management module 141.

[0156] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0157] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0158] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0159] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0160] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0161] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0162] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite based augmentation system (SBAS).

[0163] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0164] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0165] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0166] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0167] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0168] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0169] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0170] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0171] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0172] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0173] The terminal device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0174] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0175] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0176] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the ear.

[0177] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The terminal device 100 can be provided with at least one microphone 170C. In other embodiments, the terminal device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 100 can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, realization of directional recording function, etc.

[0178] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0179] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal device 100 detects the touch operation intensity according to the pressure sensor 180A. The terminal device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0180] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0181] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0182] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0183] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in various directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal device and applied to applications such as horizontal and vertical screen switching and pedometers.

[0184] The distance sensor 180F is used to measure the distance. The terminal device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0185] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light outward through the light emitting diode. The terminal device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can use the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and pocket mode automatically unlocks and locks the screen.

[0186] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket to prevent accidental touch.

[0187] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0188] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 heats the battery 142 to avoid abnormal shutdown of the terminal device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0189] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the terminal device 100, which is different from the position of the display screen 194.

[0190] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0191] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100.

[0192] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0193] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0194] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the terminal device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The terminal device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0195] Fig.13 It is a software structure block diagram of the terminal device 100 according to an embodiment of the present application.

[0196] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0197] The application layer can include a series of application packages.

[0198] like Fig.13 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0199] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0200] like Fig.13 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0201] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0202] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0203] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0204] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including connection, disconnection, etc.).

[0205] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0206] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0207] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0208] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0209] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0210] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0211] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0212] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0213] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0214] A 2D graphics engine is a drawing engine for 2D drawings.

[0215] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0216] Optionally, an embodiment of the present application further provides a terminal device, comprising: one or more processors and a memory. The memory is coupled to the one or more processors, and the memory is used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the terminal device performs one or more steps in any of the above methods.

[0217] Optionally, an embodiment of the present application further provides a computer-readable storage medium, which stores instructions, and when the computer-readable storage medium is executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods.

[0218] Optionally, an embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer or a processor, enables the computer or the processor to execute one or more steps in any of the above methods.

[0219] Optionally, an embodiment of the present application further provides a chip system, which may include a memory and a processor, and the processor executes a computer program stored in the memory to implement one or more steps in any of the above methods. The chip system may be a single chip or a chip module composed of multiple chips.

[0220] Optionally, an embodiment of the present application further provides a chip system, which may include a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement one or more steps in any of the above methods. The chip system may be a single chip or a chip module composed of multiple chips.

[0221] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0222] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0223] Finally, it should be noted that the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A terminal application control method, characterized in that: The method comprises: The first terminal device sets a first tag for a page element in the target application interface, and collects content of the page element to obtain page layout information, where the page layout information includes the first tag and content of the page element; The first terminal device obtains the type and sharing authority of the second terminal device, and sends the page layout information, the type and sharing authority of the second terminal device to the server; The server determines a target page template according to the type of the second terminal device, renders a page corresponding to the target page template based on the page layout information, and generates an address identifier of the rendered page; The first terminal device acquires the address identifier and sends the address identifier to the second terminal device; The second terminal device opens the address identifier through a browser and displays a target page adapted to the interface of the browser, wherein the target page includes the target application interface.

2. The method according to claim 1, characterized in that The method further comprises: The second terminal device monitors the operation event acting on the target page through the browser, and sends the operation event to the server, wherein the operation event includes data, type, operation mode and window title of the operation event; The server sends the operation event to the first terminal device; The first terminal device updates the page element of the target application according to the operation event.

3. A terminal application control method, characterized in that: Applicable to a first terminal device, the method includes: Setting a first tag for a page element in a target application interface, and collecting content of the page element to obtain page layout information, wherein the page layout information includes the first tag and content of the page element; Acquire the type of the second terminal device, and send the page layout information and the type of the second terminal device to the server; receiving an address identifier sent by a server, the address identifier being obtained by the server according to a rendered page, the rendered page being obtained by the server determining a target page template according to a type of the second terminal device and rendering the target page template according to the page layout information; The address identifier is sent to the second terminal device, so that the second terminal device opens the address identifier through a browser and displays a target page adapted to the interface of the browser, wherein the target page includes the target application interface.

4. The method according to claim 3, characterized in that The method comprises: Receiving an operation event sent by the server, wherein the operation event includes data, type, operation mode and window title of the operation event, and the operation event is an operation event received by the server and acts on the target page; generating a control instruction according to the operation event; The control instruction is executed to update the page elements of the target application.

5. The method according to claim 3, characterized in that: The second terminal device includes a plurality of types of second terminal devices, the address identifier is a plurality of, and each page address corresponds to a type of second terminal device; The sending the address identifier to the second terminal device includes: Adding a unique identifier to each of the address identifiers; The added unique address identifier is sent to the corresponding second terminal device.

6. The method according to claim 3, characterized in that The second terminal device includes a second terminal device A and a second terminal device B, the sharing permission of the second terminal device A is to display the interface of the target application, and the sharing permission of the second terminal device B is to control the target application; The method further comprises: The address identifier is sent to the second terminal device A, and the control information of the target application is sent to the second terminal device B, so that the second terminal device B can control the second terminal device A to display the target page.

7. A terminal application control method, characterized in that: Applicable to a server, the method comprises: Receiving page layout information sent by a first terminal device and a type of a second terminal device, the page layout information including a first tag and content of a page element in a target application interface in the first terminal device, the first tag being a tag set for the page element; determining a target page template according to the type of the second terminal device, and rendering the target page template according to the page layout information; After the target page template is rendered, an address identifier of the rendered page is generated; The address identifier is sent to the first terminal device so that the first terminal device sends the address identifier to the second terminal device, and the second terminal device opens the address identifier through a browser and displays a target page adapted to the interface of the browser, wherein the target page includes the target application interface.

8. The method according to claim 7, characterized in that The page element is a first control in the target application interface, and one first control corresponds to one first label; there are multiple types of the second terminal device, each type corresponds to at least one target page template, each target page template includes multiple second controls, and each second control corresponds to a second label; The rendering of the target page template according to the page layout information includes: Determining second controls corresponding to each first control according to a preset label correspondence relationship between the first label and the second label; The content of the first control is associated with the corresponding second control, and the target page template is rendered.

9. The method according to claim 7, characterized in that: The method further comprises: Receiving an operation event acting on the browser interface of the second terminal device, wherein the operation event includes data, type, operation mode and window title of the operation event; The operation event is sent to the first terminal device, so that the first terminal device updates the page element of the target application according to the operation event.

10. The method according to claim 9, characterized in that The method further comprises: Receiving sharing permissions of a second terminal device sent by the first terminal device; the sharing permissions include displaying an interface of the target application, and displaying an interface of the target application and controlling the target application; If the sharing permission is to display the interface of the target application, the step of receiving the operation event acting on the browser interface of the second terminal device is not performed; If the sharing permission is to display the interface of the target application and control the target application, the step of receiving an operation event acting on the browser interface of the second terminal device is performed.

11. The method according to claim 7, characterized in that The second terminal device includes a second terminal device A and a second terminal device B, the sharing permission of the second terminal device A is to display the interface of the target application, and the sharing permission of the second terminal device B is to control the target application; The sending the address identifier to the first terminal device includes: The address identifier and the control information of the target application are sent to the first terminal device, so that the first terminal device sends the address identifier to the second terminal device A, and sends the control information to the second terminal device B, so that the second terminal device B can control the second terminal device A to display the target page.

12. A terminal device, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, the memory being configured to store computer program code, the computer program code comprising computer instructions; When the one or more processors execute the computer instructions, the terminal device executes the method according to any one of claims 3 to 6, or the method according to any one of claims 7 to 11.

13. A chip system, characterized in that: The chip system includes a processor coupled to a memory, and the processor executes a computer program stored in the memory to implement the method as described in any one of claims 3 to 6, or the method as described in any one of claims 7 to 11.

Images