Interface display method and related device

Abstract

The invention provides an interface display method and a related device. The method comprises the following steps: a first device receives a first moving operation for a first identifier when displaying the first identifier; if the first identifier moves to the preset edge of the first device, the first device sends a first crossing request carrying the position information of the first identifier to a second device; when the first identifier is located in the first area, the second equipment sends a third proportional relation to the first equipment; the third proportional relation is the relation between the size of the display area of the second equipment and the size of the collaborative window; the first device synchronizes a first drop point coordinate of the first identifier confirmed on the second device based on the third proportional relation to the second device, so as to display the first identifier on the second device; or when the first identifier is located in the second area, the second device displays the second identifier based on the first position information. In this way, the identifier of the input device can be moved from the first device to the collaboration window, and can also be moved to an area outside the collaboration window.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an interface display method and related devices. Background Art

[0002] With the development and progress of society, mobile phones, tablets, personal computers (PCs) and other terminal devices have become an indispensable part of people's lives. Users can connect multiple terminal devices for coordinated use.

[0003] In the keyboard and mouse sharing scenario, device A and device B can share each other's original input devices, such as mouse, keyboard, etc. The input device of device A can be inputted in both device A and device B. In the collaborative scenario, in the screen scenario of device A and tablet collaboration, device A can transfer part of the content it needs to display to device B for display, and device B displays a collaborative window for displaying part of the content of device A.

[0004] However, when device A and device B establish both a keyboard and mouse sharing connection and a collaborative connection, the input device of device A cannot input in windows other than the collaborative window in device B, resulting in a poor human-computer interaction experience. Summary of the invention

[0005] The embodiment of the present application provides an interface display method and related devices, which are applied to the field of terminal technology. The first device and the second device can confirm the landing points moved to different areas based on the first proportional relationship and the third proportional relationship respectively. The identifier of the input device can be moved from the first device to the collaborative window or to an area outside the collaborative window.

[0006] The keyboard and mouse sharing application of the second device and the tablet collaborative application can interact with each other. The first device can synchronize the size of the area used to display part of the content of the second device to the second device. Then, the first device and the second device can confirm whether crossing occurs based on the display area of ​​the collaborative window, and then confirm the calculation method to achieve that the mouse cannot be moved from the display area of ​​the collaborative window to other areas of the second device or to the functional area of ​​the collaborative window, etc.

[0007] In the first aspect, the embodiment of the present application proposes an interface display method. Applied to a collaborative system, the collaborative system includes a first device and a second device, and the method includes: when the first device displays a first identifier, it receives a first moving operation for the first identifier, and the first identifier is the identifier of an input device connected to the first device; when the first identifier moves to a preset edge of the first device, the first device sends the first position information corresponding to the first identifier to the second device; the preset edge is the edge of the first device in the direction of connection between the first device and the second device; when the first identifier of the second device based on the first position information is located in the first area, the second device displays the first identifier in the first area based on the first position information and a third proportional relationship; the first area is an area in the second device that displays part of the content of the first device; when the first identifier of the second device based on the first position information is located in the second area, the second device displays the second identifier in the second area based on the first position information; the second area is a display area of ​​the second device other than the first area, and the second identifier is the identifier of the input device.

[0008] The input device may be a mouse or a touch screen, etc., and the embodiment of the present application does not limit the specific type of the input device. The first mark may be a cursor or a mark in other forms; the second mark may be a dot or a mark in other forms; the embodiment of the present application does not limit the forms of the first mark and the second mark.

[0009] In the embodiment of the present application, the preset edge is related to the connection direction between the first device and the second device. When the connection direction is horizontal to the right and the first device is located on the left side of the second device, the preset edge is the right edge of the first device; when the connection direction is horizontal to the left and the first device is located on the right side of the second device, the preset edge is the left edge of the first device.

[0010] In this way, the first device and the second device can confirm the landing positions of different areas based on different proportional relationships, so that the identifier of the input device can be moved from the first device to the collaborative window or to an area outside the collaborative window.

[0011] Optionally, when the first identifier of the second device based on the first position information is located in the first area, the second device displays the first identifier in the first area based on the first position information and the third proportional relationship, including: when the first identifier moves to a preset edge of the first device, the first device sends a first crossing request to the second device, and the first crossing request includes: the first position information corresponding to the first identifier; when the position of the first identifier in the first position information is located in the first area, the second device sends a message to the first device to indicate a crossing failure; the message indicating a crossing failure includes: the third proportional relationship; the first area is an area in the second device that displays part of the content of the first device, and the third proportional relationship is the relationship between the size of the display area of ​​the second device and the size of the collaborative window displayed on the display screen; after receiving the message indicating a crossing failure, the first device synchronizes the first landing point coordinates of the first identifier on the second device to the second device, and the first landing point coordinates are confirmed based on the first position information and the third proportional relationship; the second device displays the first identifier in the first area based on the first landing point coordinates.

[0012] In this way, the second device confirms whether a crossing occurs for the position of the first identifier; when no crossing occurs, the third proportional relationship is returned to the first device, so that the first device can calculate the position of the first identifier in the collaborative window, and the input device moves from the first device to the first area.

[0013] Optionally, when the first identifier based on the first location information is located in the second area, the second device displays the second identifier in the second area based on the first location information, including: when the position of the first identifier is located in the second area, the second device displays the second identifier based on the first location information.

[0014] In this way, when crossing occurs, the second device displays the second marker based on the position of the first marker, and the input device moves from the first device to the second area.

[0015] Optionally, the first position information includes: first proportional coordinates of the first identifier, the first proportional coordinates are confirmed based on the coordinates of the first identifier at a preset edge and a first proportional relationship; the first proportional relationship is the relationship between the size of the display area of ​​the first device and the size of the display area of ​​the second device; or, the first position information includes: the first proportional relationship and the coordinates of the first identifier at a preset edge.

[0016] In this way, it is convenient for the second device to confirm the position of the first identifier.

[0017] Optionally, the first proportional relationship satisfies: x is the horizontal coordinate of the first marker on the first device, y is the vertical coordinate of the first marker on the first device, x0 is the horizontal length of the first device; y0 is the horizontal length of the first device, radioX is the horizontal coordinate of the first marker in the proportional coordinate, radioY is the vertical coordinate of the first marker in the proportional coordinate, m0 is the horizontal length of the second device; n0 is the horizontal length of the second device.

[0018] The first proportional relationship is the relationship between the display area sizes of the two devices, which facilitates the first device or the second device to subsequently confirm the position of the identifier of the input device.

[0019] Optionally, the third proportional relationship satisfies: L is the horizontal length of the collaborative window, h is the vertical length of the collaborative window, L1 is the distance between the landing point of the first marker and the vertical edge of the collaborative window, and h1 is the distance between the landing point of the first marker and the horizontal edge of the collaborative window.

[0020] This facilitates the first device or the second device to subsequently confirm the position of the identifier of the input device.

[0021] Optionally, L satisfies: L=m-m1, h1 satisfies: h1=n-m1, m is the horizontal coordinate of the landing point of the first identifier, n is the vertical coordinate of the landing point of the first identifier; m1 is the horizontal coordinate corresponding to the first corner of the collaborative window, and n1 is the vertical coordinate corresponding to the first corner.

[0022] The first corner may be the upper left corner or the upper right corner, and the embodiment of the present application does not limit the specific position of the first corner.

[0023] Optionally, if the coordinates of the first mark at the preset edge are (x0, y) or (0, y), the ordinate of the first scale coordinate is When the first scale coordinate is located in the first area, the ordinate of the first landing point coordinate is When the first scale coordinate is located in the second area, the ordinate of the first landing point coordinate is

[0024] Optionally, the method also includes: the first device receives a second moving operation for the first identifier displayed in the first area; when the first identifier moves to the first edge of the first area, the first device sends a second crossing request to the second device, and the second crossing request includes: a second proportional coordinate; the second proportional coordinate is confirmed by the first device based on a third proportional relationship and the coordinates of the first identifier at the edge of the first area; the first edge is the edge where the first area and the second area in the first area contact; the second device displays the second identifier based on the second proportional coordinate; or, when the first identifier moves to the second edge of the first area, the first device displays the first identifier based on the third proportional relationship, the first proportional relationship and the coordinates of the first identifier at the second edge, and the second edge is the edge of the first area in the direction of connection between the second area and the first area.

[0025] In this way, the identifier of the input device in the first area can be moved to the second area or to the first device. The first device can confirm the area corresponding to the landing point position of the identifier of the input device based on the edge, which is convenient for subsequent confirmation of the landing point coordinates.

[0026] Optionally, if the coordinates of the first mark at the first edge are (m2, n2), the second scale coordinates are ; Or, if the coordinates of the first mark at the second edge are (m3, n3), the ordinate of the first mark displayed by the first device is:

[0027] Optionally, the method also includes: when the second device displays the second identifier in the second area, it receives a third move operation for the second identifier; when the second identifier moves to the third edge of the second area, the second device sends a third crossing request to the first device, and the third crossing request includes: the coordinates of the second identifier at the third edge; the third edge is the edge in the second area in the connection direction of the second device and the first device; the first device displays the first identifier based on the first proportional relationship and the coordinates of the second identifier at the third edge; or, when the second identifier moves to the fourth edge of the second area, the second device sends a fourth crossing request to the first device, and the fourth crossing request includes third landing point coordinates, and the third landing point coordinates are confirmed based on the third proportional relationship and the coordinates of the second identifier at the fourth edge; the fourth edge is the edge of the second area in the second area where the second area contacts the first area; the first device synchronizes the third landing point coordinates and part of the content of the first device to the second device; the second device displays part of the content of the first device in the first area, and displays the first identifier based on the third landing point coordinates.

[0028] In this way, the identifier of the input device in the second area can be moved to the first area or to the first device. The second device can confirm the area corresponding to the landing point position of the identifier of the input device based on the edge, which facilitates the subsequent confirmation of the landing point coordinates.

[0029] Optionally, if the coordinates of the second marker at the third edge are (m4, n4), the ordinate of the first marker determined by the first device is If the coordinates of the second mark at the fourth edge are (m5, n5), the coordinates of the third landing point are

[0030] Optionally, the collaborative window includes: a function area, a display area and a sidebar area, the function area is an area for displaying one or more controls, and the controls are used to implement control and adjustment of the collaborative window; the display area is an area for displaying part of the content of the first device, and the sidebar area is an area corresponding to the border of the collaborative window; the range of the first area is less than or equal to the range of the display area, and the range of the display area satisfies: the range of the display area = the range of the collaborative window - the range of the function area - the range of the sidebar area.

[0031] Thus, the first area is the display area.

[0032] Optionally, the second device also displays one or more first windows, and the first window is located on the upper layer of the collaborative window; when the first window does not block the collaborative window, the range of the first area is equal to the range of the display area; when the first window blocks the collaborative window, the range of the first area is smaller than the range of the display area; the range of the first area satisfies: the range of the first area = the range of the display area - the range of the first intersection area, and the first intersection area is the intersection area of ​​the collaborative window and the first window.

[0033] When the display area is blocked, the first area is the difference between the display area and the blocked area. The first area is smaller than the display area.

[0034] Optionally, when the range of the collaborative window exceeds the display area of ​​the second device, the range of the first area is smaller than the range of the display area; when the truncation area is the intersection area between the collaborative window and the display area of ​​the second device, the range of the first area satisfies: the range of the first area = the range of the truncation area - the range of the first intersection area - the range of the sidebar area - the range of the function area; when the truncation area is the intersection area between the display area and the display area of ​​the second device, the range of the first area satisfies: the range of the first area = the range of the truncation area - the range of the first intersection area.

[0035] The collaborative window may be truncated, and the first area is the difference between the display area and the blocked area on the display area of ​​the second device.

[0036] Optionally, when the collaborative window is in a suspended state, the second device confirms the range of the collaborative window based on an initial range of the collaborative window and a scaling ratio of the collaborative window.

[0037] Confirm the collaboration window range based on the zoom ratio to facilitate resizing and calculation.

[0038] Optionally, the range of the collaborative window is the product of the initial range of the collaborative window and the scaling ratio of the collaborative window; or, the range of the collaborative window is the product of the initial range of the collaborative window and the scaling ratio of the collaborative window and then rounded to an integer.

[0039] Rounding facilitates subsequent calculations and displays and reduces the amount of calculation required by the device.

[0040] Optionally, before the first device receives the first movement operation, the method also includes: the first device and the second device establish a collaborative connection and a keyboard and mouse sharing connection; after establishing the collaborative connection and the keyboard and mouse sharing connection, the second device sends a first collaborative message to the first device; the first collaborative message is used by the first device to confirm the scope of the first area.

[0041] In this way, the relevant information of the synchronization collaboration window, such as the range, connection direction, truncation direction, etc., is convenient for the subsequent confirmation of the corresponding area of ​​the movement and the confirmation of the landing point of the subsequent input device identification.

[0042] Optionally, the method also includes: the second device receives an operation for indicating a change in the collaborative window; in response to the operation for indicating a change in the collaborative window, the second device synchronizes second collaborative information to the first device, the second collaborative information being information for describing the changed collaborative window.

[0043] When the collaborative window changes, the information of the changed collaborative window is synchronized to facilitate the first device to adjust the proportion relationship, calculation method, etc.

[0044] Optionally, the method further includes: the operation for indicating the change of the collaborative window includes: an operation for indicating the adjustment of the collaborative window size, an operation for indicating the adjustment of the collaborative window position, and an operation for indicating the switching of the collaborative window form.

[0045] In the second aspect, the embodiment of the present application provides an interface display method, which is applied to a first device. The method includes: when the first device displays a first identifier, it receives a first moving operation for the first identifier, and the first identifier is the identifier of the input device connected to the first device; when the first identifier moves to the preset edge of the first device, the first device sends a first crossing request to the second device, and the first crossing request includes: first position information corresponding to the first identifier, the preset edge is the edge of the first device in the direction of connection between the first device and the second device; the collaborative window is a window in the second device for displaying part of the content of the first device; when the position of the first identifier in the first position information is located in the first area, the first device receives a message transmitted by the second device to indicate a crossing failure; the message for indicating a crossing failure includes: a third proportional relationship; the first area is an area in the second device that displays part of the content of the first device, and the third proportional relationship is the relationship between the size of the display area of ​​the second device and the size of the collaborative window displayed on the display screen; the first device synchronizes the first landing point coordinates of the first identifier on the second device to the second device, the first landing point coordinates are confirmed based on the first position information and the third proportional relationship, and the first landing point coordinates are used for the second device to display the first identifier in the first area;

[0046] Optionally, the method also includes: the first device receives a second move operation for the first identifier displayed in the first area; when the first identifier moves to the first edge of the first area, the first device sends a second crossing request to the second device, and the second crossing request includes: second proportional coordinates, which are confirmed by the first device based on the third proportional relationship and the coordinates of the first identifier at the edge of the first area; the first edge is the edge where the first area and the second area in the first area contact; the second proportional coordinates are used to display the second identifier on the second device; or, when the first identifier moves to the second edge of the first area, the first device displays the first identifier based on the third proportional relationship, the first proportional relationship and the coordinates of the first identifier at the second edge, and the second edge is the edge of the first area in the direction of connection between the second area and the first area.

[0047] Optionally, the method further includes: the first device receives a third crossing request from the second device, the third crossing request includes: the coordinates of the second identifier at the third edge of the second area; the third edge is the edge in the second area in the connection direction between the second device and the first device; the first device displays the first identifier based on the first proportional relationship and the coordinates of the second identifier at the third edge; or the first device receives a fourth crossing request from the second device, the fourth crossing request includes: the third landing point coordinates; the third landing point coordinates are confirmed based on the third proportional relationship and the coordinates of the second identifier at the fourth edge; the fourth edge is the edge where the second area in the second area contacts the first area;

[0048] The first device synchronizes the third landing point coordinates and part of the content of the first device to the second device; the second device displays part of the content of the first device and displays the first identifier based on the third landing point coordinates.

[0049] Optionally, before the first device receives the first movement operation, the method also includes: the first device and the second device establish a collaborative connection and a keyboard and mouse sharing connection; after establishing the collaborative connection and the keyboard and mouse sharing connection, the first device receives first collaborative information from the second device; the first collaborative information is used by the first device to confirm the scope of the first area.

[0050] In a third aspect, an embodiment of the present application provides an interface display method, which is applied to a second device. The method includes:

[0051] The second device receives a first crossing request from the first device, the first crossing request including: first position information corresponding to the first identifier; the first crossing request is generated when the first device receives a first moving operation for the first identifier and the first identifier moves to a preset edge of the first device, the preset edge being an edge in the first device in the direction of connection between the first device and the second device; when the position of the first identifier in the first position information is located in the first area, the second device sends a message indicating a crossing failure to the first device; the message indicating a crossing failure includes: a third proportional relationship; the first area is an area in the second device that displays part of the content of the first device, the third proportional relationship is a relationship between the size of the display area of ​​the second device and the size of the collaborative window displayed on the display screen; the collaborative window is a window in the second device that is used to display part of the content of the first device; the second device receives a first landing point coordinate from the first device, the first landing point coordinate is confirmed based on the first position information and the third proportional relationship; the second device displays the first identifier in the first area based on the first landing point coordinate; or, when the position of the first identifier is located in the second area, the second device displays the second identifier based on the first position information; the second area is a display area of ​​the second device other than the first area, and the second identifier is an identifier of an input device connected to the first device.

[0052] Optionally, the method also includes: the second device receives a second crossing request from the first device, the second crossing request includes: second scale coordinates; the second scale coordinates are confirmed by the first device based on a third scale relationship and the coordinates of the first identifier at the edge of the first area; the second crossing request is generated by the first device when it receives a second move operation for the first identifier displayed in the first area, and the first identifier moves to the first edge of the first area; the first edge is the edge of the first area in the first area where the first area contacts the second area; the second device displays the second identifier based on the second scale coordinates.

[0053] Optionally, the method also includes: when the second device displays the second identifier in the second area, the second device receives a third move operation for the second identifier; when the second identifier moves to the third edge of the second area, the second device sends a third crossing request to the first device, and the third crossing request includes: the coordinates of the second identifier at the third edge; the third edge is the edge in the connection direction of the second device and the first device in the second area; the coordinates of the second identifier at the third edge are used to display the first identifier on the first device; or, when the second identifier moves to the fourth edge of the second area, the second device sends a fourth crossing request to the first device, and the fourth crossing request includes third landing point coordinates, which are confirmed based on the third proportional relationship and the coordinates of the second identifier at the fourth edge; the fourth edge is the edge where the second area in the second area contacts the first area; the second device receives the third landing point coordinates and part of the content of the first device from the first device; the second device displays part of the content of the first device in the first area, and displays the first identifier based on the third landing point coordinates.

[0054] Optionally, before the second device receives the first crossing request, the method also includes: the second device and the first device establish a collaborative connection and a keyboard and mouse sharing connection; after establishing the collaborative connection and the keyboard and mouse sharing connection, the second device sends a first collaborative message to the first device; the first collaborative message is used by the first device to confirm the scope of the first area.

[0055] In a fourth aspect, an embodiment of the present application provides a terminal device, which may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc.

[0056] The terminal device comprises: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method of the second aspect or the method of the third aspect.

[0057] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method of the second aspect or the method of the third aspect is implemented.

[0058] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is run, it enables a computer to execute the method of the second aspect or the method of the third aspect.

[0059] In a seventh aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute a method such as the second aspect, or a method such as the third aspect.

[0060] It should be understood that the second to seventh aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1A A schematic diagram of an interface for establishing keyboard and mouse sharing provided in an embodiment of the present application;

[0062] Figure 1B A schematic diagram of a collaborative establishment interface provided in an embodiment of the present application;

[0063] Figure 2A This is a schematic diagram of the interface of the second device when the collaborative window is in full screen mode;

[0064] Figure 2B This is a schematic diagram of the interface of the second device when the collaborative window is in suspended form;

[0065] Figure 3A A schematic diagram of an interface between a first device and a second device in a possible design;

[0066] Figure 3B A schematic diagram of an interface between a first device and a second device in a possible design;

[0067] Figure 4 A schematic diagram of an interface between a first device and a second device provided in an embodiment of the present application;

[0068] Figure 5A A schematic diagram of an interface of a second device when a collaborative window is in full-screen mode provided in an embodiment of the present application;

[0069] Figure 5B A schematic diagram of an interface of a second device when a collaborative window is in full-screen mode provided in an embodiment of the present application;

[0070] Figure 5C A schematic diagram of an interface of a second device when a collaborative window is in a suspended state provided in an embodiment of the present application;

[0071] Figure 5D A schematic diagram of an interface of a second device when a collaborative window is in a suspended state provided in an embodiment of the present application;

[0072] Figure 5E A schematic diagram of an interface of a second device when a collaborative window is in a suspended state provided in an embodiment of the present application;

[0073] Figure 6 A flowchart of an interface display method provided in an embodiment of the present application;

[0074] Fig. 7A A schematic diagram of an application scenario provided for an embodiment of the present application;

[0075] Figure 7B A schematic diagram of an application scenario provided for an embodiment of the present application;

[0076] Figure 7C A schematic diagram of an application scenario provided for an embodiment of the present application;

[0077] Fig.7D A schematic diagram of an application scenario provided for an embodiment of the present application;

[0078] Figure 8 A flowchart of an interface display method provided in an embodiment of the present application;

[0079] Fig.9A A schematic diagram of an application scenario provided for an embodiment of the present application;

[0080] Fig. 9B A schematic diagram of an application scenario provided for an embodiment of the present application;

[0081] Fig.10 A flowchart of an interface display method provided in an embodiment of the present application;

[0082] Fig.11A A schematic diagram of an application scenario provided for an embodiment of the present application;

[0083] Fig. 11B A schematic diagram of an application scenario provided for an embodiment of the present application;

[0084] Fig.12A schematic diagram of the software architecture provided for an embodiment of the present application;

[0085] Fig.13A A schematic diagram of an interaction process corresponding to first establishing a keyboard and mouse sharing connection and then establishing a collaborative connection provided in an embodiment of the present application;

[0086] Fig. 13B A schematic diagram of an interaction process corresponding to first establishing a collaborative connection and then establishing a keyboard and mouse sharing connection provided in an embodiment of the present application;

[0087] Fig.14 A schematic diagram of an interaction process corresponding to a sidebar direction switching provided in an embodiment of the present application;

[0088] Fig.15A A schematic diagram of an interactive process corresponding to switching from a full-screen mode to a floating mode provided in an embodiment of the present application;

[0089] Fig. 15B A schematic diagram of an interactive process corresponding to switching from a suspended state to a full-screen state provided in an embodiment of the present application;

[0090] Fig.16 A schematic diagram of an interactive process corresponding to collaborative window movement and scaling provided in an embodiment of the present application;

[0091] Fig.17 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0092] To facilitate understanding of the embodiments of the present application, some terms involved in the present application are first briefly explained.

[0093] 1. Screen mirroring refers to mirroring all the contents on the screen of the first device to the screen of the second device. The contents of the first device displayed on the second device are the same as those displayed on the first device, and change as the contents displayed on the first device change.

[0094] 2. Screen extension: Use the screen of the second device as the secondary screen of the first device. The first device can transfer part of the content it needs to display to the secondary screen for display. The screen of the first device and the screen of the second device can display content together, expanding the display capability of the first device.

[0095] 3. Keyboard and mouse sharing: The first device and the second device can share each other's original input devices, such as a mouse, keyboard, etc. For example, the input device of the first device is shared with the second device, and the shared input device can be used for input in both the first device and the second device. In addition, in the keyboard and mouse sharing type, files can also be transferred between the first device and the second device.

[0096] 4. Other terms

[0097] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first device and the second device are only used to distinguish different devices, and do not limit their order. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0098] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0099] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0100] 2. Terminal equipment

[0101] The terminal device of the embodiment of the present application may also be any form of electronic device. For example, the electronic device may include a handheld device, a vehicle-mounted device, etc. For example, some electronic devices are: mobile phones, tablet computers, PDAs, laptop computers, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (public land mobile The embodiments of the present application do not limit this.

[0102] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0103] In addition, in the embodiments of the present application, the electronic device can also be a terminal device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0104] The electronic devices in the embodiments of the present application may also be referred to as: terminal equipment, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0105] In an embodiment of the present application, the electronic device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux operating system, Unix operating system, Android operating system, iOS operating system, or Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0106] In daily work and life, users may use multiple terminal devices at the same time. In order to improve the convenience of collaborative use of multiple terminal devices, the first device and the second device can be connected wirelessly or wired to work collaboratively.

[0107] For example, Figure 1A A schematic diagram of a keyboard and mouse sharing interface provided in an embodiment of the present application. Figure 1A As shown,

[0108] Figure 1AThe interface shown in a in includes: multiple setting items, and the details page corresponding to the selected setting item. The setting items include, but are not limited to: smart interconnection 101, multi-screen collaboration 102, system optimization, application recommendation, smart audio-visual, game skills, troubleshooting, official services or other types of setting items. When the selected setting item is smart interconnection 101, the interface will display the device identification 103 of the first device and the device identification 104 of the second device. In the embodiment of the present application, the first device can be a computer, and the second device can be a tablet or a terminal device with a folding screen.

[0109] The device identification 103 of the first device and the device identification 104 of the second device may be graphical identifications, such as icons, or other skeuomorphic elements, etc. This embodiment of the present application does not limit this.

[0110] The user may drag the device identification 104 of the second device toward the device identification 103 of the first device through a drag operation.

[0111] When the drag operation ends at a position near the device identifier 103 of the first device (for example, the distance between the device identifier 104 of the second device and the device identifier 103 of the first device is less than a certain threshold, and the device identifier 104 of the second device is dragged to a fixed position), the master device establishes a keyboard and mouse connection with the slave device, and the first device enters Figure 1A The interface shown in b in FIG. The interface includes: a plurality of setting items, a device identification 103 of the first device, and a device identification 104 of the second device. The device identification 104 of the second device is located near the device identification 103 of the first device.

[0112] For example, Figure 1B A schematic diagram of a collaborative interface provided in an embodiment of the present application. Figure 1B As shown,

[0113] Figure 1B The interface shown in a in the figure includes: multiple setting items, and the details page corresponding to the selected setting item. The setting items include but are not limited to: smart interconnection 101, multi-screen collaboration 102, system optimization, application recommendation, smart audio-visual, game skills, troubleshooting, official services or other types of setting items. When the selected setting item is multi-screen collaboration 102, the interface will display a list of available devices and the selection controls corresponding to the available devices. Figure 1B As shown in a in FIG. 1 , the device list of available devices includes: tablet 105 and folding screen mobile phone 106. Selection control 107 corresponds to tablet 105, and selection control 108 corresponds to folding screen mobile phone 106.

[0114] When the first device receives a click operation on the selection control 107, the first device establishes a collaborative connection with the tablet and enters Figure 1BThe interface shown in b in FIG. The interface displays two modes of collaborative connection, namely, mirror mode and extended mode, a selection control corresponding to the mirror mode, and a selection control 109 corresponding to the extended mode. The mirror mode corresponds to screen mirroring. The extended mode corresponds to screen extension.

[0115] When the first device receives a click operation on the selection control 109 , the cooperative connection between the first device and the tablet is screen extension.

[0116] It is understandable that the above Figure 1A and Figure 1B The connection shown is initiated from the first device side, and the above two connections can also be initiated from the second device side. This embodiment of the application will not be described in detail. Figure 1A and Figure 1B The interface shown is for reference only and may include more or less content, which is not limited here.

[0117] In some embodiments, a keyboard and mouse sharing connection or a collaborative connection can also be established through shortcuts. For example, the mouse touches the side edge of the screen of the first device twice in succession to establish a keyboard and mouse sharing connection; after the keyboard and mouse sharing connection, drag a window of the first device to the second device to establish a collaborative connection. The embodiments of the present application do not limit the type of shortcuts, operation methods, etc.

[0118] Combine the following Figure 2A and Figure 2B The interface of the second device after the collaborative connection is established is described.

[0119] For example, Figure 2A This is a schematic diagram of the interface of the second device when the collaborative window is in full screen mode. Figure 2A As shown, the interface of the second device includes a collaboration window 201. The collaboration window 201 includes: a display area 202 and a function area 203. The display area 202 is used to display part of the content of the first device. The function area 203 is used to implement some functions of the collaboration, such as switching to mirror mode, exiting collaboration, etc.

[0120] When the collaborative window 201 is in full screen mode, the collaborative window 201 is displayed in full screen, and no other windows are displayed. The display area 202 and the function area 203 occupy the screen (display screen) of the second device.

[0121] For example, Figure 2B This is a schematic diagram of the interface of the second device when the collaborative window is in suspended state. Figure 2BAs shown, the interface of the second device includes a collaborative window 201 and a first window 204. The collaborative window 201 includes: a display area 202, a function area 203 and a sidebar area 205. The sidebar area 205 is used to display the adjustment controls corresponding to the collaborative window 201 to adjust the collaborative window 201, for example, switching to full screen mode, zooming, moving, etc. The functions of the display area 202 and the function area 203 can be referred to the above description, and will not be repeated here. The first window 204 is used to display the content of the second device.

[0122] When the collaborative window 201 is in a floating state, the collaborative window 201 is not displayed in full screen, but is suspended above the first window 204 .

[0123] However, when the first device and the second device establish both a keyboard and mouse sharing connection and a collaborative connection, the input device of the first device cannot input other areas of the second device except the collaborative window, resulting in a poor human-computer interaction experience.

[0124] In a possible design, the input device of the first device can be moved between the first device and the display area 202 of the second device. The input device of the first device cannot be moved from the area where the content of the first device is displayed to other areas of the second device.

[0125] Taking the collaborative window in full screen form as an example, the input device of the first device can be moved between the display area 202 of the first device and the second device. The input device of the first device cannot be moved from the first device to other areas of the second device, for example, windows other than the collaborative window in the second device, the function area 203 or the sidebar area 205 of the collaborative window in the second device.

[0126] Take the full-screen collaborative window as an example. Figure 3A As shown, the first identifier 301 cannot be moved from the first device to the function area 203 of the collaborative window 201; the first identifier 301 cannot be moved from the display area 202 of the second device to the function area 203. The first identifier 301 is an identifier corresponding to the input device of the first device.

[0127] Take the collaborative window in suspended form as an example. Figure 3B As shown, the first identifier 301 can be moved between the display area 202 of the first device and the second device. The first identifier 301 cannot be moved from the first device to the first window 204; the input device of the first device cannot be moved from the collaborative window 201 of the second device to the first device.

[0128] In addition, in a possible design, the first identifier 301 cannot be moved from the display area of ​​the collaborative window to other areas of the second device, or to areas outside the display area of ​​the collaborative window. Figure 2BTaking the interface shown as an example, the input device mouse of the first device cannot be moved from the display area 202 of the collaborative window 201 to the first window 204 , the function area 203 of the collaborative window 201 , or the sidebar area 205 .

[0129] In view of this, an embodiment of the present application provides a control method, in which the keyboard and mouse sharing application of the second device and the tablet collaborative application can interact, and the first device can synchronize the size of the area used to display part of the content of the second device to the second device, and then the first device and the second device can confirm whether crossing occurs based on the display area of ​​the collaborative window, and then confirm the calculation method to achieve that the mouse cannot be moved from the display area of ​​the collaborative window to other areas of the second device or to the functional area of ​​the collaborative window, etc.

[0130] For ease of understanding, the coordinate conversion between the first device and the second device is explained below.

[0131] For example, Figure 4 This is a schematic diagram of the interface between the first device and the second device provided in the embodiment of the present application. Figure 4 As shown, the length of the display area of ​​the first device in the horizontal direction is x0, and the length of the display area of ​​the first device in the vertical direction is y0. The length of the display area of ​​the second device in the horizontal direction is m0, and the length of the display area of ​​the first device in the vertical direction is n0. The length of the collaborative window in the horizontal direction is L, and the length of the collaborative window in the vertical direction is h.

[0132] In the embodiment of the present application, the first coordinate of the first device is (x, y), the proportional coordinate converted between the first device and the second device is defined as (radioX, radioY), and the second coordinate of the second device is (m, n).

[0133] It can be understood that the coordinates of the identifier corresponding to the input device transmitted from the first device to the second device are (x, y), which are usually proportionally converted according to the screen size of the second device.

[0134] When the identifier corresponding to the input device moves from the first device to the second device, the first device converts the first coordinate of the first device and the size of the second device based on the first proportional relationship to obtain proportional coordinates (radioX, radioY). The proportional coordinates are transmitted to the second device. The first proportional relationship is the relationship between the size of the display area of ​​the first device and the size of the display area (display screen) of the second device. The first proportional relationship satisfies: but

[0135] Specifically, when the scale coordinates are not located in the first area, the second device determines the scale coordinates as the coordinates of the landing point of the identifier corresponding to the input device in the second area. Then the coordinates of the landing point of the identifier corresponding to the input device in the second device are: m=radioX, n=radioY. The first area is the area corresponding to the display of part of the content of the first device in the second device. The second area is the area in the display area of ​​the second device other than the first area.

[0136] When the proportional coordinates are within the first area, the first device determines the coordinates of the point where the identifier corresponding to the input device falls within the first area based on the third proportional relationship from the second device, and synchronizes the coordinates to the second device. The third proportional relationship is the relationship between the size of the display area of ​​the second device and the size of the collaborative window displayed on the interface.

[0137] The third proportional relationship is satisfied: L is the length of the collaborative window in the horizontal direction, and h is the length of the collaborative window in the vertical direction. L1 is the distance between the landing point of the second device and the left edge of the collaborative window, and h1 is the distance between the landing point of the second device and the upper edge of the collaborative window. Then the landing point coordinates of the input device on the second device are m=L1+m1, that is n=h1+n1, that is

[0138] The third proportional relationship can also satisfy: The coordinates of the second device's landing point (m, n) L are the length of the collaborative window in the horizontal direction, and h is the length of the collaborative window in the vertical direction. m1 is the horizontal coordinate of the upper left corner of the display area of ​​the collaborative window, and n1 is the vertical coordinate of the upper left corner of the display area of ​​the collaborative window. Then the coordinates of the second device's landing point are

[0139] It should be noted that the embodiment of the present application describes and calculates the window range by the upper left corner and the lower right corner of the window. Adaptively, the coordinates of the landing point of the first mark in the collaborative window are also calculated based on the coordinates of the upper left corner of the window. The window range and the coordinates of the landing point of the mark of the input device can also be described and calculated in other ways, which are not limited in the embodiment of the present application.

[0140] For ease of understanding, the following Figure 5A-Figure 5B The first area, the blocked area, the truncated area and other areas in the second device are described.

[0141] The first area is an area corresponding to a portion of the content of the first device displayed in the interface displayed by the second device. The blocked area is an area where the window located in the upper layer of the collaborative window blocks the display area. Or it can also be understood as the intersection area of ​​the display area and the window located in the upper layer of the collaborative window. In some embodiments, the blocked area may also refer to an area where the window located in the upper layer of the collaborative window blocks the collaborative window. This is not limited here.

[0142] In the embodiment of the present application, the first area is the difference between the display area and the shielding area. The display area satisfies: display area = collaborative window - functional area - sidebar area. The first area is smaller than or equal to the display area.

[0143] It is understandable that there may be a window in the second device located above the collaborative window, which may block the collaborative window. When the collaborative window is blocked, the first area is smaller than the display area. When the collaborative window is not blocked, the first area is equal to the display area.

[0144] The window located on the upper layer of the collaborative window may be a pop-up window, a pop-up window corresponding to a notification message, a window of another application, a toast pop-up window, etc. This embodiment of the present application does not limit this.

[0145] For example, Figure 5A This is a schematic diagram of the interface of the second device when the collaborative window is in full screen mode provided in an embodiment of the present application. Figure 5A As shown in ad in FIG. 1 , the collaboration window 201 is displayed in full screen.

[0146] Take the upper left corner of the display screen of the second device as the origin, the horizontal right as the horizontal axis, the vertical downward as the vertical axis, the horizontal length as m0, the vertical length as n0, and the preset edge as the right edge of the first device as an example. Figure 5A In the interface shown in ad, the range of the collaborative window 201 is (0, 0, m0, n0). In the embodiment of the present application, the range of the collaborative window 201 is described by the coordinates of the upper left corner and the lower right corner of the collaborative window. The range of the collaborative window 201 can also be described by other methods, which are not limited here.

[0147] Figure 5A In the interface shown in a, the function area 203 is located on the left side of the display area 202. Then the area corresponding to the function area 203 is (0, 0, W, n0); the area coordinates corresponding to the display area 202 are (W, 0, m0, n0). The scale coordinates of the first area are (W, 0, m0, n0). When the first mark moves to the right edge of the first device, a crossing occurs. When the horizontal coordinate of the first mark is x0, or the coordinates of the first mark are between [(x0, 0), (x0, y0)], the first mark crosses.

[0148] Figure 5AIn the interface shown in b, the function area 203 is located on the right side of the display area 202. Then the area corresponding to the function area 203 is (m0-W, 0, m0, n0); the coordinates of the display area 202 are (0, 0, m0-W, n0); the scale coordinates of the first area are (0, 0, m0-W, n0). When the first mark moves to the right edge of the first device, no crossing occurs. When the horizontal coordinate of the first mark is x0, or the coordinates of the first mark are located at [(x0, 0), (x0, y0)], the first mark does not cross.

[0149] Figure 5A In the interface shown in c, the function area 203 is located at the top of the collaborative window. The area corresponding to the function area is (0, 0, mo, s); the coordinates of the display area are (0, s, m0, n0). The scale coordinates of the first area are (0, s, m0, n0). When the first mark moves to the right edge of the first device, it is confirmed whether the crossing occurs based on the position of the first mark. When the vertical coordinate of the first mark is located at to y0, or when the coordinates of the first marker are between [(x0, ), (x0, y0)], no crossing occurs. When the coordinates of the first marker are between [(0,0), (x0, )], a crossing occurs.

[0150] Figure 5A In the interface shown in d, the function area 203 is located at the bottom of the collaborative window. The area corresponding to the function area is (0, n0-s, m0, n0); the coordinates of the display area are (0, 0, m0, n0-s). The scale coordinates of the first area transmitted from the second device to the first device are (0, 0, m0, n0-s). When the first marker moves to the right edge of the first device, confirm whether the crossing occurs based on the position of the first marker. When the vertical coordinate of the first marker is between 0 and or, when the coordinates of the first marker are between [(x0, 0), (x0, )], no crossing occurs; when the vertical coordinate of the first mark is at to y0, or when the coordinates of the first marker are between [(x0, ),(x0,y0)], crossing occurs.

[0151] In other implementations, when the function area is located at the top or the bottom, when the identifier of the input device moves from the first device to the second device, no crossing occurs and the identifier moves to the display area. The embodiments of the present application do not limit the specific implementation.

[0152] In some embodiments, a partial window may obstruct the collaborative window.

[0153] For example, Figure 5B This is a schematic diagram of the interface of the second device when the collaborative window is in full screen mode provided in an embodiment of the present application. Figure 5B As shown, the collaborative window 201 is displayed in full screen. The interface of the second device also includes a second window 501. The second window 501 blocks the collaborative window 201.

[0154] The shielding area is the intersection area of ​​the second window 501 and the display area 202. Figure 5B As shown, the blocked area is the second window 501. In a possible implementation, the blocked area is the intersection area of ​​the second window 501 and the collaborative window 201.

[0155] The range of the first area satisfies: the range of the first area = the range of the collaborative window 201 - the range of the functional area 203 - the range of the shielding area. Figure 5B As shown, the range of the first area = the range of the collaborative window 201 - the range of the functional area 203 - the range of the second window 501 , that is, the range of the first area = the range of the display area 202 - the range of the second window 501 .

[0156] In some embodiments, the second device stores an initial range and a zoom ratio of the second window. The second device confirms the range of the second window based on the initial range and the zoom ratio.

[0157] In possible implementation mode 1, the range of the second window may be the product of the initial range corresponding to the second window and the scaling ratio corresponding to the second window. Exemplarily, the value of the second window in the horizontal direction = the initial value corresponding to the second window in the horizontal direction * the scaling ratio corresponding to the second window. The value of the second window in the vertical direction = the initial value corresponding to the second window in the vertical direction * the scaling ratio corresponding to the second window.

[0158] In possible implementation mode 2, the range of the second window can be a value calculated by rounding the product of the initial range corresponding to the second window and the scaling ratio corresponding to the second window. Exemplarily, the value of the second window in the horizontal direction = the initial value corresponding to the second window in the horizontal direction * the scaling ratio corresponding to the second window. The value of the second window in the vertical direction = the initial value corresponding to the second window in the vertical direction * the scaling ratio corresponding to the second window.

[0159] In some embodiments, the second device may confirm the range of the occluded area through the setIntersect method.

[0160] The blocked area = setIntersect (the range of the second window, the range of the display area of ​​the collaborative window). Then the range of the first area = the range of the collaborative window 201 - the range of the function area - setIntersect (the range of the second window, the range of the display area of ​​the collaborative window).

[0161] For example, Figure 5C This is a schematic diagram of the interface of the second device when the collaborative window is in a suspended state provided in an embodiment of the present application. Figure 5C As shown, the interface displays a collaborative window 201 and a first window 204. The first window 204 is located below the collaborative window 201. Figure 5C The collaboration window is shown in suspended form.

[0162] In some embodiments, the second device stores an initial range and a zoom ratio of the collaborative window. The second device confirms the range of the collaborative window based on the initial range and the zoom ratio of the collaborative window.

[0163] In a possible implementation method 1, the range of the collaborative window may be the product of the initial range corresponding to the collaborative window and the scaling ratio corresponding to the collaborative window. Exemplarily, the value of the collaborative window in the horizontal direction = the initial value corresponding to the collaborative window in the horizontal direction × the scaling ratio corresponding to the collaborative window. The value of the collaborative window in the vertical direction = the initial value corresponding to the collaborative window in the vertical direction × the scaling ratio corresponding to the collaborative window.

[0164] In possible implementation mode 2, the range of the collaborative window can be the product of the initial range corresponding to the collaborative window and the scaling ratio corresponding to the collaborative window, and then rounded to obtain the value. Exemplarily, the value of the collaborative window in the horizontal direction = the initial value corresponding to the collaborative window in the horizontal direction × the scaling ratio corresponding to the collaborative window, and then rounded to obtain the value. The value of the collaborative window in the vertical direction = the initial value corresponding to the collaborative window in the vertical direction × the scaling ratio corresponding to the collaborative window, and then rounded to obtain the value.

[0165] The range of the first area satisfies: the range of the first area=the range of the collaborative window 201 −the range of the function area 203 −the range of the sidebar area 205 .

[0166] Taking the floating form as an example, in which the function area is a rectangle and is located in the center of the right side of the collaborative window, the range of the function area is (the difference between the horizontal coordinate of the right edge of the collaborative window and the horizontal length of the function area, the difference between the vertical coordinate corresponding to the center of the collaborative window and half of the horizontal length of the function area, the sum of the horizontal coordinate of the right edge of the collaborative window, the vertical coordinate corresponding to the center of the collaborative window and half of the horizontal length of the function area).

[0167] In the suspended form, the functional area can also be circular, etc. The embodiment of the present application does not limit the range calculation of the functional area.

[0168] In some embodiments, some windows may block the collaborative window. Figure 5D As shown, the range of the first area satisfies: the range of the first area=the range of the collaborative window 201 −the range of the functional area 203 −the range of the sidebar area 205 −the range of the shielding area 502 .

[0169] On the basis of the above embodiments, in the embodiments of the present application, the user can control the floating window to move, enlarge or reduce through the input device. When the floating window moves, enlarges or reduces to the edge of the second device, the collaborative window may not be fully displayed and the display area is truncated. The range of the first area = the range of the truncated area - the range of the function area 203 - the range of the sidebar area 205 - the range of the blocked area 502. The truncated area refers to the area of ​​the display area displayed on the display screen when the collaborative window is not fully displayed.

[0170] For example, Figure 5E This is a schematic diagram of the interface of the second device when the collaborative window is in a suspended state provided in an embodiment of the present application. Figure 5E As shown, the interface displays a collaborative window 201 and a first window 204. The first window 204 is located below the collaborative window 201. Figure 5E The collaborative window shown is in a suspended state. Part of the collaborative window 201 is located outside the display screen, and the collaborative window 201 is not fully displayed.

[0171] The cut-off area is the intersection of the collaborative window and the display area of ​​the display screen. In the embodiment of the present application, the cut-off area may also be the intersection of the display area of ​​the collaborative window and the display area of ​​the display screen.

[0172] If the second device determines the range of the collaborative window based on the scaling ratio, the range of the truncated area 503 satisfies: the range of the truncated area = the intersection of the range of the display area 202 of the collaborative window and the range of the display area of ​​the display screen. The range of the display area 202 = the range of the collaborative window - the range of the sidebar area - the range of the function area.

[0173] In a possible implementation, the range of the display area 202 = (the initial range of the collaborative window - the initial range of the sidebar area - the initial range of the functional area) * the scaling ratio of the collaborative window. In a possible implementation, the range of the display area 202 = (the initial range of the collaborative window - the initial range of the sidebar area - the initial range of the functional area) * the scaling ratio of the collaborative window, and then rounded. The embodiment of the present application does not limit the specific calculation method of the range of the display area 202.

[0174] The range of the first area satisfies: the range of the first area=the range of the truncated area 503 −the range of the blocked area and the range of the functional area 203 .

[0175] It can be understood that if the truncated area includes the functional area 203 , then the range of the first area=the range of the truncated area 503 −the range of the blocked area −the range of the functional area 203 .

[0176] In some embodiments, the function area 203 is located in the middle right part of the display area of ​​the collaborative window. The function area 203 can also be located in the middle left part of the display area, or any other position. The embodiment of the present application does not limit the position of the function area.

[0177] Combine the following Figure 6-15B The interface display method provided in the embodiment of the present application is described. Moving from a first device to a second device; moving from a first area to a second area, or a first device; moving from a second area to a first area or a first device.

[0178] For example, Figure 6 The following is a flow chart of an interface display method provided in an embodiment of the present application. Figure 6 As shown, the interface display method includes:

[0179] S601: A first device and a second device establish a collaborative connection and a keyboard and mouse sharing connection.

[0180] In the embodiment of the present application, the first device and the second device may first establish a collaborative connection and then establish a keyboard and mouse sharing connection; the first device and the second device may first establish a keyboard and mouse sharing connection and then establish a collaborative connection. The embodiment of the present application does not limit the order of the collaborative connection and the keyboard and mouse sharing connection.

[0181] For example, the process of establishing a collaborative connection first and then establishing a keyboard and mouse sharing connection can refer to the following Fig. 13B The process of establishing a keyboard and mouse sharing connection first and then establishing a collaborative connection can refer to the following Fig.13A No further details will be given here.

[0182] S602: After the keyboard and mouse sharing connection is established, the second device synchronizes collaborative window information with the first device.

[0183] The collaborative window information includes, but is not limited to: window visibility information, window shape information, mode information, window bounds information, etc.

[0184] The window visibility information is used to indicate whether the window is visible and displayed on the display screen.

[0185] The window form information is used to indicate whether the window is in a floating form. In the embodiment of the present application, when the collaborative window is in full screen state, the collaborative window information also includes: information such as the display direction of the function area and the range of the function area. When the collaborative window is in a floating window state, the collaborative window information also includes information such as the zoom ratio and the range of the function bar.

[0186] The mode information is used to indicate the tablet collaboration mode, for example, mirror mode and extended mode.

[0187] The window range information is used to indicate the range of the window on the display screen. The window range information includes but is not limited to: the range of the window display area, the range of the function area, etc.

[0188] S603: When displaying the first identifier, the first device receives a move operation for the first identifier.

[0189] The first identifier is an identifier representing an input device in the display area of ​​the first device.

[0190] S604: When the first identifier moves to the edge of the first device, the first device sends a crossing request to the second device. The crossing request includes: location information corresponding to the first identifier.

[0191] The position information corresponding to the first identifier may be a scaled coordinate of the first identifier, and the scaled coordinate of the first identifier and the first coordinate of the first device have a first proportional relationship.

[0192] In some embodiments, the position information of the first identifier may also be the first coordinates of the first identifier on the first device and the display screen size of the first device. Adaptively, the second device may confirm the proportional coordinates of the first identifier based on the display screen size of the first device, the coordinates of the first identifier on the first device, and the display screen size of the second device.

[0193] S605: When the position of the first mark is within the first area, the second device sends a message to the first device indicating that the crossing failed. The message includes: a third proportional relationship.

[0194] S606: The first device confirms the first landing point coordinates of the first marker based on the third proportional relationship.

[0195] When the scale coordinates of the first marker are within the range coordinates of the first area, the second device confirms the landing point coordinates of the first marker based on the third scale relationship.

[0196] The calculation method of the landing point coordinates can refer to the corresponding instructions above and will not be repeated here.

[0197] S607: The first device synchronizes the first landing point coordinates of the first identifier and part of the content of the first device to the second device.

[0198] Adaptively, the second device displays the first identifier and part of the content of the first device in the display area of ​​the collaborative window.

[0199] Specifically, the first device synchronizes the first landing point coordinates of the first identifier and part of the content of the first device to the collaborative application of the second device.

[0200] S608: When the scale coordinates of the first marker are not located in the first area, the second device displays the second marker based on the position information corresponding to the first marker.

[0201] When the position information corresponding to the first marker is the first scaled coordinates of the first marker, the second landing point coordinates corresponding to the second marker are the scaled coordinates of the first marker.

[0202] In this way, the identifier of the input device can be transferred from the first device to the display area of ​​the second device except the display area of ​​the collaborative window, which facilitates the user to operate the second device through the input device and improves the user experience.

[0203] On the basis of the above embodiment, when the collaborative window is in full screen mode and the function area is hidden, the first device may not execute S604-S606. The first device may directly execute S607 when transmitting to the second device.

[0204] For example, Fig. 7A This is a schematic diagram of an application scenario provided by an embodiment of the present application. Taking the function area direction as the left side as an example, Fig. 7A As shown, the collaborative window is displayed in full screen on the second device, and the function area is located on the left side of the collaborative function window. The range of the first area is: (W, 0, m0, n0).

[0205] When the first device receives an operation to move the first mark to the right edge, and the first mark moves to the right edge of the first device, the first device sends a crossing request to the second device, and the crossing request includes: a first scale coordinate. For example, the coordinates of the first mark at the right edge are (x0, y). The first scale coordinates are (0, ). Since (0, ) is not located in the first area, and the second device displays the second mark 302 in the second area based on the first scale coordinates.

[0206] For example, Figure 7B This is a schematic diagram of an application scenario provided by an embodiment of the present application. Taking the function area direction as the left side as an example, Figure 7B As shown, the collaborative window is displayed in full screen on the second device, and the function area is hidden and not displayed. The range of the first area is: (0, 0, m0, n0).

[0207] In a possible implementation method 1, when the first device receives an operation to move the first mark to the right edge, and the first mark moves to the right edge of the first device, the first device sends a crossing request to the second device. The crossing request includes: a first scale coordinate. For example, the coordinates of the first mark at the right edge are (x0, y), the first scale coordinates are (radioX, radioY), radioX = 0,

[0208] Since (0, ) is located in the first area, and the second device sends a message to the first device indicating that the crossing failed. The message includes: a third proportional relationship. The third proportional relationship is:

[0209] The first landing point coordinates determined by the first device based on the third proportional relationship are ( ); Since the coordinates of the upper left corner of the collaborative window are (0, 0), m1 = 0, n1 = 0; the coordinates of the first landing point are (0, ).

[0210] The second device displays the first mark 301 based on the first landing point coordinates. The position corresponding to the first mark 301 is (0, ).

[0211] In the second possible implementation, since the collaborative window is displayed in full screen on the second device and the function area is hidden, the range of the first area is: (0, 0, m0, n0). When the first device receives an operation to move the first identifier to the right edge, and the first identifier moves to the right edge of the first device, the confirmed first landing point coordinates are (0, ). The second device displays the first mark 301 based on the first landing point coordinates. The position corresponding to the first mark 301 is (0, ).

[0212] For example, Figure 7C and Fig.7D A schematic diagram of an application scenario provided for an embodiment of the present application.

[0213] like Figure 7C As shown, the second device displays a collaborative window and a first window. Specifically, when the first identifier 301 moves to the position corresponding to the line segment 702 on the right edge of the first device, the first device sends a crossing request to the second device. The crossing request includes: a first scale coordinate. The second device confirms that a crossing occurs based on the first scale coordinate and the range of the first area. The second device confirms that the landing coordinates of the first identifier are the first scale coordinates. The second device displays the second identifier 302 based on the first scale coordinates.

[0214] like Fig.7DAs shown, the second device displays the collaborative window and the first window. Specifically, when the first identifier 301 moves to the position corresponding to the right edge line segment 701 of the first device, the first device sends a crossing request to the second device. The crossing request includes: scale coordinates. For example, the coordinates of the first identifier on the right edge are (x0, y), the scale coordinates are (radioX, radioY), radioX = 0,

[0215] Since (0, ) is located in the first area, and the second device sends a message to the first device indicating that the crossing failed. The message includes: a third proportional relationship. The third proportional relationship is:

[0216] The first landing point coordinates determined by the first device based on the third proportional relationship are ( ); Since the coordinates of the upper left corner of the collaborative window are (0, 0), m1 = 0, n1 = 0; the coordinates of the first landing point are (0, ).

[0217] The second device displays the first mark 301 based on the first landing point coordinates. The position corresponding to the first mark 301 is (0, ).

[0218] For example, Figure 8 The following is a flow chart of an interface display method provided in an embodiment of the present application. Figure 8 As shown, the interface display method includes:

[0219] S801. A first device and a second device establish a collaborative connection and a keyboard and mouse sharing connection.

[0220] S802. The second device synchronizes collaborative window information with the first device through keyboard and mouse sharing.

[0221] The collaborative window information includes but is not limited to: the form of the collaborative window, the scale coordinates of the first area. When the collaborative window is in full screen mode, the collaborative window information also includes the direction of the functional area. When the collaborative window is in floating mode, the collaborative window information also includes: the location information of the first area.

[0222] S803: When the second device displays the first identifier, a move operation for the first identifier is received.

[0223] S804: When the first identifier moves to the first edge of the first area, the first device sends a crossing request to the second device, the crossing request including: second scale coordinates, the second scale coordinates are determined based on the third scale relationship and the coordinates of the edge of the first identifier in the first area. The first edge is the edge where the first area contacts the second area in the first area.

[0224] S805: The second device displays a second identifier based on the second scale coordinates.

[0225] S806. When the first identifier moves to the second edge of the first area, the first device displays the first identifier. The position of the first identifier is confirmed based on the third proportional relationship and the coordinates of the first identifier at the second edge. The second edge is an edge of the first area in the direction of connection between the second area and the first area. Alternatively, the second edge can also be understood as an edge that is cut off by the edge of the display area of ​​the second device in the direction of the functional area, or can be understood as an edge that does not contact the second area.

[0226] The preset edge is the edge of the first device in the connection direction between the first device and the second device. When the collaborative connection is established, if the logo of the second device is located to the right of the logo of the first device, the preset edge of the first device is the right edge; when the collaborative connection is established, if the logo of the second device is located to the left of the logo of the first device, the preset edge of the first device is the left edge.

[0227] In an embodiment of the present application, when the collaborative window is in full-screen form, the first edge can also be understood as the edge corresponding to the direction of the functional area. In some embodiments, when the collaborative window is in full-screen form, the second device synchronizes the direction corresponding to the first edge to the first device. Exemplarily, the second device synchronizes a byte type parameter to the first device. 1 indicates that the direction can be crossed, and 0 indicates that the direction cannot be crossed. Bit0: indicates that the lower direction of the first area can be crossed; bit1: indicates that the right direction of the first area can be crossed; bit2: indicates that the upper direction of the first area can be crossed; bit3: indicates that the left direction of the first area can be crossed. For example, for example, the byte type binary number sent by the second device to the first device is: 0000 0100, which means that only the upper direction of the first area can be crossed.

[0228] In this way, the input device can be moved between the first area and the second area in the second device, which is convenient for the user to operate the second device through the input device.

[0229] For example, Fig.9A As shown, the four edges of the first area are all first edges. If the second device is Fig.9A The interface shown receives a move operation for the first identifier 301. When the first identifier 301 moves to the edge of the first area, the first device sends a crossing request to the second device, and the crossing request includes the second scale coordinates. For example, the coordinates of the first identifier 301 at the edge of the first area are (m, n), and the second scale coordinates are ( ). The coordinates of the second mark 302 displayed by the second device in the second area based on the second scale coordinates are ( ).

[0230] For example, Fig. 9B As shown, the left edge of the first area is the second edge, and the upper edge of the first area, the lower edge of the first area, and the right edge of the first area are all the first edge.

[0231] If the second device receives a moving operation for the first mark 301, when the first mark 301 moves to the left edge of the first area, the first device displays the first mark. For example, the coordinates of the first mark 301 at the edge of the first area are (m, n), m = 0, m1 = 0, and the coordinates of the first mark are (x0, ).

[0232] If the second device receives a moving operation for the first identifier 301. When the first identifier 301 moves to the upper edge of the first area, the lower edge of the first area, or the right edge of the first area, the second device displays the second identifier. Taking the coordinates of the first identifier 301 at the edge of the first area as (m, n), the first device sends a crossing request to the second device, and the crossing request includes the second scale coordinates. Taking the coordinates of the first identifier 301 at the edge of the first area as (m, n) as an example, the second scale coordinates are ( ). The coordinates of the second mark 302 displayed by the second device in the second area based on the second scale coordinates are ( ).

[0233] Above Figure 8 In the illustrated embodiment, the first device determines whether to cross the edge of the first area, thereby realizing the display of the identifier of the input device.

[0234] In some other embodiments, the second device may confirm whether the crossing occurs based on the second scale coordinates, where the second scale coordinates are scale coordinates corresponding to the first landing point coordinates. The first landing point coordinates are confirmed based on the second scale coordinates and the third scale relationship.

[0235] Exemplarily, the first device and the second device may not execute S803-S806. When the first identifier moves to the edge of the first area, the first device sends a crossing request to the second device, and the crossing request includes: a second scale coordinate, and the second scale coordinate is a scale coordinate corresponding to the first landing point coordinate. The first landing point coordinate is confirmed based on the second scale coordinate and the third scale relationship. When the second scale coordinate is located in the second area, the second device displays the second identifier based on the second scale coordinate. S806. When the second scale coordinate is located in the first area, the second device sends a message to the first device indicating that the crossing failed. S807. The first device displays the first identifier. The position of the first identifier is confirmed based on the first scale relationship and the second scale coordinate.

[0236] In this way, the input device can also be moved between the first area and the second area in the second device, so that the user can operate the second device through the input device.

[0237] For example, Fig.9A As shown, the second device Fig.9A The interface shown receives a move operation for the first identifier 301. When the first identifier 301 moves to the edge of the first area, the first device sends a crossing request to the second device, and the crossing request includes the second scale coordinates. For example, the coordinates of the first identifier 301 at the edge of the first area are (m, n), and the second scale coordinates are ( ).

[0238] When the second scale coordinate is not located in the first area, the second device confirms that the crossing occurs, and the second device displays the coordinates of the second mark 302 in the second area based on the second scale coordinate as ( ).

[0239] For example, Fig. 9B As shown, the second device receives a moving operation for the first identifier 301. When the first identifier 301 moves to the edge of the first area, the first device sends a crossing request to the second device, and the crossing request includes a second scale coordinate.

[0240] Taking the coordinates of the first mark 301 at the edge of the first area as (m, n), m=0, m1=0 as an example, the second proportional coordinates are ( ), the coordinates of the first mark on the first device are (x0, ).

[0241] When the second scale coordinate is within the first area, the second device confirms that no crossing has occurred. The second device sends a message to the first device indicating that the crossing has failed. The second device confirms that the coordinates of the first identifier 301 are (x0, ).

[0242] For example, Fig.10 The following is a flow chart of an interface display method provided in an embodiment of the present application. Fig.10 As shown, the interface display method includes:

[0243] S1001. A first device and a second device establish a collaborative connection and a keyboard and mouse sharing connection.

[0244] S1002. The second device synchronizes collaborative window information with the first device through keyboard and mouse sharing.

[0245] The collaborative window information includes but is not limited to: the form of the collaborative window, the scale coordinates of the first area. When the collaborative window is in full screen mode, the collaborative window information also includes the direction of the functional area. When the collaborative window is in floating mode, the collaborative window information also includes: the location information of the first area.

[0246] S1003: When the second device displays the second identifier, a move operation for the second identifier is received.

[0247] S1004: When the second identifier moves to the third edge of the second area, the second device sends a crossing request to the first device, where the crossing request includes a third coordinate, which is the coordinate of the second identifier at the third edge of the second area.

[0248] The third edge is an edge in the second area in the connection direction between the second device and the first device. Alternatively, the third edge can also be understood as an edge intercepted by the edge of the display area of ​​the second device in the connection direction, or can be understood as an edge that does not contact the first area.

[0249] S1005: The first device displays a first identifier based on the third coordinate and the first proportional relationship.

[0250] S1006. When the second identifier moves to the fourth edge of the second area, the second device sends a crossing request to the first device, and the crossing request includes the third landing point coordinates. The third landing point coordinates are determined based on the third proportional relationship and the coordinates of the second identifier at the fourth edge of the second area. The fourth edge is the edge of the second area in the second area that contacts the first area.

[0251] S1007. The first device synchronizes the third landing point coordinates and part of the content of the first device to the second device.

[0252] S1008: The second device displays the first logo and part of the content of the first device in the first area.

[0253] In this way, the identifier of the input device can also be moved from the second area of ​​the second device to the first area or the display area of ​​the first device.

[0254] Above Fig.10 In the illustrated embodiment, the second device transmits different crossing requests to the first device based on the edge of the second area, thereby enabling display of the identification of the corresponding input device.

[0255] For example, Fig.11A As shown, the second device Fig.11AThe interface shown receives a moving operation for the second identifier 302. When the second identifier 302 moves to the fourth edge of the second area, the first device sends a crossing request to the second device, and the crossing request includes the third landing point coordinates. Taking the coordinates of the second identifier as (m, n) as an example, the third landing point coordinates are ( ).

[0256] For example, Fig. 11B As shown, the second device receives a moving operation for the second identifier 302. When the second identifier 302 moves to the third edge of the second area, the second device sends a crossing request to the first device, and the crossing request includes the third coordinates. For example, the coordinates of the first identifier 301 at the third edge of the second area are (m, n), m=0, m1=0, and the coordinates of the first identifier on the first device are (x0, ).

[0257] When the second identifier 302 moves to the fourth edge of the second area, the second device sends a crossing request to the first device, and the crossing request includes the third landing point coordinates. Taking the coordinates of the first identifier 301 at the third edge of the second area as (m, n), m=0, m1=0 as an example, taking the coordinates of the first identifier 301 at the edge of the second area as (m, n), m=0, m1=0 as an example, the third landing point coordinates are (x0, ).

[0258] In some other embodiments, the second device may transmit the third scale coordinate to the first device, and the first device confirms the display of the identifier of the corresponding input device based on the position of the third scale coordinate.

[0259] Exemplarily, the first device and the second device may not execute S1004-S1008. When the second identifier moves to the edge of the second area, the second device sends a crossing request to the first device, and the crossing request includes a third scale coordinate. The third scale coordinate is confirmed based on the third scale relationship and the coordinates of the second identifier at the edge of the second area. When the third scale coordinate is located in the first area, the first device sends the third scale coordinate to the second device. The second device displays the first identifier in the first area based on the third scale coordinate. When the third scale coordinate is not located in the first area, the first device displays the first identifier. The position of the first identifier is determined based on the third scale coordinate and the first scale relationship.

[0260] In this way, the identifier of the input device may also be moved from the second area of ​​the second device to the first area or the display area of ​​the first device.

[0261] For example, Fig.11A As shown, the second device Fig.11AThe interface shown receives a moving operation for the second identifier 302. When the second identifier 302 moves to the edge of the second area, the first device sends a crossing request to the second device, and the crossing request includes a third scale coordinate. For example, the coordinates of the second identifier are (m, n), and the third scale coordinates are ( );

[0262] When the third scale coordinate is located in the first area, the second device displays the first mark 302 in the first area based on the third scale coordinate. The coordinates of the first mark 302 are ( ).

[0263] For example, Fig. 11B As shown, the second device receives a moving operation for the second identifier 302. When the second identifier 302 moves to the edge of the first area, the second device sends a crossing request to the first device, and the crossing request includes a third scale coordinate.

[0264] Taking the coordinates of the first mark 301 at the edge of the second area as (m, n), m=0, m1=0 as an example, the third proportional coordinates are ( ), the coordinates of the first mark on the first device are (x0, ).

[0265] When the third scale coordinate is not within the first area, the first device displays a first mark, which is confirmed based on the third scale coordinate, the third scale relationship and the first scale relationship. The coordinates of the first mark 301 are (x0, ).

[0266] In the above embodiment, the content displayed in the first area is determined by the first device, and the position of the first mark in the first area is determined by the first device. Alternatively, it can be understood that the first area is the secondary screen of the first device. The position of the second mark in the second area of ​​the second device is confirmed by the second device.

[0267] For ease of understanding, the crossing scenario involved in the embodiments of the present application is described below.

[0268] Table 1 Crossing rules table

[0269]

[0270] When a crossing occurs, the coordinates of the landing point are calculated as follows:

[0271] When the input device passes from the display area of ​​the collaborative window of the second device to outside the display area of ​​the collaborative window of the second device, the landing point coordinates can be calculated as follows: the first device calculates the proportional coordinates based on the first proportional relationship; the second device confirms the length of the collaborative window occupied based on the proportional coordinates and the third proportional relationship, and then confirms the landing point coordinates based on the coordinates of the upper left corner of the collaborative window.

[0272] Exemplarily, the abscissa calculation: the first device first performs a proportional conversion: The second device receives the scale coordinates and calculates the width of the occupied collaborative window: Therefore, the final landing point coordinates are: Calculation of vertical coordinate: The first device performs proportional conversion: The second device receives the scale coordinates and calculates the height of the occupied collaborative window: Therefore, the final landing point coordinates are:

[0273] When the input device passes from an area outside the display area of ​​the collaborative window to the display area of ​​the collaborative window, the second device confirms the length of the collaborative window occupied based on the coordinates of the upper left corner of the collaborative window, and then confirms the proportional coordinates based on the third proportional relationship and sends them to the first device. The first device confirms the landing point coordinates based on the first proportional relationship.

[0274] Exemplarily, the second device first calculates the occupied collaborative window width: L1=m-m1, and then calculates the screen ratio: The radioX is sent to the first device, and the first device converts according to the first proportional relationship to calculate the coordinates of the landing point. The second device first calculates the occupied collaborative window height: h1 = n-h1, and then calculates the screen ratio: The radioX is sent to the first device, and the first device calculates the coordinates of the landing point according to the screen ratio conversion (first ratio relationship)

[0275] In the above embodiment, the second device determines whether the identifier of the input device crosses. In some embodiments, the first device may also determine whether the identifier of the input device crosses based on the collaborative window information.

[0276] In the above embodiment, the first device confirms the coordinates of the landing point in the first area, and the second device confirms the coordinates of the landing point in the second area. Since the second device synchronizes the collaborative window information to the first device, the coordinates of the landing point in the first area can also be confirmed by the second device, and the coordinates of the landing point in the second area can also be confirmed by the first device. The specific process is similar to the process of the above method, and will not be repeated here.

[0277] For example, Fig.12 A schematic diagram of the software architecture provided for an embodiment of the present application. Fig.12 A schematic diagram of the software architecture of a first device and a second device provided in an embodiment of the present application. Fig.12 As shown, the software architecture of the first device includes: an application layer 1a and a transport layer 2a.

[0278] The application layer 1a includes: PC Manager, which is responsible for the creation and management of virtual screens, position detection of input devices (eg, mouse), calculation of crossing areas, calculation of crossing coordinates, and the like.

[0279] The transport layer 2a is used for data transmission to perform security authentication, device discovery, data transmission, etc. when the device is connected.

[0280] The software architecture of the second device includes: application layer 1b, framework layer and transport layer 2b. Application layer 1b includes: keyboard and mouse sharing application and tablet collaboration application.

[0281] The keyboard and mouse sharing application is used for position detection, crossing area calculation, crossing coordinate calculation, callback registration and monitoring of an input device (eg, a mouse).

[0282] The tablet collaborative application is used to adjust the collaborative window's form, position, size, etc., and to render the collaborative window. In the embodiment of the present application, the tablet collaborative application can also monitor the collaborative window form switching and the function area direction switching, and transmit the switched collaborative window information to the keyboard and mouse sharing application.

[0283] In the embodiment of the present application, after the first device and the second device establish a keyboard and mouse sharing connection and a collaborative connection, the keyboard and mouse sharing application can register a callback with the window manager to monitor the window displayed by the terminal device. When the window displayed by the terminal device changes, the keyboard and mouse sharing application can update the area in the collaborative window used to display part of the content of the first device to facilitate subsequent input device traversal calculations.

[0284] Changes in the window displayed on the terminal device include, but are not limited to: changes in the size of the coordinated window, movement of the position, truncation, obstruction, etc.

[0285] The keyboard and mouse sharing application can monitor the framework side window callback to handle the scaling, moving, truncation, and occlusion of the collaborative window when it is in a floating state.

[0286] The framework layer is used for the callback management module to report the mouse position in real time, to callback the scaling and movement of the collaborative window when it is in a suspended state, to cover the collaborative window, and to change the visibility of the collaborative window. The business side receives the callback and performs corresponding processing.

[0287] The transport layer 2b is used to achieve communication with other devices. The transport layer 2b includes modules such as identity authentication, device discovery, device connection, and data transmission.

[0288] The above embodiment is implemented on the basis of establishing a keyboard and mouse sharing connection and establishing a collaborative connection. Fig.13A and Fig. 13B The interaction between the first device and the second device when establishing a connection is described.

[0289] Fig.13A A schematic diagram of an interaction process corresponding to first establishing a keyboard and mouse sharing connection and then establishing a collaborative connection is provided in an embodiment of the present application.

[0290] S1601: After a keyboard and mouse sharing connection is established, a first device receives an operation for instructing to establish a collaborative connection, and the first device establishes a collaborative connection with a second device.

[0291] In the embodiment of the present application, the user can drag any window to the second device to establish a collaborative connection, or can use the above Figure 1B The process shown establishes a collaborative connection and is not limited here.

[0292] S1602: The tablet collaborative application of the second device queries the keyboard and mouse sharing application of the second device whether a keyboard and mouse sharing connection has been established.

[0293] Exemplarily, the tablet collaborative application transmits a message for querying the keyboard and mouse sharing connection to the keyboard and mouse sharing application. When the keyboard and mouse sharing connection has been established, the keyboard and mouse sharing application transmits a message for indicating that the sharing connection has been established to the tablet collaborative application.

[0294] S1603: When the keyboard and mouse sharing connection is established, the tablet collaborative application synchronizes the first collaborative window information with the keyboard and mouse sharing application.

[0295] The first collaborative window information includes, but is not limited to: window visibility information, window shape information, mode information, window bounds information, etc.

[0296] The window visibility information is used to indicate whether the window is visible and displayed on the display screen.

[0297] The window form information is used to indicate whether the window is in a floating form. In the embodiment of the present application, when the collaborative window is in a full-screen state, the collaborative window information also includes information such as the display direction of the function area and the range of the function area.

[0298] When the collaborative window is in a floating window state, the collaborative window information also includes the zoom ratio and the range information of the function area.

[0299] The mode information is used to indicate the tablet collaboration mode, for example, mirror mode and extended mode.

[0300] The window range information is used to indicate the range size of the window on the display screen. The window range information includes but is not limited to: the area coordinates of the window display area, the area coordinates of the function area, etc.

[0301] S1604: The keyboard and mouse sharing application confirms the scope of the collaborative window based on the first collaborative window information.

[0302] Specifically, the keyboard and mouse sharing application calculates the bounds of the collaborative window on the actual physical screen surface. When in suspended form, the collaborative window will be scaled, and the bounds of the collaborative window displayed on the second device are calculated based on the scaling ratio and the initial bounds.

[0303] In the suspended state, the keyboard and mouse sharing application calculates whether the collaborative window exceeds the screen boundary. If it exceeds the boundary, it can be regarded as a truncated scene and the truncation area can be calculated.

[0304] S1605. The keyboard and mouse sharing application transmits the second collaborative window information to the PC manager of the second device.

[0305] The second collaborative window information includes, but is not limited to: window visibility information, window status information, window truncation information, window truncation area range (bounds) information, blocked window information, etc.

[0306] Fig. 13B The following is a schematic diagram of an interactive process corresponding to first establishing a collaborative connection and then establishing a keyboard and mouse sharing connection provided in an embodiment of the present application. Fig. 13B As shown, the interaction process includes:

[0307] S1611. PC Manager transmits a message indicating a keyboard and mouse sharing connection to a keyboard and mouse sharing application of a second device.

[0308] S1612: The keyboard and mouse sharing application queries the tablet collaborative application for collaborative connection window information.

[0309] S1613: The tablet collaborative application synchronizes the first collaborative window information with the keyboard and mouse sharing application.

[0310] S1614: The keyboard and mouse sharing application confirms the scope of the collaborative window based on the first collaborative window information.

[0311] S1615. The keyboard and mouse sharing application transmits the second collaborative window information to the PC manager of the second device.

[0312] The description of S1613-S515 can refer to the above S1603-S1605, and will not be repeated here in detail.

[0313] On the basis of the above embodiment, when the collaborative window changes, the second device will synchronize the collaborative window information to the first device to facilitate the subsequent confirmation of the landing coordinates of the identifier of the input device.

[0314] It is understandable that when the collaborative window changes, such as sidebar direction switching, form switching, function area hiding, collaborative window position movement, window size adjustment, and occlusion area changes, the second device will reconfirm the scope of the collaborative window and update the second collaborative window information to the first device.

[0315] Combine the following Figure 14-16 The interactive processes corresponding to sidebar direction switching, form switching, window movement and scaling are explained.

[0316] For example, Fig.14 A schematic diagram of an interactive process corresponding to a sidebar direction switch provided in an embodiment of the present application. Fig.14 As shown, taking the sidebar direction change as an example, the interaction process includes:

[0317] S1701: When the first device receives an operation for instructing to change the sidebar direction, PC Manager transmits a message for instructing to change the sidebar direction to the tablet collaborative application of the second device.

[0318] S1702: When the keyboard and mouse sharing connection is established, the tablet collaborative application transmits the adjusted first collaborative window information to the keyboard and mouse sharing application.

[0319] S1703: The keyboard and mouse sharing application confirms the range of the collaborative window based on the adjusted first collaborative window information.

[0320] S1704. The keyboard and mouse sharing application transmits the adjusted second collaborative window information to the PC manager of the second device.

[0321] For example, Fig.15A and Fig. 15B The following is a schematic diagram of the interaction process corresponding to the form switching provided in the embodiment of the present application. Fig.15A As shown in the figure, taking the switching from full screen mode to floating mode as an example, the interaction process includes:

[0322] S1901: When the second device receives an operation for instructing switching to a suspended state, the tablet collaborative application transmits a message for instructing switching to a suspended state to the framework layer of the second device.

[0323] S1902: When the keyboard and mouse sharing connection is established, the framework layer transmits the scaling ratio of the collaborative window to the tablet collaborative application.

[0324] S1903: The tablet collaborative application synchronizes the first collaborative window information corresponding to the suspended state to the keyboard and mouse sharing application.

[0325] S1904: The keyboard and mouse sharing application confirms the scope of the collaborative window based on the first collaborative window information in the suspended state.

[0326] S1905: The keyboard and mouse sharing application transmits the second collaborative window information in a suspended state to the PC manager of the second device.

[0327] Combine the following Fig. 15B The interactive process of switching from floating mode to full screen mode is described. Fig. 15B As shown, the interaction process includes:

[0328] S1911: When the second device receives an operation for instructing switching to a full-screen mode, the tablet collaborative application transmits a message for instructing switching to a full-screen mode to the framework layer of the second device.

[0329] S1912: When the keyboard and mouse sharing connection is established, the framework layer transmits the scaling ratio of the collaborative window to the tablet collaborative application.

[0330] S1913. The tablet collaborative application synchronizes first collaborative window information corresponding to the full-screen mode to the keyboard and mouse sharing application.

[0331] S1914: The keyboard and mouse sharing application confirms the range of the collaborative window based on the first collaborative window information in full-screen form.

[0332] S1915. The keyboard and mouse sharing application transmits the second collaborative window information in full screen format to the PC manager of the second device.

[0333] Combine the following Fig.16 The interactive process corresponding to the movement and scaling of the collaborative window is described. Fig.16 As shown, the interaction process includes: the keyboard and mouse sharing connection registers the collaborative window monitoring with the framework layer. When the collaborative window changes position or size, the framework layer updates the scaling ratio and the position information of the collaborative window to the keyboard and mouse sharing application.

[0334] S2001. The keyboard and mouse sharing application registers a collaborative window listener with the framework layer to monitor the position change and / or size change of the collaborative window.

[0335] S2002: When the second device receives an operation for indicating a position change or a size change of a window, the tablet collaborative application transmits a message for indicating a position change or a size change to the framework layer of the second device.

[0336] S2003. When the position changes, the framework layer transmits the position information of the collaborative window to the tablet collaborative application; when the size changes, the framework layer transmits the scaling ratio of the collaborative window to the tablet collaborative application.

[0337] S2004. The tablet collaborative application synchronizes the first collaborative window information after the position change or the first collaborative window information after the size change to the keyboard and mouse sharing application.

[0338] S2005. The keyboard and mouse sharing application confirms the range of the collaborative window after the position change based on the first collaborative window information after the position change, or confirms the range of the collaborative window after the size change based on the first collaborative window information after the size change.

[0339] S2006. The keyboard and mouse sharing application transmits the second collaborative window information after the synchronization position change or the second collaborative window information after the size change to the PC manager of the second device.

[0340] It is understandable that when the functional area is hidden or displayed in full-screen mode, the second device will also update the collaborative window information to the first device.

[0341] On the basis of the above embodiment, the keyboard and mouse sharing application can also register a window callback with the framework layer to monitor the occlusion of the collaborative window by the second window, thereby facilitating the subsequent confirmation of the scope of the first area. When refreshing the window, the framework layer traverses all windows in the device to obtain the second window, which is a window with a window level above the collaborative window. And calculate the intersection area of ​​each second window and the collaborative window. The framework layer calls back the intersection area of ​​each second window and the collaborative window to the keyboard and mouse sharing application. The keyboard and mouse sharing application updates the occlusion area and synchronizes the occlusion area with the first device.

[0342] In some embodiments, the second device stores the intersection area of ​​each second window and the collaborative window in a list. When the window does not block the collaborative window, the intersection area of ​​the window and the collaborative window in the list is deleted.

[0343] The framework layer calls back the list to the keyboard and mouse sharing application. The keyboard and mouse sharing application updates the list and synchronizes the list with the first device.

[0344] The second device can add and delete items from the list based on the identifier of the window (eg, a hash value of the window).

[0345] The method of the embodiment of the present application has been described above, and the device for executing the above method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above method.

[0346] The method provided in the embodiment of the present application can be applied to an electronic device with a display function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the above related description, which will not be repeated here.

[0347] An embodiment of the present application provides a terminal device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the above method.

[0348] For example, Fig.17 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Fig.17 As shown, the terminal device includes: the terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0349] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0350] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0351] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0352] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0353] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0354] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the terminal device. In other embodiments of the present application, the terminal device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0355] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of a terminal device. While the charging management module 140 is charging the battery 142, it can also power the electronic device through the power management module 141.

[0356] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0357] The terminal device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0358] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0359] The terminal device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0360] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0361] The camera 193 is used to capture static images or videos. In some embodiments, the terminal device may include 1 or N cameras 193, where N is a positive integer greater than 1. The types of the multiple cameras 193 may be different. For example, the camera 193 may include a camera for acquiring color images or a TOF camera, etc. In the embodiment of the present application, the camera may be understood as a camera. The camera includes: an AO camera, a TOF camera, etc. in the embodiment of the present application.

[0362] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0363] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 can include a program storage area and a data storage area.

[0364] The terminal device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0365] The button 190 includes a power button, a volume button, etc. The button 190 may be a mechanical button. It may also be a touch button. The terminal device may receive the button input and generate a key signal input related to the user settings and function control of the terminal device. The indicator 192 may be an indicator light, which may be used to indicate the charging status, power change, message, missed call, notification, etc.

[0366] In addition, on top of the above components, the device also runs an operating system, such as an iOS operating system, an Android operating system, or a Windows operating system, etc. Applications can be installed and run on the operating system.

[0367] The software system of the terminal device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture, etc., which will not be elaborated here.

[0368] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0369] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0370] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0371] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0372] The present application embodiment is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the present application embodiment. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow chart and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart or multiple flows and / or one box or multiple boxes of the block chart.

[0373] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0374] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An interface display method, characterized in that: Applied to a collaborative system, the collaborative system includes a first device and a second device, and the method includes: When the first device displays a first identifier, the first device receives a first movement operation for the first identifier, where the first identifier is an identifier of an input device connected to the first device; When the first identifier moves to a preset edge of the first device, the first device sends a first crossing request to the second device, where the first crossing request includes: first position information corresponding to the first identifier; the preset edge is an edge of the first device in a direction in which the first device and the second device are connected, and the collaborative window is a window in the second device for displaying part of the content of the first device; When the position of the first identifier in the first position information is located in the first area, the second device sends a message to the first device indicating a crossing failure; the message indicating a crossing failure includes: a third proportional relationship; the first area is an area in the second device that displays part of the content of the first device, and the third proportional relationship is a relationship between the size of the display area of ​​the second device and the size of the collaborative window displayed by the second device; After receiving the message indicating the crossing failure, the first device synchronizes the first landing point coordinates of the first identifier on the second device to the second device, where the first landing point coordinates are confirmed based on the first position information and the third proportional relationship; The second device displays the first mark in the first area based on the first landing point coordinates; Alternatively, when the location of the first identifier is within the second area, the second device displays a second identifier based on the first location information; The second area is a display area of ​​the second device except the first area, and the second identifier is an identifier of the input device.

2. The method according to claim 1, characterized in that The first position information includes: first proportional coordinates of the first mark, the first proportional coordinates are determined based on coordinates of the first mark at the preset edge and a first proportional relationship; the first proportional relationship is a relationship between a size of a display area of ​​the first device and a size of a display area of ​​the second device; Alternatively, the first position information includes: the first proportional relationship and the coordinates of the first mark at the preset edge.

3. The method according to claim 2, characterized in that The first proportional relationship satisfies: The x is the horizontal coordinate of the first identifier on the first device, the y is the vertical coordinate of the first identifier on the first device, the x0 is the length of the first device in the horizontal direction; the y0 is the length of the first device in the horizontal direction, the radioX is the horizontal coordinate in the proportional coordinate of the first identifier, the radioY is the vertical coordinate in the proportional coordinate of the first identifier, the m0 is the length of the second device in the horizontal direction; and the n0 is the length of the second device in the horizontal direction.

4. The method according to claim 3, characterized in that The third proportional relationship satisfies: L is the length of the collaborative window in the horizontal direction, h is the length of the collaborative window in the vertical direction, L1 is the distance between the landing point of the first identifier and the edge of the collaborative window in the vertical direction, and h1 is the distance between the landing point of the first identifier and the edge of the collaborative window in the horizontal direction.

5. The method according to claim 4, characterized in that The L satisfies: the L=m-m1, the h1 satisfies: the h1=n-m1, The m is the horizontal coordinate of the landing point of the first mark, and the n is the vertical coordinate of the landing point of the first mark; The m1 is the horizontal coordinate corresponding to the first corner of the collaborative window, and the n1 is the vertical coordinate corresponding to the first corner.

6. The method according to any one of claims 3 to 5, characterized in that: If the coordinates of the first mark at the preset edge are (x0, y) or (0, y), the ordinate of the first scale coordinate is When the first scale coordinate is located in the first area, the ordinate of the first landing point coordinate is When the first scale coordinate is located in the second area, the ordinate of the first landing point coordinate is 7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The first device receives a second movement operation on the first identifier displayed in the first area; When the first identifier moves to a first edge of the first area, the first device sends a second crossing request to the second device, where the second crossing request includes: second scale coordinates; the second scale coordinates are determined by the first device based on the third scale relationship and the coordinates of the first identifier at the edge of the first area; the first edge is an edge of the first area where the first area contacts the second area; The second device displays the second identifier based on the second scale coordinates; Alternatively, when the first identifier moves to the second edge of the first area, the first device displays the first identifier based on the third proportional relationship, the first proportional relationship and the coordinates of the first identifier at the second edge, and the second edge is the edge of the first area in the direction connecting the second area and the first area.

8. The method according to claim 7, characterized in that If the coordinates of the first mark at the first edge are (m2, n2), the second scale coordinates are Alternatively, if the coordinates of the first mark at the second edge are (m3, n3), the vertical coordinate of the first mark displayed by the first device is:

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: When the second device displays the second mark in the second area, a third movement operation on the second mark is received; When the second identifier moves to a third edge of the second area, the second device sends a third crossing request to the first device, the third crossing request including: coordinates of the second identifier at the third edge; the third edge is an edge in the second area in a direction in which the second device and the first device are connected; The first device displays the first mark based on the first proportional relationship and the coordinates of the second mark at the third edge; Alternatively, when the second identifier moves to a fourth edge of the second area, the second device sends a fourth crossing request to the first device, the fourth crossing request including third landing point coordinates, the third landing point coordinates being determined based on the third proportional relationship and the coordinates of the second identifier at the fourth edge; the fourth edge being an edge of the second area where the second area contacts the first area; The first device synchronizes the third landing point coordinates and part of the content of the first device to the second device; The second device displays part of the content of the first device in the first area, and displays the first logo based on the third landing point coordinates.

10. The method according to any one of claims 1 to 9, characterized in that: If the coordinates of the second mark at the third edge are (m4, n4), the vertical coordinate of the first mark determined by the first device is If the coordinates of the second mark at the fourth edge are (m5, n5), the coordinates of the third landing point are 11. The method according to any one of claims 1 to 10, characterized in that: The collaborative window includes: a function area, a display area and a sidebar area, wherein the function area is an area for displaying one or more controls, and the controls are used to implement control and adjustment of the collaborative window; the display area is an area for displaying part of the content of the first device, and the sidebar area is an area corresponding to the border of the collaborative window; The range of the first area is less than or equal to the range of the display area, and the range of the display area satisfies: the range of the display area = the range of the collaborative window - the range of the function area - the range of the sidebar area.

12. The method according to claim 11, characterized in that The second device further displays one or more first windows, wherein the first windows are located on an upper layer of the collaborative window; When the first window does not block the collaborative window, the range of the first area is equal to the range of the display area; When the first window blocks the collaborative window, the range of the first area is smaller than the range of the display area; The range of the first area satisfies: the range of the first area=the range of the display area-the range of the first intersection area, and the first intersection area is the intersection area of ​​the collaborative window and the first window.

13. The method according to claim 11 or 12, characterized in that: When the range of the collaborative window exceeds the display area of ​​the second device, the range of the first area is smaller than the range of the display area; When the cutoff area is the intersection area between the collaborative window and the display area of ​​the second device, the range of the first area satisfies: the range of the first area = the range of the cutoff area - the range of the first intersection area - the range of the sidebar area - the range of the function area; When the cutoff area is an intersection area between the display area and the display area of ​​the second device, the range of the first area satisfies: the range of the first area=the range of the cutoff area-the range of the first intersection area.

14. The method according to claim 13, characterized in that When the collaborative window is in a floating state, the second device confirms the range of the collaborative window based on an initial range of the collaborative window and a scaling ratio of the collaborative window.

15. The method according to claim 14, characterized in that The range of the collaborative window is the product of the initial range of the collaborative window and the scaling ratio of the collaborative window; Alternatively, the range of the collaborative window is a value obtained by rounding the product of the initial range of the collaborative window and the scaling ratio of the collaborative window.

16. The method according to any one of claims 1 to 15, characterized in that: Before the first device receives the first first movement operation, the method further includes: The first device and the second device establish a collaborative connection and a keyboard and mouse sharing connection; After the collaborative connection and the keyboard and mouse sharing connection are established, the second device sends a first collaborative message to the first device; the first collaborative message is used by the first device to confirm the scope of the first area.

17. The method according to any one of claims 1 to 16, characterized in that: The method further comprises: The second device receives an operation for indicating a change in the collaborative window; In response to an operation for indicating a change in the collaborative window, the second device synchronizes the second collaborative information to the first device, where the second collaborative information is information for describing the changed collaborative window.

18. The method according to claim 17, characterized in that The method further comprises: The operation for indicating the change of the collaborative window includes: an operation for indicating adjustment of the collaborative window size, an operation for indicating adjustment of the collaborative window position, and an operation for indicating switching of the collaborative window form.

19. An interface display method, characterized in that: Applied to a first device, the method includes: When the first device displays a first identifier, the first device receives a first movement operation for the first identifier, where the first identifier is an identifier of an input device connected to the first device; When the first identifier moves to a preset edge of the first device, the first device sends a first crossing request to the second device, where the first crossing request includes: first position information corresponding to the first identifier, where the preset edge is an edge of the first device in a direction in which the first device and the second device are connected; When the position of the first identifier in the first position information is located in the first area, the first device receives a message transmitted by the second device to indicate a crossing failure; the message indicating a crossing failure includes: a third proportional relationship; the first area is an area in the second device that displays part of the content of the first device, and the third proportional relationship is a relationship between the size of the display area of ​​the second device and the size of the collaborative window displayed by the second device; the collaborative window is a window in the second device that is used to display part of the content of the first device; The first device synchronizes the first landing point coordinates of the first identifier on the second device to the second device, the first landing point coordinates are confirmed based on the first position information and the third proportional relationship, and the first landing point coordinates are used by the second device to display the first identifier in the first area.

20. The method according to claim 19, characterized in that The method further comprises: The first device receives a second movement operation on the first identifier displayed in the first area; When the first identifier moves to a first edge of the first area, the first device sends a second crossing request to the second device, the second crossing request comprising: second scale coordinates, the second scale coordinates being determined by the first device based on the third scale relationship and the coordinates of the first identifier at the edge of the first area; the first edge being an edge of the first area where the first area contacts the second area; the second scale coordinates being used to display the second identifier on the second device; Alternatively, when the first identifier moves to the second edge of the first area, the first device displays the first identifier based on the third proportional relationship, the first proportional relationship and the coordinates of the first identifier at the second edge, and the second edge is the edge of the first area in the direction connecting the second area and the first area.

21. The method according to claim 19 or 20, characterized in that The method further comprises: The first device receives a third crossing request from the second device, the third crossing request including: coordinates of the second identifier at a third edge of the second area; the third edge is an edge in the second area in a direction in which the second device and the first device are connected; The first device displays the first mark based on the first proportional relationship and the coordinates of the second mark at the third edge; Alternatively, the first device receives a fourth crossing request from the second device, the fourth crossing request comprising: third landing point coordinates; the third landing point coordinates are confirmed based on the third proportional relationship and the coordinates of the second mark at the fourth edge; the fourth edge is an edge where the second area in the second area contacts the first area; The first device synchronizes the third landing point coordinates and part of the content of the first device to the second device; the second device displays part of the content of the first device and displays the first identifier based on the third landing point coordinates.

22. The method according to any one of claims 19 to 21, characterized in that: Before the first device receives the first first movement operation, the method further includes: The first device and the second device establish a collaborative connection and a keyboard and mouse sharing connection; After the collaborative connection and the keyboard and mouse sharing connection are established, the first device receives first collaborative information from the second device; the first collaborative information is used by the first device to confirm the scope of the first area.

23. An interface display method, characterized in that: Applied to the second device, the method includes: The second device receives a first crossing request from the first device, the first crossing request including: first position information corresponding to the first identifier; the first crossing request is generated when the first device receives a first movement operation for the first identifier, and the first identifier moves to a preset edge of the first device, the preset edge being an edge of the first device in a direction in which the first device and the second device are connected; When the position of the first identifier in the first position information is within the first area, the second device sends a message to the first device indicating a failure of crossing; the message indicating a failure of crossing includes: a third proportional relationship; the first area is an area in the second device that displays part of the content of the first device, and the third proportional relationship is a relationship between the size of the display area of ​​the second device and the size of the collaborative window displayed by the second device; the collaborative window is a window in the second device that is used to display part of the content of the first device; The second device receives first landing point coordinates from the first device, where the first landing point coordinates are confirmed based on the first position information and a third proportional relationship; The second device displays the first mark in the first area based on the first landing point coordinates; Alternatively, when the location of the first identifier is within the second area, the second device displays a second identifier based on the first location information; The second area is a display area of ​​the second device except the first area, and the second identifier is an identifier of an input device connected to the first device.

24. The method according to claim 23, characterized in that The method further comprises: The second device receives a second crossing request from the first device, the second crossing request including: second scale coordinates; the second scale coordinates are confirmed by the first device based on the third scale relationship and the coordinates of the first mark at the edge of the first area; The second crossing request is generated when the first device receives a second movement operation on the first identifier displayed in the first area, and the first identifier moves to a first edge of the first area; the first edge is an edge of the first area in which the first area contacts the second area; The second device displays the second identifier based on the second scale coordinates.

25. The method according to claim 23 or 24, characterized in that The method further comprises: When the second device displays the second mark in the second area, a third movement operation on the second mark is received; When the second identifier moves to a third edge of the second area, the second device sends a third crossing request to the first device, the third crossing request including: coordinates of the second identifier at the third edge; the third edge is an edge in the second area in a direction of connection between the second device and the first device; the coordinates of the second identifier at the third edge are used to display the first identifier on the first device; Alternatively, when the second identifier moves to a fourth edge of the second area, the second device sends a fourth crossing request to the first device, the fourth crossing request including third landing point coordinates, the third landing point coordinates being determined based on the third proportional relationship and the coordinates of the second identifier at the fourth edge; the fourth edge being an edge of the second area where the second area contacts the first area; The second device receives the third landing point coordinates and part of the content of the first device from the first device; The second device displays part of the content of the first device in the first area, and displays the first logo based on the third landing point coordinates.

26. The method according to any one of claims 23 to 25, characterized in that: Before the second device receives the first traversal request, the method further includes: The second device establishes a collaborative connection and a keyboard and mouse sharing connection with the first device; After the collaborative connection and the keyboard and mouse sharing connection are established, the second device sends a first collaborative message to the first device; the first collaborative message is used by the first device to confirm the scope of the first area.

27. A terminal device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the terminal device executes the method according to any one of claims 19 to 22, or the method according to any one of claims 23 to 26.

28. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 19 to 22 is implemented, or the method according to any one of claims 23 to 26 is implemented.

29. A computer program product, characterized in that The method comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 19 to 22 or the method according to any one of claims 23 to 26.

Images