Screen projection display system, screen projection display method and related apparatus

Abstract

A screen projection display system, a screen projection display method and a related apparatus, which are used for reducing the dependence between a screen projection effect and a file for screen projection, and improving the flexibility and convenience of screen projection display. The system comprises a first device and a second device. The method comprises: a first device acquiring a target file, the target file comprising at least one page element; the first device recognizing page elements in the target file; the first device determining a first page element on the basis of the recognized page elements in the target file, generating display indication information of the first page element, and sending the display indication information of the first page element to a second device; and the second device displaying the first page element on the basis of the display indication information of the first page element.

Description

Screen projection display system, screen projection display method and related devices Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a screen projection display system, a screen projection display method and related devices. Background Art

[0002] As screen projection technology becomes more widely used in offices and conferences, the demand for projection quality is increasing. However, current projection quality depends on the layout of the projection file itself. Files with a well-designed layout will produce excellent projection quality after projection. However, files with a poor layout will produce poor projection quality after projection. Reducing the dependency between projection quality and the projection file is an urgent issue that needs to be addressed.

[0003] Summary of the Invention

[0004] The purpose of this application is to provide a screen projection display system, a screen projection display method and related devices, which are used to reduce the dependence between the screen projection effect and the screen projection file, and improve the flexibility and convenience of screen projection display.

[0005] In a first aspect, an embodiment of the present application provides a screen projection display system, including a first device and a second device, wherein the first device and the second device are communicatively connected, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal;

[0006] The first device obtains a target file, which includes at least one page element; the first device identifies the page elements in the target file; the first device determines a first page element based on the identified page elements in the target file, generates display indication information of the first page element, and sends the display indication information of the first page element to the second device; the second device displays the first page element according to the display indication information of the first page element.

[0007] In other words, when a first device projects its screen onto a second device, it projects the current page elements of the target file onto the second device, re-laying out the projected content and reducing the dependency between the projection effect and the projected file. This allows the first device to project only the desired page elements, while the second device remains unaware of the current display on the first device. This significantly improves the flexibility and convenience of the projection process.

[0008] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device opens the target file and displays at least a portion of the content of the target file on the display interface of the first device.

[0009] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device identifies page elements of the currently displayed content based on the currently displayed content of the target file in the display interface of the first device.

[0010] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device receives a first instruction;

[0011] The first device identifies page elements in the at least a portion of the content according to the first instruction.

[0012] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first instruction includes position information, and the position information represents a range selected by the user in the display interface;

[0013] The first device identifies page elements in the selection range.

[0014] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first instruction is sent from the second device to the first device, or is triggered by the first device.

[0015] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device detects at least a portion of the content and identifies multiple page elements.

[0016] In one possible implementation, an embodiment of the present application provides a projection display system, wherein the first device detects page elements in the at least a portion of the content, identifies multiple page elements, and determines the element type of each page element, wherein the element type is any one of a text type, a picture type, and a table type.

[0017] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device parses the target file, identifies the page elements in the target file, and the element type of each page element.

[0018] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device determines the page element identified in the selection range as the first page element.

[0019] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein a first device receives a second instruction indicating a page element selected by a user from at least a portion of the content; the first device determines the selected page element as the first page element based on the second instruction. The user can also select any page element on the first device and project it onto the second device.

[0020] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device receives a third instruction, and the third instruction instructs the user to select at least one element type from the element types of all identified page elements; the first device determines at least one page element of the at least one selected element type as the first page element based on the third instruction.

[0021] In this way, the user can select multiple page elements of the same element type on the first device and project them to the second device. It can be seen that during the projection process, the multiple page elements in the target file are rearranged.

[0022] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device receives search information;

[0023] The first device searches for a page element containing the search information, or searches for a page element whose element type is the search information, among the page elements in the identified target file; and the first device determines the found page element as the first page element.

[0024] In such a design, users can find the desired page elements by searching for a specified element type or a specified theme.

[0025] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device determines a target element type of the first page element;

[0026] The first device generates display instruction information for the first page element according to a target data format corresponding to the target element type, and sends the display instruction information for the first page element to the second device, wherein the display instruction information for the first page element includes target element data of the first page element in the target data format;

[0027] The second device parses the target element data and displays the first page element.

[0028] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device generates at least one first target window, and the at least one first target window is used to display the first page element;

[0029] The first device records the screen of the at least one first target window to generate video data corresponding to the recorded content, and the display indication information of the first page element includes the video data;

[0030] The second device performs display based on the first video data.

[0031] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device identifies an element type of each page element in the first page elements, and the first page elements include page elements of at least one element type;

[0032] The first device generates at least one second target window, where the at least one second target window is used to display the first page element, wherein one second target window is used to display a page element of one element type among the at least one element type.

[0033] In such a design, page elements of different element types are displayed in different second target windows, so that the target file can be re-layouted in the screen projection scenario, reducing the dependence on the original layout of the target file.

[0034] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first page element includes page elements of multiple element types; the at least one second target window is multiple second target windows, wherein different second target windows are used to display page elements of different element types.

[0035] In this way, the user can view the page elements corresponding to each element type in the target file on the first device, which provides a better user experience.

[0036] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device receives a fourth instruction, and the fourth instruction instructs switching a current page element in the first target window;

[0037] The first device switches the page element in the first target window to the next page element of the current page element according to the fourth instruction.

[0038] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device identifies a page element in the target file in response to an identification instruction;

[0039] The identification instruction is triggered by the user on the first device; or the identification instruction is triggered by the user on the second device and sent by the second device to the first device.

[0040] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the screen projection display system further includes: an acquisition device;

[0041] The capture device captures images within the viewing range of the second device. The target device, one of the first and second devices, determines an element of interest within the first page based on the images captured by the capture device. The second device increases the proportion of the element of interest in the display area. This design allows users to promptly understand the page elements that viewers are interested in, improving the user experience.

[0042] In a second aspect, an embodiment of the present application provides a screen projection display system, including a first device and a second device, wherein the first device and the second device are communicatively connected, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal;

[0043] The first device obtains a target file, which includes at least one page element; the first device identifies the page elements in the target file; the first device determines a first page element based on the identified page elements in the target file; the first device generates a derivative page element of the first page element, generates display indication information of the derivative page element, and sends the display indication information of the derivative page element to the second device; the second device displays the derivative page element according to the display indication information of the derivative page element.

[0044] That is to say, when the first device projects the screen to the second device, the first device modifies or converts the page elements currently displayed in the target file to obtain derived page elements. The first device can project page elements that do not belong to the target file to the second device. In such a design, the effect of modifying the target file can be achieved in the projection scenario, and the presentation effect of the page elements in the target file can be optimized. It is convenient for users to rearrange the page elements in the target file during the projection process. Reduce the dependency between the projection effect and the projection file. In this way, the first device can enable the user to project only the desired page elements, and the second device can be unaware of the current display on the first device. It can be seen that the flexibility and convenience of the projection process are improved.

[0045] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device opens the target file and displays at least a portion of the content of the target file on the display interface of the first device.

[0046] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device receives a first instruction;

[0047] The first device identifies page elements in the at least a portion of the content according to the first instruction.

[0048] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first instruction includes location information, and the location information represents a range selected by the user in a display interface of the first device;

[0049] The first device identifies page elements in the selection range;

[0050] The page element identified by the first device in the selection range is determined as the first page element.

[0051] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device receives a second instruction, where the second instruction indicates a page element selected by a user in the at least a portion of the content;

[0052] And the first device determines the selected page element as the first page element based on the second instruction.

[0053] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the first device receives a fifth instruction, where the fifth instruction instructs to modify a display style of the first page element;

[0054] The first device modifies the display style of the first page element based on the fifth instruction to obtain the derived page element.

[0055] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the element type of the first page element is a text type;

[0056] The first device modifies the display style of the first page element in at least one of the following ways:

[0057] Adjust fonts;

[0058] Adjust font size;

[0059] Modify the content of the first page element;

[0060] Rearrange the text.

[0061] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the element type of the first page element is a picture type;

[0062] The first device modifies the display style of the first page element in at least one of the following ways:

[0063] Adjust the resolution;

[0064] Resize;

[0065] Add or delete image elements.

[0066] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the element type of the first page element is a table type;

[0067] Modify the display style of the first page element in at least one of the following ways:

[0068] Adjust the number of rows;

[0069] Adjust the number of columns;

[0070] Modify at least one data.

[0071] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the element type of the first page element is a first element type, and the first element type is any one of a text type, a table type, and a picture type;

[0072] The first device receives a sixth instruction, where the sixth instruction instructs to modify the element type of the first page element to a second element type;

[0073] The first device generates a derived page element based on the sixth instruction and the first page element, and the element type of the derived page element is the second element type.

[0074] In a possible implementation, the embodiment of the present application provides a screen projection display system, wherein the first element type is a text type, and the second element type is a picture type; or,

[0075] The first element type is a text type, and the second element type is a table type; or,

[0076] The first element type is a picture type, and the second element type is a text type; or,

[0077] The first element type is a picture type, and the second element type is a table type; or,

[0078] The first element type is a table type, and the second element type is a text type; or,

[0079] The first element type is a table type, and the second element type is a picture type; or,

[0080] The first element type is a table type, and the second element type is a chart type.

[0081] In a third aspect, an embodiment of the present application provides a screen projection display system, including a first device and a second device, wherein the first device and the second device are communicatively connected, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal;

[0082] The first device obtains a target file; the first device displays at least a portion of the target file, generates display indication information for the at least a portion of the content, and sends the display indication information for the at least a portion of the content to the second device; the second device displays the at least a portion of the content according to the display indication information for the at least a portion of the content; the second device receives a seventh instruction, the seventh instruction including position information, the position information representing a selection range on a display interface of the second device; the second device generates a target image corresponding to the selection range and displays the target image, wherein the target image occupies a larger proportion of the display interface than the selection range. In this way, the user can select the content to be enlarged on the second device and change the layout of the current target file.

[0083] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second device performs quality enhancement processing on the target image and displays the enhanced image.

[0084] In a fourth aspect, an embodiment of the present application provides a screen projection display system, including a first device and a second device, wherein the first device and the second device are communicatively connected, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal;

[0085] The first device obtains the target file;

[0086] The first device opens the target file and displays at least a portion of the content of the target file on a display interface of the first device;

[0087] The first device generates display instruction information of the at least part of the content, and sends the display instruction information of the at least part of the content to the second device;

[0088] The second device identifies a page element in the at least a portion of the content according to the display indication information of the at least a portion of the content, determines a second page element according to the identified page element, and displays the second page element.

[0089] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second device receives an eighth instruction, the eighth instruction may include location information, and the location information represents a range selected by the user in a display interface of the second device;

[0090] The second device identifies page elements in the selection range according to the eighth instruction;

[0091] The second device determines the recognized page element in the selection range as the second page element.

[0092] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second device receives a ninth instruction, the ninth instruction indicating a page element selected by the user from among the identified page elements;

[0093] The second device determines the selected page element as the second page element based on the ninth instruction.

[0094] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second device identifies an element type of each identified page element;

[0095] The second device receives a tenth instruction, wherein the tenth instruction instructs the user to select at least one element type from among all identified element types of the page elements;

[0096] The second device determines, based on the tenth instruction, at least one page element of the selected at least one element type as the second page element.

[0097] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second device generates at least one third target window, and the at least one third target window is used to display the second page element.

[0098] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second device identifies an element type of each page element in the second page elements, and the second page elements include page elements of at least one element type;

[0099] The second device generates at least one fourth target window, where the at least one fourth target window is used to display the second page element, wherein one of the fourth target windows is used to display a page element of one element type among the at least one element type.

[0100] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second page element includes page elements of multiple element types;

[0101] The at least one fourth target window is a plurality of the second target windows, wherein different fourth target windows are used to display page elements of different element types.

[0102] In such a design, page elements of different element types are displayed in different fourth target windows, so that the target file can be re-layouted in the screen projection scenario, reducing the dependence on the original layout of the target file.

[0103] In a possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second device receives an eleventh instruction, and the eleventh instruction instructs to modify a display style of the second page element;

[0104] The second device modifies the display style of the second page element based on the eleventh instruction and displays the modified display style.

[0105] In one possible implementation, an embodiment of the present application provides a screen projection display system, wherein the second page element is a third element type, and the first element type is any one of a text type, a table type, and a picture type;

[0106] The second device receives a twelfth instruction, where the twelfth instruction instructs to modify the element type of the second page element to a fourth element type;

[0107] The second device generates a derived page element corresponding to the second page element based on the twelfth instruction, and displays the derived page element, where the element type of the derived page element is the fourth element type.

[0108] In a fifth aspect, an embodiment of the present application provides a screen projection display method, which can be applied to a first device, the method comprising:

[0109] The first device obtains a target file, where the target file includes at least one page element;

[0110] The first device identifies a page element in the target file;

[0111] The first device determines a first page element according to the identified page elements in the target file, generates display indication information of the first page element, and sends the display indication information of the first page element to the second device.

[0112] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0113] The first device opens the target file and displays at least a portion of the target file on a display interface of the first device.

[0114] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device identifies a page element in the target file, including:

[0115] The first device identifies page elements of the currently displayed content based on the currently displayed content of the target file in the display interface of the first device.

[0116] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device identifies a page element in the target file, including:

[0117] The first device identifies the page element in the target file in response to the identification instruction;

[0118] The identification instruction is triggered by the user on the first device; or the identification instruction is triggered by the user on the second device and sent by the second device to the first device.

[0119] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device identifies a page element in the target file, including:

[0120] The first device receives a first instruction;

[0121] The first device identifies page elements in the at least a portion of the content according to the first instruction.

[0122] In some examples, the first instruction includes position information, where the position information represents a range selected by the user in the display interface. The first device identifying page elements in the target file includes: the first device identifying page elements in the selected range.

[0123] The first device determines the first page element according to the identified page element in the target file, including: the first device determines the identified page element in the selection range as the first page element

[0124] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device identifies a page element in the target file, including:

[0125] The first device parses the target file, identifies page elements in the target file, and identifies the element type of each page element.

[0126] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device identifies a page element in the target file, including:

[0127] The first device detects the at least a portion of the content and identifies a plurality of page elements.

[0128] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device identifies a page element in the target file, including:

[0129] The first device detects page elements in the at least a portion of the content, identifies multiple page elements, and determines an element type of each page element, where the element type is any one of a text type, a picture type, and a table type.

[0130] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device determines a first page element based on the identified page element in the target file, including:

[0131] The first device receives a second instruction, where the second instruction indicates a page element selected by a user in the at least a portion of the content;

[0132] The first device determines the selected page element as the first page element based on the second instruction.

[0133] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device determines a first page element based on the identified page element in the target file, including:

[0134] The first device receives a third instruction, wherein the third instruction instructs a user to select at least one element type from among all identified element types of the page elements;

[0135] The first device determines, based on the third instruction, at least one page element of the selected at least one element type as the first page element.

[0136] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device determines a first page element based on the identified page element in the target file, including:

[0137] The first device receives search information;

[0138] The first device searches for a page element containing the search information, or searches for a page element whose element type is the search information, among page elements in the identified target file;

[0139] The first device determines the found page element as the first page element.

[0140] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein generating display indication information of the first page element includes:

[0141] Determining, by the first device, a target element type of the first page element;

[0142] The first device generates display indication information of the first page element according to the target data format corresponding to the target element type, and sends the display indication information of the first page element to the second device, wherein the display indication information of the first page element includes the target element data of the first page element in the target data format.

[0143] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein generating display indication information of the first page element includes:

[0144] The first device generates at least one first target window, where the at least one first target window is used to display the first page element;

[0145] The first device records the screen of the at least one first target window and generates video data corresponding to the recorded content, and the display indication information of the first page element includes the video data.

[0146] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0147] The first device identifies an element type of each page element in the first page elements, where the first page elements include page elements of at least one element type;

[0148] The first device generates at least one second target window, where the at least one second target window is used to display the first page element, wherein one second target window is used to display a page element of one element type among the at least one element type.

[0149] In a possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first page element includes page elements of multiple element types;

[0150] The at least one second target window is a plurality of second target windows, wherein different second target windows are used to display page elements of different element types.

[0151] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0152] The first device receives a fourth instruction, where the fourth instruction instructs switching a current page element in the first target window;

[0153] The first device switches the page element in the first target window to the next page element of the current page element according to the fourth instruction.

[0154] In a sixth aspect, an embodiment of the present application provides a screen projection display method, applied to a first device, the method comprising:

[0155] The first device obtains a target file, where the target file includes at least one page element;

[0156] The first device identifies a page element in the target file;

[0157] The first device determines a first page element according to the identified page element in the target file;

[0158] The first device generates a derivative page element of the first page element, generates display indication information of the derivative page element, and sends the display indication information of the derivative page element to the second device.

[0159] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0160] The first device opens the target file and displays at least a portion of the content of the target file on a display interface of the first device.

[0161] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device identifies a page element in the target file, including:

[0162] The first device receives a first instruction;

[0163] The first device identifies page elements in the at least a portion of the content according to the first instruction.

[0164] In some examples, the first instruction includes location information representing a user selection range in a display interface of the first device; the first device identifying page elements in the target file includes: the first device identifying page elements in the selection range. The first device determining the first page element based on the identified page elements in the target file includes: the first device determining the page element identified in the selection range as the first page element.

[0165] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device determines a first page element based on the identified page element in the target file, including:

[0166] The first device receives a second instruction, where the second instruction indicates a page element selected by a user in the at least a portion of the content;

[0167] And the first device determines the selected page element as the first page element based on the second instruction.

[0168] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the first device determines a first page element based on the identified page element in the target file, including:

[0169] The first device receives a fifth instruction, where the fifth instruction instructs to modify a display style of the first page element;

[0170] The first device modifies the display style of the first page element based on the fifth instruction to obtain the derived page element.

[0171] In a possible implementation, an embodiment of the present application provides a screen projection display method, wherein the element type of the first page element is a text type;

[0172] The first device modifies the display style of the first page element in at least one of the following ways:

[0173] Adjust fonts;

[0174] Adjust font size;

[0175] Modify the content of the first page element;

[0176] Rearrange the text.

[0177] In a possible implementation, the embodiment of the present application provides a screen projection display method, wherein the element type of the first page element is a picture type;

[0178] The first device modifies the display style of the first page element in at least one of the following ways:

[0179] Adjust the resolution;

[0180] Resize;

[0181] Add or delete image elements.

[0182] In a possible implementation, an embodiment of the present application provides a screen projection display method, wherein the element type of the first page element is a table type;

[0183] Modify the display style of the first page element in at least one of the following ways:

[0184] Adjust the number of rows;

[0185] Adjust the number of columns;

[0186] Modify at least one data.

[0187] In a possible implementation, an embodiment of the present application provides a screen projection display method, wherein the element type of the first page element is a first element type, and the first element type is any one of a text type, a table type, and a picture type;

[0188] The first device generates a derivative page element of the first page element, including:

[0189] The first device receives a sixth instruction, where the sixth instruction instructs to modify the element type of the first page element to a second element type;

[0190] The first device generates a derived page element based on the sixth instruction and the first page element, and the element type of the derived page element is the second element type.

[0191] In some examples, the first element type is a text type and the second element type is a picture type; or, the first element type is a text type and the second element type is a table type; or, the first element type is a picture type and the second element type is a text type; or, the first element type is a picture type and the second element type is a table type; or, the first element type is a table type and the second element type is a text type; or, the first element type is a table type and the second element type is a picture type; or, the first element type is a table type and the second element type is a chart type.

[0192] In a seventh aspect, an embodiment of the present application provides a screen projection display method, applied to a second device, the method comprising:

[0193] The second device receives display instruction information of a first page element, where the first page element is a page element in a target file obtained by the first device;

[0194] The second device displays the first page element according to the display instruction information of the first page element.

[0195] In an eighth aspect, an embodiment of the present application provides a screen projection display method, applied to a second device, the method comprising:

[0196] The second device receives display instruction information of a derived page element, where the derived page element is generated by the first device based on the first page element in the target file;

[0197] The second device displays the derived page element according to the display instruction information of the derived page element.

[0198] In a ninth aspect, an embodiment of the present application provides a screen projection display method, applied to a second device, the method comprising:

[0199] The second device receives display instruction information of at least a portion of the content of the target file, where the display instruction information is generated by the first device based on at least a portion of the content of the target file displayed by the first device;

[0200] The second device displays the at least part of the content according to the display instruction information of the at least part of the content;

[0201] The second device receives a seventh instruction, where the seventh instruction includes location information, where the location information represents a selection range in a display interface of the second device;

[0202] The second device generates a target image corresponding to the selection range and displays the target image, wherein the proportion of the target image in the display interface is greater than the proportion of the selection range in the display interface.

[0203] In a tenth aspect, an embodiment of the present application provides a screen projection display method, applied to a second device, the method comprising:

[0204] The second device receives display instruction information of at least a portion of the content of the target file, where the display instruction information of at least a portion of the content is generated by the first device based on the at least a portion of the content of the target file displayed by the first device;

[0205] The second device identifies a page element in the at least a portion of the content according to the display indication information of the at least a portion of the content, determines a second page element according to the identified page element, and displays the second page element.

[0206] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein identifying a page element in the at least a portion of the content includes: the second device receiving an eighth instruction, the eighth instruction may include location information, the location information representing a user selection range in a display interface of the second device; and the second device identifying a page element in the selection range according to the eighth instruction;

[0207] The determining of the second page element according to the identified page element includes: the second device determining the identified page element in the selection range as the second page element

[0208] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein determining a second page element based on the identified page element includes:

[0209] The second device receives a ninth instruction, where the ninth instruction indicates a page element selected by the user from among the identified page elements;

[0210] The second device determines the selected page element as the second page element based on the ninth instruction.

[0211] In one possible implementation, an embodiment of the present application provides a screen projection display method, wherein the identifying the page elements in the at least a portion of the content includes: the second device identifying the element type of each of the identified page elements;

[0212] The determining of the second page element according to the identified page element includes:

[0213] The second device receives a tenth instruction, wherein the tenth instruction instructs the user to select at least one element type from among all identified element types of the page elements;

[0214] The second device determines, based on the tenth instruction, at least one page element of the selected at least one element type as the second page element.

[0215] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0216] The second device generates at least one third target window, where the at least one third target window is used to display the second page element.

[0217] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0218] The second device identifies an element type of each page element in the second page elements, where the second page elements include page elements of at least one element type;

[0219] The second device generates at least one fourth target window, where the at least one fourth target window is used to display the second page element, wherein one of the fourth target windows is used to display a page element of one element type among the at least one element type.

[0220] In some examples, the second page element includes page elements of multiple element types;

[0221] The at least one fourth target window is a plurality of the second target windows, wherein different fourth target windows are used to display page elements of different element types.

[0222] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0223] The second device receives an eleventh instruction, where the eleventh instruction instructs to modify a display style of the second page element;

[0224] The second device modifies the display style of the second page element based on the eleventh instruction and displays the modified display style.

[0225] In one possible implementation, the present application provides a screen projection display method, further comprising:

[0226] The second page element is of a third element type, and the first element type is any one of a text type, a table type, and a picture type;

[0227] The second device receives a twelfth instruction, where the twelfth instruction instructs to modify the element type of the second page element to a fourth element type;

[0228] The second device generates a derived page element corresponding to the second page element based on the twelfth instruction, and displays the derived page element, where the element type of the derived page element is the fourth element type.

[0229] In some examples, the third element type is a text type, and the fourth element type is a picture type; or,

[0230] The third element type is a text type, and the fourth element type is a table type; or,

[0231] The third element type is a picture type, and the fourth element type is a text type; or,

[0232] The third element type is a picture type, and the fourth element type is a table type; or,

[0233] The third element type is a table type, and the fourth element type is a text type; or,

[0234] The third element type is a table type, and the fourth element type is a picture type; or,

[0235] The third element type is a table type, and the fourth element type is a chart type.

[0236] In an eleventh aspect, an embodiment of the present application provides an electronic device, which is applied to a first device in a screen projection display system, wherein the screen projection display system further includes a second device, the first device and the second device are communicatively connected, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal;

[0237] The electronic device comprises a display and a controller, wherein:

[0238] The display is used to display content;

[0239] The controller is configured to perform:

[0240] Acquire a target file, where the target file includes at least one page element;

[0241] Identifying page elements in the target file;

[0242] A first page element is determined according to the identified page elements in the target file, display indication information of the first page element is generated, and the display indication information of the first page element is sent to the second device.

[0243] In a possible implementation manner, the controller is further configured to execute:

[0244] The target file is opened, and at least a portion of the content of the target file is displayed on the display interface of the first device.

[0245] In a possible implementation, the controller is specifically configured to execute:

[0246] Based on the currently displayed content of the target file in the display interface of the first device, a page element of the currently displayed content is identified.

[0247] In a possible implementation, the controller is specifically configured to execute:

[0248] receiving a first instruction;

[0249] According to the first instruction, a page element in the at least a portion of the content is identified.

[0250] In a possible implementation, the first instruction may include position information, where the position information represents a range selected by the user in the display interface;

[0251] The controller is specifically configured to execute:

[0252] Page elements within the selection are identified.

[0253] In a possible implementation manner, the first instruction is sent by the second device to the electronic device, or is triggered on the electronic device.

[0254] In a possible implementation, the controller is specifically configured to execute:

[0255] The at least a portion of the content is detected to identify a plurality of page elements.

[0256] In a possible implementation, the controller is specifically configured to execute:

[0257] The page elements in the at least a portion of the content are detected, a plurality of page elements are identified, and an element type of each page element is determined, where the element type is any one of a text type, a picture type, and a table type.

[0258] In a possible implementation, the controller is specifically configured to execute:

[0259] The target file is parsed to identify page elements in the target file and the element type of each page element.

[0260] In a possible implementation, the controller is specifically configured to execute:

[0261] The identified page element in the selection range is determined as the first page element.

[0262] In a possible implementation, the controller is specifically configured to execute:

[0263] receiving a second instruction, wherein the second instruction indicates a page element selected by a user in the at least a portion of the content;

[0264] Based on the second instruction, the selected page element is determined as the first page element.

[0265] In a possible implementation, the controller is specifically configured to execute:

[0266] receiving a third instruction, wherein the third instruction indicates at least one element type selected by the user from among all identified element types of the page elements;

[0267] Based on the third instruction, at least one page element of the selected at least one element type is determined as the first page element.

[0268] In a possible implementation, the controller is specifically configured to execute:

[0269] Receive search information;

[0270] Searching for page elements containing the search information, or searching for page elements whose element types are the search information, among the identified page elements in the target file;

[0271] The found page element is determined as the first page element.

[0272] In a possible implementation, the controller is specifically configured to execute:

[0273] Determining a target element type of the first page element;

[0274] According to the target data format corresponding to the target element type, display indication information of the first page element is generated, and the display indication information of the first page element is sent to the second device, wherein the display indication information of the first page element includes the target element data of the first page element in the target data format.

[0275] In a possible implementation manner, the controller is further configured to execute:

[0276] generating at least one first target window, where the at least one first target window is used to display the first page element;

[0277] The at least one first target window is recorded to generate first video data corresponding to the recorded content, and the display indication information of the first page element includes the first video data.

[0278] In a possible implementation manner, the controller is further configured to execute:

[0279] identifying an element type of each page element in the first page elements, wherein the first page elements include page elements of at least one element type;

[0280] At least one second target window is generated, where the at least one second target window is used to display the first page element, wherein one second target window is used to display a page element of one element type among the at least one element type.

[0281] In a possible implementation, the first page element includes page elements of multiple element types;

[0282] The at least one second target window is a plurality of second target windows, wherein different second target windows are used to display page elements of different element types.

[0283] In a possible implementation manner, the controller is further configured to execute:

[0284] receiving a fourth instruction, wherein the fourth instruction instructs switching a current page element in the first target window;

[0285] According to the fourth instruction, the page element in the first target window is switched to the next page element of the current page element.

[0286] In one possible implementation, the controller is specifically configured to execute:

[0287] In response to the identification instruction, identifying a page element in the target file;

[0288] The identification instruction is triggered by the user on the first device; or the identification instruction is triggered by the user on the second device and sent by the second device to the first device.

[0289] In a twelfth aspect, the present application provides an electronic device, which is applied to a first device in a screen projection display system, wherein the screen projection display system further includes a second device, the first device and the second device are communicatively connected, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal;

[0290] The electronic device comprises a display and a controller, wherein:

[0291] The display is used to display content;

[0292] The controller is configured to perform:

[0293] Acquire a target file, where the target file includes at least one page element;

[0294] Identifying page elements in the target file;

[0295] Determining a first page element according to the identified page elements in the target file;

[0296] Generate a derivative page element of the first page element, generate display indication information of the derivative page element, and send the display indication information of the derivative page element to the second device.

[0297] In a thirteenth aspect, the present application provides an electronic device, which is applied to a second device in a screen projection display system, wherein the screen projection display system further includes a first device, the first device and the second device are communicatively connected, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal;

[0298] The electronic device comprises a display and a controller, wherein:

[0299] The display is used to display content;

[0300] The controller is configured to perform:

[0301] receiving display instruction information of at least a portion of content of a target file, where the display instruction information of at least a portion of content is generated by the first device based on at least a portion of content of the target file displayed by the first device;

[0302] According to the display instruction information of the at least part of the content, a page element in the at least part of the content is identified, and a second page element is determined according to the identified page element, and the second page element is displayed.

[0303] In one possible design, the controller is specifically configured to execute:

[0304] receiving an eighth instruction, wherein the eighth instruction may include position information, wherein the position information represents a range selected by a user in the display interface;

[0305] According to the eighth instruction, identifying page elements in the selection range;

[0306] The identified page element in the selection range is determined as the second page element.

[0307] In one possible design, the controller is specifically configured to execute:

[0308] receiving a ninth instruction, the ninth instruction indicating a page element selected by the user from among the identified page elements;

[0309] And based on the ninth instruction, the selected page element is determined as the second page element.

[0310] In one possible design, the controller is specifically configured to execute:

[0311] Identifying the element type of each identified page element;

[0312] receiving a tenth instruction, the tenth instruction instructing a user to select at least one element type from among all identified element types of the page elements;

[0313] Based on the tenth instruction, at least one page element of the selected at least one element type is determined as the second page element.

[0314] In one possible design, the controller is specifically configured to execute:

[0315] At least one third target window is generated, where the at least one third target window is used to display page elements in the at least one partial content.

[0316] In one possible design, the controller is specifically configured to execute:

[0317] identifying an element type of each page element in the second page elements, the second page elements including page elements of at least one element type;

[0318] At least one fourth target window is generated, where the at least one fourth target window is used to display the first page element, wherein one of the fourth target windows is used to display a page element of one element type among the at least one element type.

[0319] In one possible design, the second page element includes page elements of multiple element types;

[0320] The at least one fourth target window is a plurality of the second target windows, wherein different fourth target windows are used to display page elements of different element types.

[0321] In one possible design, the controller is further configured to execute:

[0322] receiving an eleventh instruction, wherein the eleventh instruction instructs to modify a display style of the second page element;

[0323] Based on the eleventh instruction, the display style of the second page element is modified, and the modified display style is displayed.

[0324] In one possible design, the second page element is of the third element type, and the first element type is any one of a text type, a table type, and a picture type;

[0325] The controller is further configured to perform:

[0326] receiving a twelfth instruction, wherein the twelfth instruction instructs to modify the element type of the second page element to a fourth element type;

[0327] Based on the twelfth instruction, a derived page element corresponding to the second page element is generated and displayed, wherein the element type of the derived page element is the fourth element type.

[0328] In the fourteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program. When the computer program runs on a computer, the computer executes a method in any possible implementation manner as described in aspects 5 to 10.

[0329] In the fifteenth aspect, a computer program product is also provided, comprising a computer program, which, when run on a computer, enables the computer to execute a method in any possible implementation manner as in the fifth to tenth aspects.

[0330] In the sixteenth aspect, a graphical user interface on an electronic device is also provided, wherein the electronic device has a display screen, a memory, and a processor, wherein the processor is used to execute one or more computer programs stored in the memory, and the graphical user interface includes a graphical user interface displayed when the electronic device executes the method in any possible implementation manner as described in aspects five to ten.

[0331] In the seventeenth aspect, an embodiment of the present application also provides a chip, which is coupled to a memory in an electronic device, and is used to call a computer program stored in the memory and execute the steps performed by the first device or the second device in any of the aforementioned embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0332] FIG1A is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0333] FIG1B exemplarily shows a schematic diagram of a software architecture;

[0334] 2A to 2I are schematic diagrams illustrating the working process of a screen projection display system according to an embodiment of the present application;

[0335] FIG2J is a schematic diagram of modifying the display style of page elements provided in an embodiment of the present application;

[0336] FIG3A is a schematic diagram of the working process of the screen projection display system provided in one embodiment of the present application;

[0337] FIG3B is a schematic diagram of modifying the display style of page elements provided in an embodiment of the present application;

[0338] 4A to 4G are schematic diagrams of the working process of the screen projection display system provided in one embodiment of the present application;

[0339] 5A to 5G are schematic diagrams of the working process of the screen projection display system provided in one embodiment of the present application;

[0340] 6A and 6B are schematic diagrams of the working process of the screen projection display system provided in one embodiment of the present application;

[0341] FIG7 is a schematic diagram of the working process of the screen projection display system provided in one embodiment of the present application;

[0342] FIG8 is a flow chart of a screen projection display method according to an embodiment of the present application;

[0343] FIG9 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0344] FIG10 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0345] FIG11 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0346] FIG12 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0347] FIG13 is a flow chart of a screen projection display method according to an embodiment of the present application;

[0348] FIG14 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0349] FIG15 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0350] FIG16 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0351] FIG17 is a schematic diagram of a flow chart of a screen projection display method provided in an embodiment of the present application;

[0352] FIG18 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0353] The at least one involved in the embodiments of the present application includes one or more; wherein, more means greater than or equal to two. In addition, it should be understood that in the description of this application, the words "first" and "second" are only used for the purpose of distinguishing descriptions and cannot be understood as expressing or implying relative importance or order.

[0354] The technical solutions provided in the embodiments of this application are applicable to screen projection scenarios. Generally speaking, a screen projection scenario includes a screen projection sending terminal and a screen projection receiving terminal. In this application, the screen projection sending terminal is referred to as the sending terminal, and the screen projection receiving terminal is referred to as the receiving terminal. The sending terminal can send the content used for screen projection to the receiving terminal, and the receiving terminal displays it.

[0355] As shown in Figure 1A, the screen projection display system provided in an embodiment of the present application may include a first device and a second device. The first device may serve as a screen projection sending terminal, and the second device may serve as a screen projection receiving terminal. Optionally, the number of second devices in the screen projection display system may be one or more. This application does not impose any additional restrictions on this.

[0356] In some possible application scenarios, the first device may be a mobile phone, tablet computer, personal computer, mobile phone, handwriting tablet, television, etc. The second device may be a large-screen device such as a display, an all-in-one computer, a tablet computer, a personal computer, a television, a game console, etc. It is understandable that in addition to the first device and the second device, the sending terminal and the receiving terminal may also have other names, for example, the sending terminal is an active screen projection device, and the receiving terminal is a passive screen projection device; or, the sending terminal is the main device, and the receiving terminal is the auxiliary device; or, the sending terminal is the source device, and the receiving terminal is the target device; or, the sending terminal is the first device, and the receiving terminal is the second device, etc., and this application does not limit this. Among them, the roles of the sending terminal and the receiving terminal can be interchanged, that is, the sending terminal can project the screen to the receiving terminal, and correspondingly, the receiving terminal can also project the screen to the sending terminal.

[0357] Currently, related screen projection technologies include screen recording and DLNA projection.

[0358] Screen recording and casting refers to the encoder of the sending terminal encoding the current display information on the sending terminal display screen to generate a video stream. The receiving terminal decodes and displays the received display information through the decoder. The common screen recording and casting protocol is the Miracast protocol. In the screen recording and casting method, the sending terminal and the receiving terminal can display the same picture. In other words, the content currently displayed by the sending terminal is the content that is cast to the receiving terminal. In this way, although it is possible to cast all the content currently displayed on the sending terminal display screen, or to cast the interface (surface) of a running application (APP) of the sending terminal. However, this method not only displays documents, but also displays content outside of documents, such as the operating system interface, the chat application interface, etc., not just documents. It can be seen that this method cannot select part of the content on the interface on the sending terminal display screen for casting,

[0359] DLNA screen projection refers to the transfer of multimedia content (such as photos, music, videos, etc.) from a sending terminal to a DLNA-supported receiving terminal for playback. However, the sending terminal does not play or display the multimedia content. In office or meeting scenarios, documents that require high screen projection are not compatible with DLNA, and therefore DLNA screen projection is difficult to be widely used in office or meeting scenarios.

[0360] As can be seen, some existing screen projection technologies cannot support document projection. Other screen projection methods, while they can achieve document projection, only project the content currently displayed on the sending terminal screen or the running app, rather than just the document. Moreover, the document projection effect also depends on the production layout of the document itself.

[0361] The screen projection display system provided in the embodiment of the present application may include at least one first device and at least one second device. In other words, the screen projection display system may include at least one sending terminal and at least one receiving terminal.

[0362] The first device and the second device can be communicatively connected. In some examples, the first device and the second device can communicate wirelessly. Optionally, the first device and the second device can implement wireless communication based on any existing wireless communication technology. In some possible embodiments, the first device and the second device can be wirelessly connected based on Wi-Fi technology. The first device and the second device are both connected to the same local area network. In other possible embodiments, one of the first device or the second device has a wireless hotspot function, and the other device can implement a wireless communication connection between the first device and the second device by connecting to the hotspot.

[0363] Optionally, the first device may include a screen projector. Alternatively, the first device may be connected to an external screen projector. The screen projector may be in communication with the second device, such as via wireless communication or wired communication.

[0364] The screen projector can be used to pair with a second device so that the device connected to the screen projector (ie, the first device) can project the screen to the second device. The screen projector can record and encode the content displayed on the screen of the first device, or the content displayed on the screen window, and generate display indication information of the content displayed on the screen and send it to the second device, so that the second device can display the projected content according to the received display indication information. Optionally, the screen of the first device can be a first device display interface, or a virtual display window in the background of the first device. In the present application, the display indication information may include but is not limited to data of page elements in the content displayed on the screen in a data format (recorded as element data), and the data format of the page elements. Alternatively, the display indication information may include video data, and the video data may be obtained by recording the page elements. The electronic device can display the page elements according to the display indication information. In an embodiment of the present application, the page elements may include but are not limited to elements in the form of text, pictures, tables, charts, formulas, etc.

[0365] The first device may be a portable electronic device, such as a mobile phone, a tablet computer, a portable computer, a wearable device with wireless communication function (such as a smart watch, smart glasses, a smart bracelet, or a smart helmet, etc.), or a vehicle-mounted device, etc. Exemplary embodiments of portable electronic devices include but are not limited to devices equipped with Or other portable electronic devices with other operating systems. The second device can be a large-screen device such as a tablet computer, a personal computer, a television, an interactive all-in-one machine with a touch function, etc.

[0366] In a possible implementation, in the screen projection display system provided in the embodiment of the present application, the first device and the second device may be provided with an Android system. FIG1B exemplarily shows a software architecture diagram.

[0367] In some embodiments, the Android system is divided into four layers, which are, from top to bottom, the application layer, the application framework layer, the Android runtime (Android Runtime) and system library, and the kernel layer.

[0368] The application layer may include a series of application packages. As shown in FIG1B , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0369] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0370] As shown in FIG1B , the application framework layer may include a window manager (WindowManager), a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0371] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0372] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0373] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0374] The phone manager is used to provide communication functions for electronic devices, such as call status management (including answering, hanging up, etc.).

[0375] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0376] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to announce the completion of downloads, message reminders, and so on. The Notification Manager can also display notifications in the form of icons or scrolling text in the top status bar, such as notifications from background applications, or in the form of dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating the communication terminal, and flashing indicator lights.

[0377] Android Runtime includes core libraries and a virtual machine. Android Runtime is responsible for scheduling and management of the Android system.

[0378] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0379] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0380] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0381] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0382] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0383] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0384] A 2D graphics engine is a drawing engine for 2D drawings.

[0385] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0386] An embodiment of the present application provides a screen projection display system, wherein the first device can obtain a target file, and the target file includes at least one page element. The first device can identify the page elements in the target file. The first device can determine the first page element based on the identified page elements in the target file, generate display indication information of the first page element, and send the display indication information of the first page element to the second device. The second device can display the first page element according to the display indication information of the first page element. The first device can project the first page element to the second device.

[0387] In the present application, the display indication information may include data of the page element in a data format (recorded as element data). The display indication information may include data of the page element in a data format (recorded as element data), and the type of the page element. The display indication information may include data of the page element in a data format (recorded as element data), and indication information of the data format. Alternatively, the display indication information may include video data, and the video data may be obtained by recording the window displaying the page element. The electronic device may display the page element according to the display indication information.

[0388] The first device can determine the first page element from the page elements contained in the target file. The first page element may include one or more page elements. The first device can project the first page element to the second device. That is to say, during the process of projecting the screen from the first device to the second device, the layout of the projected content can be changed, so that the user can change the layout of the projected content during the projecting process, improve the flexibility of the projecting, and adapt to more application scenarios. Optionally, the target file can be a file of the type of word, PPT, PDF, etc. The target file can also be a video file. Each frame of the video may include a page element.

[0389] As shown in (a) of Figure 2A, the page elements in the target file identified by the first device are page element 1, page element 2, and page element 3, respectively. The first page element determined by the first device may be page element 1. The first device may generate display indication information of page element 1 and send it to the second device. As shown in (b) of Figure 2A, the second device may display page element 1 according to the display indication information of page element 1. This enables the first device to project page element 1 onto the second device.

[0390] In some application scenarios, the target file can be implemented as any type of document, and at least a portion of the content of the target file can be a page or an image. Alternatively, the target file can be implemented as a video, and at least a portion of the content of the target file can be implemented as at least one frame of the video, such as a keyframe image.

[0391] The first device may obtain the target file in any of the following ways: the first device may receive the target file, or the first device may read the target file from a local storage device, or read the target file from an external storage device (memory) connected to the first device. The target file may be a video or a document.

[0392] The first device can perform page element recognition on the target file and identify the page elements contained in the target file. In the embodiment of the present application, the page elements may refer to page elements such as text, pictures, tables, and charts. The text can be in the form of any language or symbols. The embodiment of the present application does not impose too many restrictions on this. In the embodiment of the present application, the element type of the page element may include text type, picture type, table type, chart type, etc.

[0393] Based on the screen projection display system provided in the above embodiment, the first device may identify page elements in part or all of the content of the target file using, but not limited to, any of the following identification methods:

[0394] Method 1: Use layout analysis to identify;

[0395] Method 2: Use analytical method for identification;

[0396] Method three: use detection method for identification.

[0397] Method 1: Identification using layout analysis

[0398] Layout analysis generally refers to the analysis, identification, and understanding of images (pictures), text, table information, and positional relationships within a layout. Layout analysis can be used to identify the element type, position, and semantic information of page elements such as text, images, tables, and formulas in a publication. The layout analysis function can be implemented using a text recognition algorithm (or detection algorithm), a table recognition algorithm (or detection algorithm), and an image recognition algorithm (or detection algorithm). These recognition algorithms or detection algorithms can be implemented using, but are not limited to, machine learning technologies such as deep learning technology and neural network technology. This application does not impose too many restrictions on this.

[0399] Optionally, the first device can use a text recognition algorithm to identify the text in the detected text area to obtain text content. The text recognition algorithm may include but is not limited to FasterRCNN, YOLO and other algorithms. Optionally, the first device can crop the detected table area to obtain a picture containing a table. The first device can crop the detected picture area to obtain picture elements. The first device can identify the detected table area to obtain table data. Optionally, the first device can use an end-to-end table recognition algorithm, such as TableMaster, SLANet and other algorithms, which can output table structure information and cell coordinates in HTML format based on the table image. The first device can use a text detection and recognition method to obtain the text content in the table image, and after matching the recognized text content with the table structure, it can generate table data.

[0400] Optionally, layout analysis can also identify the relationships between page elements such as text, images, and tables, such as positional relationships, information relationships, etc.

[0401] A positional relationship can be understood as two page elements meeting pre-defined positional relationship conditions. An informational relationship can be understood as two page elements sharing identical, close, or similar semantic information. These two page elements can be of the same or different types.

[0402] The first device uses a layout analysis method to identify the displayed content. For example, it identifies a screenshot of the displayed content. The first device can detect the position of page elements of various element types in the screenshot of the displayed content, such as text, pictures, tables, formulas, etc.

[0403] In some examples, the first device can virtually display the target file in the background instead of displaying the target file in the display interface of the first device. The first device generates a virtual display window, and the first device can create a virtual display window through the window manager (WindowManager) of the Android system. Optionally, the virtual display window can be a floating window, but it is not displayed in the display interface of the first device. The first device can turn the page of the target file in the virtual display window, take a screenshot of the content in each virtual display window, and obtain a screenshot of the content in the virtual display window. The first device identifies the screenshot and detects the position of the page elements of each element type in the screenshot, such as detecting text, pictures, tables, formulas, etc.

[0404] In other examples, the first device may have a long screenshot capability. When the content of the target file is switched to be displayed on the display interface of the first device, the first device may take a long screenshot of the displayed content of the target file to obtain a screenshot of the displayed content. The first device identifies the screenshot and detects the position of page elements of various types in the screenshot, such as text, images, tables, formulas, etc.

[0405] Method 2: Identify using analytical method

[0406] The first device is configured with a document parsing tool. The first device can use the document parsing tool to analyze the target file layout to determine the location of page elements such as text, images, tables, and charts in the target file, as well as the content of each page element. The document parsing tool may include, but is not limited to, tools such as pdfminer and win32com.

[0407] Method 3: Identification by detection

[0408] The first device can detect the displayed content. For example, the first device can detect the data of the displayed content in the cache. For another example, the first device can perform grayscale detection on a screenshot of the displayed content. The method for the first device to obtain the screenshot of the displayed content can be referred to the relevant introduction in the above-mentioned method 1, and will not be repeated here. The first device can identify page elements such as text, pictures, tables, and charts by detecting the grayscale in the screenshot of the displayed content.

[0409] In this application, the electronic device in the projection display system identifies the page elements in part or all of the content of the target file. Please refer to the relevant introduction of the first device identifying part or all of the content of the target file in this embodiment, and will not be repeated in the following text.

[0410] It can be seen that the first device can detect page elements in part or all of the content of the target file through one or more of the aforementioned identification methods to identify the page elements. Optionally, the first device can also identify the element type of each page element.

[0411] After the first device opens the target file, the display interface of the first device may display at least a portion of the contents of the target file. Optionally, a display area in the display interface of the first device may be used to display the contents of the target file. For example, in FIG2B (a), the display area in the display interface of the first device is used to display at least a portion of the contents of the target file.

[0412] Optionally, after displaying at least a portion of the contents of the target file, the first device may, in response to the recognition instruction, perform an operation of identifying page elements in the target file. The recognition instruction may represent the use of a flexible screen projection mode for screen projection. The user may trigger the recognition instruction by selecting the screen projection mode on the first device. Alternatively, the user may trigger the recognition instruction by selecting the screen projection mode on the second device, and the recognition instruction is sent by the second device to the first device.

[0413] In some examples, the user can trigger the recognition instruction through a user instruction input device. The instruction input device may include, but is not limited to, input devices such as buttons, mice, and touch screens. The user can trigger the recognition instruction by clicking a button. Alternatively, the user can trigger the recognition instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the recognition instruction by directly touching the display screen. Optionally, the user can trigger the recognition instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be, but is not limited to, a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the recognition instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the recognition instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the recognition instruction by issuing a voice control instruction.

[0414] In a possible implementation, the first device may automatically identify page elements in the target file after displaying at least a portion of the content of the target file.

[0415] In some examples, when the first device identifies page elements in the target file, the first device may identify page elements of the currently displayed content based on the currently displayed content of the target file in the display interface of the first device. For example, the first device may employ the aforementioned method 1 to identify page elements in the currently displayed content of the target file. Optionally, the first device may also identify the element type of each page element.

[0416] In other examples, when the first device identifies page elements in the target file, the first device may detect at least a portion of the content of the target file displayed on the display interface of the first device to identify the page elements. For example, the first device may employ the aforementioned method three to identify page elements in the content currently displayed in the target file. Optionally, the first device may also identify the element type of each page element.

[0417] Optionally, in the two aforementioned examples, if the currently displayed content of the target file in the display interface of the first device changes, the first device may identify page elements of the changed displayed content. For example, in (a) of FIG2B , the display area in the display interface of the first device is used to display the content of the target file, and the first device may identify page elements of the content in the display area. If the content in the display area changes, the first device may identify or detect page elements of the changed content.

[0418] In some other examples, after opening the target file, the first device may parse the target file and identify page elements and element types of the target file. For example, the first device may use the aforementioned method 2 to identify page elements and element types of the target file.

[0419] As shown in (a) of Figure 2B, it is assumed that the page elements in the target file identified by the first device are page element 1, page element 2, and page element 3, respectively. The first page element determined by the first device may be page element 1. The first page element determined by the first device may be page element 1. The first device may generate display indication information of page element 1 and send it to the second device. As shown in (b) of Figure 2B, the second device may display page element 1 according to the display indication information of page element 1. The first device may project page element 1 to the second device.

[0420] In a possible implementation, the first device may identify page elements in the target file according to an instruction after displaying at least a portion of the content of the target file.

[0421] As shown in Figure 2C, after the first device opens the target file, the display interface of the first device can display at least a portion of the content of the target file. The first device can receive a first instruction, and the first instruction can be used to instruct the first device to identify the content currently displayed in the target file. When the first device identifies the page elements in the target file, the first device can identify the display interface of the first device to display at least a portion of the content of the target file according to the first instruction.

[0422] In some possible application scenarios, the user can instruct the input device to trigger the first instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the first instruction by clicking a button. Alternatively, the user can trigger the first instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the first instruction by directly touching the display screen. Optionally, the user can trigger the first instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the first instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the first instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the first instruction by issuing a voice control instruction.

[0423] In one possible scenario, the first instruction received by the first device may be triggered by a user on the first device. In other words, the first instruction may be generated on the first device.

[0424] In some examples, as shown in FIG2C , a function area may be configured in the display interface of the first device. The function area may be a fixed area in the interface or an area suspended in the interface. A function button is provided in the function area, which can be used to trigger a first instruction. The user can trigger the first instruction by clicking or touching function button 1, and the first instruction can instruct the first device to identify the current display content of the target file, such as the display content in the display area.

[0425] In other examples, the first instruction may be triggered by the first device detecting a preset gesture or a preset voice command.

[0426] In some other examples, the user can set a selection range in the currently displayed content of the target file. In some examples, the user can click and drag the mouse to form a drag track. Optionally, as shown in FIG2C , the drag track can form a closed figure, and the closed figure can be used as the selection range. The content in the closed figure is also the content in the selection range. Alternatively, the drag track can be an unclosed figure. The first device can infer a similar closed image based on the unclosed figure, and the inferred closed figure can be used as the selection range.

[0427] In other examples, the user forms a touch track by touching the display screen. Optionally, as shown in FIG2C , the touch track can form a closed figure, and the closed figure can be used as a selection range. The content in the closed figure is also the content in the selection range. Alternatively, the touch track can be an unclosed figure. The first device can infer a similar closed image based on the unclosed figure, and the inferred closed figure can be used as the selection range.

[0428] In some other examples, the first device may be configured with a frame selection control. After the user clicks the frame selection control, the user can click and drag the mouse to form a drag track. The first device may determine the selection range based on the starting point and end point of the drag track. For example, the first graphic is determined as the selection range, optionally, the first graphic is a rectangle, and the diagonal line of the rectangle is the line connecting the starting point and midpoint of the drag track. Alternatively, the first graphic is a circle, and the diameter of the circle is the line connecting the starting point and midpoint of the drag track. For another example, the first device may determine the content between the starting point and midpoint of the drag track as the selection range.

[0429] In some other examples, the first device may be configured with a frame selection control. After the user clicks the frame selection control, the user can touch the display screen to form a touch track. The first device may determine the selection range based on the starting point and end point of the touch track. For example, the first graphic is determined as the selection range, optionally, the first graphic is a rectangle, and the diagonal line of the rectangle is the line connecting the starting point and the midpoint of the touch track. Alternatively, the first graphic is a circle, and the diameter of the circle is the line connecting the starting point and the midpoint of the touch track. For another example, the first device may determine the content between the starting point and the midpoint of the touch track as the selection range.

[0430] It should be understood that the above-mentioned method of setting the selection range is used for illustration only, and more methods may be used to set the selection range in actual application scenarios.

[0431] In some possible application scenarios, the user may trigger the first instruction after completing the selection of the range. Alternatively, in some possible application scenarios, the user may trigger the instruction through an instruction input device. The instruction input device may include, but is not limited to, a button, a mouse, a touch screen, or other input devices. The user may trigger the first instruction by clicking a button. Alternatively, the user may trigger the first instruction by clicking a button on the display interface with a mouse. Alternatively, the user may trigger the first instruction by directly touching the display screen. Optionally, the user may trigger the first instruction by performing a single click, continuous clicks, or a long press on an object on the touch display screen (the object may include, but is not limited to, a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user may trigger the first instruction by sliding on the touch screen to form a track or a shape. Optionally, the user may trigger the first instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user may trigger the first instruction by issuing a voice control instruction. The first instruction may include location information, which indicates the range selected by the user on the display interface of the first device. The first instruction may instruct the first device to identify content in the selection range.

[0432] In another possible scenario, the first instruction received by the first device may be sent by the second device to the first device. The first instruction may be triggered by the second device. Optionally, as shown in Figure 2D, the display interface of the second device may be configured with a functional area. The functional area may be a fixed area in the interface or an area floating in the interface. The functional area may include a function button that can be used to trigger the first instruction. Alternatively, the first instruction may be triggered by the second device detecting a preset gesture or a preset voice command. Alternatively, the user may set a selection range within the currently displayed content of the target file in the display interface of the second device. After the user completes setting the selection range, the first instruction may be triggered. Alternatively, after the second device detects the selection range, the user may trigger the first instruction through a command input device, where the first instruction may include location information indicating the user's selection range in the display interface of the first device. The command input device may include, but is not limited to, a button, a mouse, a touch screen, or other input devices. The user may trigger the first instruction by clicking a button. Alternatively, the user may trigger the first instruction by clicking a button in the display interface with a mouse. Alternatively, the user may trigger the first instruction by directly touching the display screen. Optionally, the user can trigger the first instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the first instruction by sliding on the touch screen to form a trajectory, or sliding to form a graphic. Optionally, the user can trigger the first instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the first instruction by issuing a voice control instruction.

[0433] The second device may send a first instruction to the first device, where the first instruction may instruct the first device to identify the content in the selection range.

[0434] Thus, it can be seen that the first device can identify at least one page element in the operation of identifying the page elements in the target file. Optionally, the first device can also identify the element type of each page element.

[0435] The first device may determine the first page element based on the identified page elements, or the first device may determine the first page element based on the identified page elements and the element types of the page elements.

[0436] In some examples, the first device performs a first instruction in the aforementioned embodiment, and the first instruction includes position information, and the position information represents the range selected by the user in the display interface. After the first device identifies the page element of the content in the selection range, it can determine the identified page element in the selection range as the aforementioned first page element, and cast the screen to the second device. Optionally, the first device can directly cast the first page element to the second device after determining the first page element. Or the first device can cast the first page element to the second device after detecting the casting instruction corresponding to the first page element after the first page element.

[0437] In some other examples, the first device may receive a second instruction, which indicates the page element selected by the user from the content currently displayed in the target file. The selected page element may include at least one page element. Based on the second instruction, the first device determines the selected page element as the first page element. The user can trigger the second instruction by performing a page element selection operation on the first device. This application does not specifically limit the way in which the user performs a page element selection operation on the first device. The page element selection operation may include, but is not limited to, dragging the page element, long pressing the page element, clicking the page element, clicking the screen projection option from the operation menu corresponding to the page element, etc.

[0438] As shown in (a) of Figure 2B, it is assumed that the page elements in the content currently displayed by the target file identified by the first device are page element 1, page element 2, and page element 3. The user can select at least one page element from page element 1, page element 2, and page element 3. For example, the page element selected by the user is page element 1. The first device receives a second instruction, and the second instruction can indicate page element 1. The first device can determine page element 1 as the aforementioned first page element based on the second instruction. The first device can generate display indication information of page element 1 and send it to the second device. As shown in (b) of Figure 2B, the second device can display page element 1 based on the display indication information of page element 1. The first device can project page element 1 to the second device.

[0439] In some possible application scenarios, the user can instruct the input device to trigger the second instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the second instruction by clicking a button. Alternatively, the user can trigger the second instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the second instruction by directly touching the display screen. Optionally, the user can trigger the second instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the second instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the second instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the second instruction by issuing a voice control instruction.

[0440] In some further examples, after identifying the content currently displayed by the target file, the first device may identify page elements and types of each page element in the content currently displayed by the target file. The first device receives a third instruction, wherein the third instruction indicates at least one element type selected by the user from the element types of the identified page elements. Based on the third instruction, the first device determines at least one page element of the at least one selected element type as the first page element.

[0441] Assume that the page elements in the content currently displayed by the target file identified by the first device are page element 1, page element 2, page element 3, and page element 4. The element type of page element 1 is text type, the element type of page element 4 is text type, the element type of page element 2 is picture type, and the element type of page element 3 is table type. The user can select at least one element type from text type, picture type, and table type. Assume that the user selects text type. The third instruction received by the first device can indicate the text type. The first device can determine, according to the third instruction, the page elements whose element type is text type, such as all or part of the page elements in page element 1 and page element 4, as the aforementioned first page elements.

[0442] In some possible application scenarios, the user can instruct the input device to trigger the third instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the third instruction by clicking a button. Alternatively, the user can trigger the third instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the third instruction by directly touching the display screen. Optionally, the user can trigger the third instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the third instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the third instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the third instruction by issuing a voice control instruction.

[0443] The content currently displayed by the target file may include one or more element types. The user can select at least one element type from the element types in the content currently displayed by the target file for screen projection. The user can trigger the third instruction by performing an element type selection operation on the first device. This application does not specifically limit the manner in which the user performs the element type selection operation on the first device. The element type selection operation may include, but is not limited to, selecting one or more element types from a list of page elements in the currently displayed content.

[0444] In some further examples, after identifying a target file, the first device may identify page elements and types of each page element in the target file. The first device receives a third instruction, wherein the third instruction instructs a user to select at least one element type from among all element types contained in the target file. Based on the third instruction, the first device determines at least one page element of the at least one selected element type as the first page element.

[0445] Assume that the first device can identify the target file and the identified page elements are page element 1, page element 2, page element 3, page element 4, and page element 5. The element type of page element 1 is text type, the element type of page element 4 is text type, the element types of page elements 2 and page element 5 are picture type, and the element type of page element 3 is table type. The user can select at least one element type from text type, picture type, and table type. Assume that the user selects picture type. The third instruction received by the first device can indicate the picture type. According to the third instruction, the first device can determine the page elements whose element type is picture type, such as all or part of the page elements in page element 2 and page element 5, as the aforementioned first page elements.

[0446] In some other examples, as shown in (a) of FIG2E , a search input box may be further provided in the display interface of the first device for receiving search information. The user may input the information to be searched in the search input box.

[0447] The first device receives the search information. The first device searches for search results in the page elements in the identified target file. The search results may be page elements containing the search information, or page elements whose search element type is the search information. The first device determines the found page element as the first page element, that is, casts the search result as the first page element to the second device, and as shown in (b) in Figure 2E, the second device displays the search results.

[0448] Optionally, after receiving the search information, the first device may determine whether the search information indicates an element type. In one possible scenario, if the search information does not indicate any element type, the first device may perform content recognition on the identified page elements to find page elements whose content includes the search information. The identified page elements may be page elements identified by the first device from the entire content of the target file. Alternatively, they may be page elements identified by the first device from the content currently displayed in the target file.

[0449] In another possible scenario, if the search information indicates a selected element type, the selected element type may include one or more element types, and the first device searches for all page elements whose element type is the selected element type from the identified page elements.

[0450] For example, the search information entered by the user is "picture." "Picture" can indicate a picture type. The first device can search for page elements whose element type is a picture type from the identified page elements. Optionally, the first device can determine some or all of the page elements whose element type is a picture type among the identified page elements as the first page elements.

[0451] Based on the screen projection display system provided by any one of the above embodiments, the first device projects the first page element to the second device operation, the first device can generate display indication information of the first page element, and the second device can display the first page element according to the received display indication information of the first page element.

[0452] In one possible implementation, the display indication information of the first page element may include element data of the page element in a data format. The first device may determine a target element type of the first page element. The first device generates the display indication information of the first page element according to a target data format corresponding to the target element type, and sends the display indication information of the first page element to the second device, where the display indication information of the first page element includes the target element data of the first page element in the target data format.

[0453] Optionally, the display indication information of the first page element may include indication information of the target element type or the target data format.

[0454] For example, the first page element includes a text page element A. The first device may generate text element data A in a data format corresponding to the text type for the text page element A. The second device may parse the text element data A, generate the text page element A, and display it.

[0455] For another example, the first page element includes an image page element B. The first device may generate image element data B in a data format corresponding to the image type for the image page element B. The second device may parse the image element data B, generate the image page element B, and display it.

[0456] For another example, the first page element includes a table page element C. The first device may generate table element data C in a data format corresponding to the table type for the table page element C. The second device may parse the table element data C, generate the table page element C, and display it.

[0457] Optionally, the second device receives each element data in the display instruction information, can automatically match the data format corresponding to each element data, parse the received element data, and display the first page element. Alternatively, the second device receives the element data in the display instruction information and the data format corresponding to each element data. The second device can parse any element data according to its corresponding data format, generate the page element corresponding to the element data, and display it.

[0458] In a possible implementation, the indication information of the first page element may include video data.

[0459] In some examples, the first device may generate at least one first target window for displaying the first page element. The first device may record the screen of the at least one first target window and generate first video data corresponding to the recorded content. The display indication information of the first page element may include the first video data. The second device may perform display based on the first video data.

[0460] In some examples, the first device may generate a first target window in the display interface, and the first target window may be implemented as a display area in the display interface of the aforementioned first device, and the display area is used to display at least a portion of the content of the target file, as shown in (a) in Figure 2F. After the first device determines the first page element, the first device may display the first page element in the display area. Assuming that the first page element is page element 1, as shown in (b) in Figure 2F, the first device displays page element 1 in the display area. The first device may record the display area and generate first video data corresponding to the recorded content, and the display indication information of the first page element includes the first video data. The second device performs display based on the first video data. In this example, the first page element may include page elements of multiple element types. Page elements of all element types in the first page element can be displayed in the first target window.

[0461] In some other examples, the first device may generate a first target window in the display interface, as shown in FIG2G . The first target window may be suspended in the display area for displaying the target file. The first target window may partially obscure the currently displayed content of the target file. Optionally, the size of the first target window is adjustable. The user may adjust the size of the first target window. In this example, the first page element may include page elements of multiple element types. Page elements of all element types in the first page element may be displayed in the first target window.

[0462] In other examples, as shown in (a) in Figure 2H, the first device can generate a first target window in the background virtual display screen. The target file continues to be displayed in the display area of ​​the display screen of the first device. The first target window generated on the background virtual display screen is used to display the first page element. The first device records the first target window generated on the background virtual display screen and generates first video data corresponding to the recorded content, and the display indication information of the first page element includes the first video data. The second device displays the first page element based on the first video data, as shown in (b) in Figure 2H. In this example, the first page element may include page elements of multiple element types. Page elements of all element types in the first page element can be displayed in the first target window.

[0463] In some examples, the first target window may display a portion of the first page elements. The page elements displayed in the first target window may be switchable.

[0464] The first device may receive a fourth instruction, the fourth instruction instructing to switch the current page element in the first target window. The first device switches the page element in the first target window to the next page element of the current page element according to the fourth instruction.

[0465] In some possible application scenarios, the user can instruct the input device to trigger the fourth instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the fourth instruction by clicking a button. Alternatively, the user can trigger the fourth instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the fourth instruction by directly touching the display screen. Optionally, the user can trigger the fourth instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the fourth instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the fourth instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the fourth instruction by issuing a voice control instruction.

[0466] In one possible design, when the first device generates the first target window, the first device can create the first target window through the window manager (WindowManager) of the Android system. Optionally, the first target window can be a floating window.

[0467] In one possible design, the first device uses the first target window as a carrier in the operation of recording the screen of the generated first target window. The first device can use the virtual display device OverlayDisplayDevice created by SurfaceControl.createDisplay() in the Android system to register with the system as the second display device client. The SurfaceComposerClient component of the Android system creates a second display device client for connecting to SurfaceFlinger, thereby obtaining a Surface object. As a consumer, it can obtain the data rendered by the current display interface of the first target window, and encode the data through the MediaCodec module to obtain the video data of the screen recording of the first target window. The first device can record the screen of the first target window.

[0468] In one possible implementation, the first device determines a first page element based on the identified page elements in the target file. The first device can identify the element type of each page element in the first page element, where the first page elements include page elements of at least one element type. The first device can generate at least one second target window, where the at least one second target window is used to display the first page element.

[0469] Among the at least one second target window, each second target window can display a page element of a single element type. Optionally, the page elements displayed in different second target windows can be of the same element type. For example, second target window 1 displays a text-type page element. Second target window 2 can also display a text-type page element.

[0470] Alternatively, the first page element includes page elements of multiple element types. The first device generates multiple second target windows, and the multiple second target windows correspond one-to-one to the multiple element types in the first page element. The element types of the page elements displayed in different second target windows are different.

[0471] Optionally, when the first device generates display indication information for the first page element, the first device may record the screen of each second target window and generate second video data corresponding to the recorded content of each second target window. The display indication information for the first page element may include the second video data corresponding to the recorded content of each second target window.

[0472] In some examples, as shown in (a) of FIG2I , the first device may display a portion of the contents of the target file in a display area of ​​the display interface. After the first device determines the first page element based on the identified page elements in the target file, the first device may generate multiple second target windows in the display area of ​​the display interface, as shown in (b) of FIG2I .

[0473] The first device may classify the first page elements according to element type. For example, page elements with an image element type may be classified into a first element type set, i.e., a picture element set; page elements with a text element type may be classified into a second element type set, i.e., a text element set; and page elements with a table element type may be classified into a third element type set, i.e., a table element set.

[0474] Each element type set corresponds to a second target window in a one-to-one manner. One second target window corresponds to one element type set. One element type set corresponds to one second target window. The second target window can display the page elements in the corresponding element type set.

[0475] After the first device categorizes each page element in the first page element, it may generate at least one second target window of the element type in the display area of ​​the first device. Alternatively, after generating at least one second target window of the element type in the display area of ​​the first device, the first device may categorize each page element in the first page element, i.e., determine a page element in the element type set corresponding to each second target window.

[0476] Optionally, the first device may generate a second target window of one element type, or generate a plurality of second target windows of multiple element types in the display area of ​​the first device.

[0477] As shown in (b) of Figure 2I , for example, the first device may generate a second target window 1 in the display area of ​​the first device. Second target window 1 is used to display image-type page elements. In other words, second target window 1 is used to display page elements in a set of image-type elements. For another example, the first device may generate a second target window 2. Second target window 2 is used to display text-type page elements. For another example, the first device may generate a second target window 3. Second target window 3 is used to display table-type page elements.

[0478] In some examples, after the first device categorizes the page elements in the target file, it can generate multiple second target windows of different element types. Optionally, the multiple second target windows can be displayed independently. When the first device detects a switching operation on any second target window, it can configure the multiple second target windows to be displayed independently based on their display methods, switching only the page elements in that second target window.

[0479] Alternatively, multiple second target windows can be displayed in an associated manner. When the first device detects a switching operation of any second target window, it can be configured for associated display according to the display method of the multiple second target windows, and the page elements in each second target window can be switched. The page elements in each second target window after switching are associated with each other. For example, the first device generates a second target window 1 and a second target window 2. When the first device detects a switching operation of the second target window 1, it switches the page element in the second target window 1 to page element A, and switches the page element in the second target window 2 to page element B, wherein page element B and page element A have an associated relationship, such as the aforementioned position association relationship or information association relationship.

[0480] Page element B and page element A having a positional association relationship can be understood as the positions of page element B and page element A satisfying a predetermined positional relationship condition, and thus the two page elements can be said to have a positional association relationship. For example, in the target file, the page elements surrounding page element A (within a predetermined range radiating from page element A) or the page elements adjacent to page element A.

[0481] The information association relationship between page element B and page element A can be understood as the fact that page element B and page element A have the same, similar, or similar corresponding semantic information. Therefore, the two page elements can be considered to have an information association relationship. These two page elements can be of the same element type or different element types. For example, page element A is "sun" and page element B is a picture containing a sun.

[0482] The display mode of the aforementioned multiple second target windows may be a default setting or may be selected by the user. Regarding the determination method of the display mode of the multiple second target windows, this application does not impose too many restrictions on this.

[0483] Optionally, the user can also select some or all of the second target windows from the multiple second target windows generated by the first device and project them onto the second device. The first device can also receive a window selection instruction, which can be used to indicate the selected second target window.

[0484] The first device may generate display indication information for the page element displayed in the selected second target window and send it to the second device. The first device may record the screen of the selected second target window to obtain video data of the screen recording of the selected second target window, and the display indication information for the first page element may include the video data. The second device may display the page element displayed in the second target window according to the display indication information for the page element displayed in the second target window.

[0485] For example, the first device generates a second target window 1 and a second target window 2, and selects the second target window 1 and the second target window 2 to be projected onto the second device. As shown in (c) in Figure 2I, the second device displays the second target window 1 and the second target window. It should be understood that the layout positions of the second target window 1 and the second target window 2 in (c) in Figure 2I are only used as examples and are not a specific limitation on the actual layout positions of the second target window 1 and the second target window 2 during the display process. When the page elements in any second target window in the first device are switched, the page elements in the second target window in the second device are also switched.

[0486] In one possible design, when the first device generates the second target window, the first device can create the second target window through the window manager (WindowManager) of the Android system. Optionally, the second target window can be a floating window.

[0487] In one possible design, the first device uses the second target window as a carrier in the operation of recording the screen of the generated second target window. The first device can use the virtual display device OverlayDisplayDevice created by SurfaceControl.createDisplay() in the Android system to register with the system as the second display device client. The SurfaceComposerClient component of the Android system creates a second display device client for connecting to SurfaceFlinger, thereby obtaining a Surface object. As a consumer, it can obtain the data rendered by the current display interface of the second target window, and encode the data through the MediaCodec module to obtain video data of the screen recording of the second target window. The first device can record the screen of the second target window.

[0488] In addition, another screen projection display system provided in an embodiment of the present application may include a first device and a second device. The first device may serve as a screen projection sending terminal, and the second device may serve as a screen projection receiving terminal. Optionally, the number of second devices in the screen projection display system may be one or more. This application does not impose too many restrictions on this.

[0489] In the screen projection display system provided in an embodiment of the present application, the first device can obtain a target file, and the target file includes at least one page element. The first device identifies the page element in the target file. The first device determines the first page element based on the identified page element in the target file. The specific execution process can be found in the relevant introduction in the aforementioned embodiment, and will not be repeated here.

[0490] In an embodiment of the present application, the first device may generate a derivative page element of the first page element, generate display indication information of the derivative page element, and send the display indication information of the derivative page element to the second device. The second device displays the derivative page element according to the display indication information of the derivative page element.

[0491] It is clear that the first page element is a page element in the target file. The derived page element of the first page element may not belong to the target file, but is a page element of related content generated by the first device based on the first page element. The first device can project the generated derived page element to the second device.

[0492] For example, if the first page element includes a text page element, the derived page element generated by the first device may be a picture represented by the text page element. For another example, if the first page element includes a table page element, the derived page element generated by the first device may be a chart page element generated based on the table page element.

[0493] Alternatively, the display style of the derived page element of the first page element may not belong to the target file, and the first device may modify the display style of the first page element to obtain the derived page element of the first page element.

[0494] In the present application, the display indication information of the derived page element may include the data of the derived page element in a data format (recorded as element data). The display indication information of the derived page element may include the data of the derived page element in a data format (recorded as element data), and the type of the derived page element. The display indication information of the derived page element may include the data of the derived page element in a data format (recorded as element data), and indication information of the data format. Alternatively, the display indication information of the derived page element may include video data, and the video data may be obtained by recording the window displaying the derived page element. The electronic device may display the derived page element according to the display indication information.

[0495] In one possible implementation, after the first device determines the first page element, the first device receives a fifth instruction, the fifth instruction instructing to modify a display style of the first page element. Based on the fifth instruction, the first device modifies the display style of the first page element to obtain the derived page element.

[0496] In some possible application scenarios, the user can instruct the input device to trigger the fifth instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the fifth instruction by clicking a button. Alternatively, the user can trigger the fifth instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the fifth instruction by directly touching the display screen. Optionally, the user can trigger the fifth instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the fifth instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the fifth instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the fifth instruction by issuing a voice control instruction.

[0497] The first page element may include one page element or the first page element may include multiple page elements of the same element type.

[0498] In some examples, if the element type of the first page element is a text type, the first device can display multiple text modification methods that can modify the display style. Optionally, the first device can use a floating text box or a floating menu to display multiple text modification methods. The first device modifies the display style of the first page element, and the first device can modify the first page element according to the target text modification method. Among them, multiple text modification methods may include but are not limited to: adjusting the font, adjusting the font size, modifying the content in the page element, rearranging the text, adjusting the color, static display, dynamic display, etc. In this way, the first page element can be readjusted, or in other words, the first page element can be rewritten. Among them, (a) in Figure 2J shows a schematic diagram of modifying the content in the page element, and (b) in Figure 2J shows a schematic diagram of rearranging the text.

[0499] As shown in (a) of Figure 3A , the element type of page element 1 is text. The first device determines that the first page element may be page element 1. The first device may modify page element 1 according to the text modification method selected by the user to obtain a derivative page element of page element 1. The first device may display the modified display style of page element 1. Optionally, the first device may generate a floating text box and display the modified display style of page element 1.

[0500] The first device may generate display indication information of the derived page elements of page element 1 and send it to the second device. The second device may display the derived page elements of page element 1, that is, display the modified display style of page element 1, as shown in (b) of FIG3A .

[0501] In other examples, if the element type of the first page element is a picture type, the first device may display multiple picture modification methods. Optionally, the first device may use a floating text box or a floating menu to display multiple picture modification methods.

[0502] The first device modifies the display style of the first page element. The first device can modify the first page element according to the target image modification method. Among them, multiple image modification methods may include but are not limited to: adjusting resolution, adjusting size, adding or deleting image elements. Generally speaking, the content in a picture is composed of points and lines. Points and lines can form graphics, characters, text, etc. In the embodiment of the present application, the picture elements can be points or lines, etc., used to form the content of the picture. Figure 3B shows a schematic diagram of adding picture elements,

[0503] The first device may modify the first page element according to the image modification method selected by the user to obtain a derivative page element of the first page element. The first device may display the derivative page element of the first page element, that is, the modified display style of the first page element. Optionally, the first device may generate a floating display area and display the derivative page element of the first page element.

[0504] The first device may generate display indication information of a derived page element of the first page element and send the information to the second device, and the second device may display the derived page element of the first page element, that is, display the modified display style of the first page element.

[0505] In other examples, if the element type of the first page element is a table type, the first device can identify the first page element and generate table data for the first page element. The first device can display multiple table modification methods. Optionally, the first device can use a floating text box or a floating menu to display multiple table modification methods.

[0506] When the first device detects the selection operation of the target table modification method, the first device can modify the first page element according to the target table modification method. Among them, multiple table modification methods may include but are not limited to: adjusting the number of rows, adjusting the number of columns, and modifying at least one data. The first device can modify the first page element according to the detected table modification method selected by the user, and obtain a derivative page element of the first page element. The first device can display the derivative page element of the first page element. Optionally, the first device can generate a floating display area and display the derivative page element of the first page element. The first device can generate display indication information of the derivative page element of the first page element and send it to the second device. The derivative page element of the first page element is displayed on the second device, that is, the modified display style of the first page element is displayed.

[0507] In one possible implementation, after the first device determines the first page element, the first device receives a sixth instruction, wherein the sixth instruction instructs the first device to modify the element type of the first page element to a second element type. Based on the sixth instruction and the first page element, the first device generates a derivative page element, wherein the element type of the derivative page element is the second element type. This can achieve conversion of the element type of the first page element.

[0508] The element type of the first page element may be a first element type, and the first element type is any one of a text type, a table type, and a picture type.

[0509] In some examples, the first element type is a table type, and the second element type is a chart type. As shown in (a) of Figure 4A , in the display area of ​​the first device, the target file currently displays content including page element 1, page element 2, and page element 3. The element type of page element 1 is text type, the element type of page element 2 is image type, and the element type of page element 3 is table type.

[0510] Assuming that the first page element determined by the first device is page element 3, the sixth instruction instructs to modify the element type of page element 3 to a chart type. Chart types may include multiple chart types, such as scatter plots, bar charts, pie charts, and the like.

[0511] Optionally, the first device may generate a chart corresponding to page element 3 based on a default chart type. Alternatively, the first device may display multiple chart types. Optionally, the first device may display multiple chart types using a floating text box or a floating menu, so that the user can select a chart type, and the chart type selected by the user is recorded as the target chart type.

[0512] In some examples, the first device generates tabular data for page element 3 and, based on the tabular data, generates a first chart corresponding to page element 3, i.e., a derived page element of page element 3, where the chart type of the first chart is the target chart type. The first device can generate display indication information for the derived page element of page element 3 and send it to the second device. The second device displays the derived page element of page element 3, i.e., displays the first chart corresponding to page element 3, as shown in (b) of FIG. 4A .

[0513] Optionally, the display instruction information for the derived page element of page element 3 may include the table data of page element 3, a drawing instruction, and a target chart type, and is sent to the second device. The second device may generate a first chart corresponding to page element 3 based on the table data of page element 3, that is, generate a derived page element of page element 3, where the chart type of the first chart is the target chart type.

[0514] In some other examples, the first device receives a first instruction. The first instruction includes location information, and the location information represents a range selected by the user in the display interface. As shown in (a) in Figure 4B, the user selects a portion of the content of page element 3 in the display interface. The first device identifies the page elements in the selection range based on the first instruction. The first device can identify that the page elements in the selection range are page elements of the table type, and can determine table Q in the selection range as the first page element. The first device can generate a derivative page element of table Q, such as a second chart corresponding to table Q. The first device can project the derivative page elements of table Q onto the second device.

[0515] In one possible design, after the first device determines the first page element, the first device receives a sixth instruction, wherein the sixth instruction instructs to modify the element type of the first page element to a second element type. A large model may be pre-installed in the first device to implement the function of converting the element type.

[0516] In some possible application scenarios, the user can instruct the input device to trigger the sixth instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the sixth instruction by clicking a button. Alternatively, the user can trigger the sixth instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the sixth instruction by directly touching the display screen. Optionally, the user can trigger the sixth instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the sixth instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the sixth instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the sixth instruction by issuing a voice control instruction.

[0517] In some examples, the first element type is a text type, and the second element type is a picture type. The first device can generate a picture related to the semantics or keywords based on the semantics of the first page element or the corresponding keywords.

[0518] As shown in (a) of Figure 4C, in the display area of ​​the first device, the currently displayed content of the target file includes page element 1, page element 2, and page element 3. The element type of page element 1 is text type, the element type of page element 2 is picture type, and the element type of page element 3 is table type.

[0519] The first device determines that the first page element is page element 1, assuming that page element 1 is a circle. The first device can generate a derivative page element of page element 1, including an image of a circular pattern. The derivative page element of page element 1 is then projected onto the second device, as shown in (b) of FIG4C .

[0520] In some other examples, the first element type is a text type, and the second element type is a table type.

[0521] In some further examples, the first element type is a picture type, and the second element type is a text type. In some further examples, the first element type is a picture type, and the second element type is a table type. In some further examples, the first element type is a table type, and the second element type is a text type. In some further examples, the first element type is a table type, and the second element type is a picture type.

[0522] Based on the screen projection display system provided by any one of the above embodiments, the first device projects the derivative page elements of the first page element to the second device operation, the first device can generate display indication information of the derivative page elements of the first page element, and the second device can display the derivative page elements of the first page element based on the received display indication information of the derivative page elements of the first page element.

[0523] In one possible implementation, the display indication information of the derived page element of the first page element may include element data of the derived page element in a data format. The first device may determine the target element type of the derived page element. The first device generates the display indication information of the derived page element according to the target data format corresponding to the target element type, and sends the display indication information of the derived page element to the second device, wherein the derived page element includes the target element data of the derived page element in the target data format.

[0524] Optionally, the display indication information of the derived page element may include indication information of the target element type or the target data format.

[0525] For example, if the derived page element includes a text page element A, the first device may generate text element data A in a data format corresponding to the text type. The second device may parse the text element data A, generate the text page element A, and display it.

[0526] For another example, if the derived page element includes an image page element B, the first device may generate image element data B in a data format corresponding to the image type for the image page element B. The second device may parse the image element data B, generate the image page element B, and display it.

[0527] For another example, if the derived page element includes a table page element C, the first device may generate table element data C in a data format corresponding to the table type for the table page element C. The second device may parse the table element data C, generate the table page element C, and display it.

[0528] Optionally, the second device receives each element data in the display instruction information, can automatically match the data format corresponding to each element data, parse the received element data, and display the derived page element. Alternatively, the second device receives the element data in the display instruction information and the data format corresponding to each element data. The second device can parse any element data according to its corresponding data format, generate the page element corresponding to the element data, and display it.

[0529] In a possible implementation, the indication information of the derived page element may include video data.

[0530] In some examples, the first device may generate at least one fifth target window, the at least one fifth target window being used to display a derived page element of the first page element. The first device records the screen of the at least one fifth target window and generates first video data corresponding to the recorded content. The display indication information of the derived page element of the first page element may include the first video data. The second device performs display based on the first video data.

[0531] In some examples, the first device can generate a fifth target window in the display interface, and the fifth target window can be implemented as a display area in the display interface of the aforementioned first device, which is used to display at least a portion of the content of the target file, as shown in (a) in Figure 4D.

[0532] After the first device determines the derived page element, the first device can display the derived page element in the display area. Assuming that the first page element is page element 1, as shown in (b) in Figure 4D, the first device displays the derived page element of page element 1 in the display area. The first device can record the display area and generate fifth video data corresponding to the recorded content. The display indication information of the derived page element includes the fifth video data. The second device performs display based on the fifth video data. In this example, the derived page elements of the first page element can include page elements of multiple element types. Page elements of all element types in the derived page elements of the first page element can be displayed in the fifth target window.

[0533] In some other examples, the first device may generate a fifth target window in the display interface, as shown in FIG4E , and the fifth target window may be suspended in the display area for displaying the target file. The fifth target window may partially obscure the content currently displayed in the target file. Optionally, the size of the fifth target window is adjustable. The user may adjust the size of the fifth target window. In this example, the derived page elements of the first page element may include page elements of multiple element types. Page elements of all element types in the derived page elements of the first page element may be displayed in the fifth target window.

[0534] In other examples, as shown in (a) of Figure 4F, the first device can generate a fifth target window in the background virtual display screen. The target file continues to be displayed in the display area of ​​the display screen of the first device. The fifth target window generated on the background virtual display screen is used to display the derivative page elements of the first page element. The first device records the fifth target window generated on the background virtual display screen and generates fifth video data corresponding to the recorded content, and the display indication information of the derivative page elements of the first page element includes the fifth video data. The second device displays the derivative page elements of the first page element based on the fifth video data, as shown in (b) of Figure 4F. In this example, the derivative page elements of the first page element may include page elements of multiple element types. Page elements of all element types in the derivative page elements of the first page element can be displayed in the fifth target window.

[0535] In some examples, the fifth target window may display a portion of the page elements derived from the first page element. The page elements displayed in the fifth target window may be switchable.

[0536] The first device may receive a switching instruction instructing to switch the current page element in the fifth target window, and the first device may switch the page element in the fifth target window to the next page element of the current page element according to the switching instruction.

[0537] In some possible application scenarios, the user can instruct the input device to trigger the switching instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the switching instruction by clicking a button. Alternatively, the user can trigger the switching instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the switching instruction by directly touching the display screen. Optionally, the user can trigger the switching instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the switching instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the switching instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the switching instruction by issuing a voice control instruction.

[0538] In one possible design, when the first device generates the fifth target window, the first device can create the fifth target window through the window manager (WindowManager) of the Android system. Optionally, the fifth target window can be a floating window.

[0539] In one possible design, the first device uses the fifth target window as a carrier in the operation of recording the screen of the generated fifth target window. The first device can use the virtual display device OverlayDisplayDevice created by SurfaceControl.createDisplay() in the Android system to register with the system as the second display device client. The SurfaceComposerClient component of the Android system creates a second display device client for connecting to SurfaceFlinger, thereby obtaining a Surface object. As a consumer, it can obtain the data rendered by the current display interface of the fifth target window, and encode the data through the MediaCodec module to obtain video data of the screen recording of the fifth target window. The first device can record the screen of the fifth target window.

[0540] In a possible implementation, the derived page element of the first page element may include a page element of at least one element type. The first device may generate at least one sixth target window, and the at least one sixth target window is used to display the derived page element.

[0541] Among the at least one sixth target window, each sixth target window can display a page element of a single element type. Optionally, the page elements displayed in different sixth target windows can be of the same element type. For example, sixth target window 1 displays a text-type page element. Sixth target window 2 can also display a text-type page element.

[0542] Alternatively, the derived page element includes page elements of multiple element types. The first device generates multiple sixth target windows, and the multiple sixth target windows correspond one-to-one to the multiple element types in the derived page element. The element types of the page elements displayed in different sixth target windows are different.

[0543] Optionally, when the first device generates display indication information for the derived page element, the first device may record the screen of each sixth target window and generate second video data corresponding to the recorded content of each sixth target window. The display indication information for the derived page element may include the second video data corresponding to the recorded content of each sixth target window.

[0544] In some examples, as shown in (a) of FIG4G , the first device may display a portion of the contents of the target file in a display area of ​​the display interface. After the first device determines the derived page elements based on the identified page elements in the target file, the first device may generate multiple sixth target windows in the display area of ​​the display interface, as shown in (b) of FIG4G .

[0545] The first device may classify the derived page elements according to element type. For example, page elements with an image element type may be classified into a first element type set, i.e., a picture element set; page elements with a text element type may be classified into a second element type set, i.e., a text element set; and page elements with a table element type may be classified into a third element type set, i.e., a table element set.

[0546] Each element type set corresponds to the sixth target window in a one-to-one correspondence. One sixth target window corresponds to one element type set. One element type set corresponds to one sixth target window. The sixth target window can display the page elements in the corresponding element type set.

[0547] After classifying each page element in the derived page elements, the first device may generate at least one sixth target window of the element type in the display area of ​​the first device. Alternatively, after generating at least one sixth target window of the element type in the display area of ​​the first device, the first device may further classify each page element in the derived page elements, that is, determine the page element in the element type set corresponding to each sixth target window.

[0548] Optionally, the first device may generate a sixth target window of one element type, or generate multiple sixth target windows of multiple element types in the display area of ​​the first device.

[0549] As shown in (b) of Figure 4G , for example, the first device may generate a sixth target window 1 in the display area of ​​the first device. The sixth target window 1 is used to display image-type page elements. In other words, the sixth target window 1 is used to display page elements in a set of image-type elements. For another example, the first device may generate a sixth target window 2. The sixth target window 2 is used to display text-type page elements. For another example, the first device may generate a sixth target window 3. The sixth target window 3 is used to display table-type page elements.

[0550] In some examples, after the first device categorizes each page element in the derived page element, it can generate a plurality of sixth target windows of different element types. Optionally, the plurality of sixth target windows can be displayed independently.

[0551] The display mode of the aforementioned plurality of sixth target windows may be a default setting or may be selected by the user. Regarding the determination method of the display mode of the plurality of sixth target windows, this application does not impose any additional restrictions.

[0552] Optionally, the user can also select some or all of the sixth target windows from the multiple sixth target windows generated by the first device and project them onto the second device. The first device can also receive a window selection instruction, which can be used to indicate the selected sixth target window.

[0553] The first device may generate display indication information for the page element displayed in the selected sixth target window and send it to the second device. The first device may record the screen of the selected sixth target window to obtain video data of the screen recording of the selected sixth target window, and the display indication information for the derived page element may include the video data. The second device may display the page element displayed in the sixth target window based on the display indication information for the page element displayed in the sixth target window.

[0554] In addition, another screen projection display system provided in an embodiment of the present application may include a first device and a second device. The first device may serve as a screen projection sending terminal, and the second device may serve as a screen projection receiving terminal. Optionally, the number of second devices in the screen projection display system may be one or more. This application does not impose too many restrictions on this.

[0555] In the screen projection display system provided in an embodiment of the present application, the first device obtains a target file. The first device opens the target file and displays at least a portion of the content of the target file in the display interface of the first device, that is, the content currently displayed by the target file. The first device generates display indication information of the at least portion of the content and sends the display indication information of the at least portion of the content to the second device.

[0556] The second device identifies a page element in the at least a portion of the content according to the display indication information of the at least a portion of the content, determines a second page element according to the identified page element, and displays the second page element.

[0557] Optionally, after receiving the display indication information of the at least part of the content, the second device may display the content of the target file currently displayed in the display interface of the first device according to the display indication information of the at least part of the content.

[0558] In some examples, the second device receives an eighth instruction, which can be used to instruct the second device to identify page elements in the content currently displayed in the target file.

[0559] In some possible application scenarios, the user can instruct the input device to trigger the eighth instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the eighth instruction by clicking a button. Alternatively, the user can trigger the eighth instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the eighth instruction by directly touching the display screen. Optionally, the user can trigger the eighth instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the eighth instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the eighth instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the eighth instruction by issuing a voice control instruction.

[0560] As shown in Figure 5A, the display interface of the second device may be configured with a functional area. The functional area may be a fixed area in the interface or an area suspended in the interface. A function button is provided in the functional area, which can be used to trigger the eighth instruction. The user can trigger the eighth instruction by clicking or touching function button 1, and the eighth instruction can instruct the second device to identify the current display content of the target file, such as the display content in the display area.

[0561] In other examples, the eighth instruction may be triggered by the second device detecting a preset gesture or a preset voice command.

[0562] In some other examples, the user can set a selection range in the currently displayed content of the target file. In some examples, the user can form a drag track by clicking and dragging the mouse. Optionally, as shown in (a) in Figure 5F, the drag track can form a closed figure, and the closed figure can be used as the selection range. The content in the closed figure is also the content in the selection range. Alternatively, the drag track can be an unclosed figure. The first device can infer a similar closed image based on the unclosed figure, and the inferred closed figure can be used as the selection range.

[0563] In other examples, the user forms a touch track by touching the display screen. Optionally, as shown in (a) in Figure 5F, the touch track can form a closed figure, and the closed figure can be used as a selection range. The content in the closed figure is also the content in the selection range. Alternatively, the touch track can be an unclosed figure. The first device can infer a similar closed image based on the unclosed figure, and the inferred closed figure can be used as the selection range.

[0564] In some other examples, the first device may be configured with a frame selection control. After the user clicks the frame selection control, the user can click and drag the mouse to form a drag track. The first device may determine the selection range based on the starting point and end point of the drag track. For example, the first graphic is determined as the selection range, optionally, the first graphic is a rectangle, and the diagonal line of the rectangle is the line connecting the starting point and midpoint of the drag track. Alternatively, the first graphic is a circle, and the diameter of the circle is the line connecting the starting point and midpoint of the drag track. For another example, the first device may determine the content between the starting point and midpoint of the drag track as the selection range.

[0565] In some other examples, the first device may be configured with a frame selection control. After the user clicks the frame selection control, the user can touch the display screen to form a touch track. The first device may determine the selection range based on the starting point and end point of the touch track. For example, the first graphic is determined as the selection range, optionally, the first graphic is a rectangle, and the diagonal line of the rectangle is the line connecting the starting point and the midpoint of the touch track. Alternatively, the first graphic is a circle, and the diameter of the circle is the line connecting the starting point and the midpoint of the touch track. For another example, the first device may determine the content between the starting point and the midpoint of the touch track as the selection range.

[0566] It should be understood that the above-mentioned method of setting the selection range is used for illustration only, and more methods may be used to set the selection range in actual application scenarios.

[0567] In response to detecting the selection range instruction, the second device may detect a selection range. For example, the user may trigger the selection range instruction by using a preset function button, a preset gesture, or a preset voice command. When the user completes setting the selection range, an eighth instruction may be triggered. The eighth instruction may include location information representing the user's selection range in the display interface. The eighth instruction may instruct the second device to identify content within the selection range.

[0568] In other examples, the second device may receive a ninth instruction, which indicates the page element selected by the user from the content currently displayed in the target file. The selected page element may include at least one page element. Based on the ninth instruction, the second device determines the selected page element as the second page element. The user can trigger the ninth instruction by performing a page element selection operation on the second device. This application does not specifically limit the way in which the user performs a page element selection operation on the second device. The page element selection operation may include, but is not limited to, dragging the page element, long pressing the page element, clicking the page element, clicking the screen projection option from the operation menu corresponding to the page element, etc.

[0569] In some possible application scenarios, the user can instruct the input device to trigger the ninth instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the ninth instruction by clicking a button. Alternatively, the user can trigger the ninth instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the ninth instruction by directly touching the display screen. Optionally, the user can trigger the ninth instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the ninth instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the ninth instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the ninth instruction by issuing a voice control instruction.

[0570] Assume that the second device identifies page elements in the content currently displayed in the target file as page element 1, page element 2, and page element 3, respectively. A user may select at least one of page element 1, page element 2, and page element 3. For example, the page element selected by the user is page element 1. The second device receives a ninth instruction, which may indicate page element 1. The second device may determine page element 1 as the aforementioned second page element based on the ninth instruction.

[0571] In some further examples, after identifying the content currently displayed by the target file, the second device may identify page elements and types of each page element in the content currently displayed by the target file. The second device receives a tenth instruction, wherein the tenth instruction indicates at least one element type selected by the user from the element types of the identified page elements. Based on the tenth instruction, the second device determines at least one page element of the at least one selected element type as the second page element.

[0572] In some possible application scenarios, the user can instruct the input device to trigger the tenth instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the tenth instruction by clicking a button. Alternatively, the user can trigger the tenth instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the tenth instruction by directly touching the display screen. Optionally, the user can trigger the tenth instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the tenth instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the tenth instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the tenth instruction by issuing a voice control instruction.

[0573] Assume that the page elements in the content currently displayed by the target file identified by the second device are page element 1, page element 2, page element 3, and page element 4. The element type of page element 1 is text type, the element type of page element 4 is text type, the element type of page element 2 is picture type, and the element type of page element 3 is table type. The user can select at least one element type from text type, picture type, and table type. Assume that the user selects text type. The tenth instruction received by the second device can indicate the text type. According to the tenth instruction, the second device can determine the page elements whose element type is text type, such as all or part of the page elements in page element 1 and page element 4, as the aforementioned second page elements.

[0574] In one possible implementation, the second device may generate a third target window in the second device display interface, as shown in FIG5B . The third target window may be suspended in the display area of ​​the second device for displaying the target file. The third target window may completely or partially obscure the currently displayed content of the target file. Optionally, the size of the third target window is adjustable. The user may adjust the size of the third target window. In this example, the second page elements may include page elements of multiple element types. Page elements of all element types in the second page elements may be displayed in the third target window.

[0575] In some examples, the third target window may display a portion of the second page elements. The page elements displayed in the third target window may be switchable.

[0576] The second device may receive a switching instruction, where the switching instruction may instruct to switch the current page element in the third target window. The second device switches the page element displayed in the third target window according to the switching instruction.

[0577] In one possible design, when the second device generates the third target window, the second device can create the third target window through the window manager (WindowManager) of the Android system. Optionally, the third target window can be a floating window.

[0578] Optionally, the second device can record the screen of the third target window and send the video data of the recorded screen to the first device so that the display interface of the first device displays the recorded content. In one possible design, the second device uses the third target window as a carrier in the operation of recording the screen of the generated third target window. The second device can use the virtual display device OverlayDisplayDevice created by SurfaceControl.createDisplay() in the Android system to register with the system as the second display device client. The SurfaceComposerClient component of the Android system creates a second display device client for connecting to SurfaceFlinger, thereby obtaining a Surface object. As a consumer, it can obtain the data rendered by the current display interface of the third target window, and perform video encoding on the data through the MediaCodec module to obtain the video data of the screen recording of the third target window. The second device can record the screen of the third target window.

[0579] In one possible implementation, the second device determines a second page element based on the identified page element in the target file. The second device can identify the element type of each page element in the second page element, where the second page elements include page elements of at least one element type. The second device can generate at least one fourth target window, where the at least one fourth target window is used to display the second page element.

[0580] Among the at least one fourth target window, each fourth target window can display a page element of a single element type. Optionally, the page elements displayed in different fourth target windows can be of the same element type. For example, fourth target window 1 displays a text-type page element. Fourth target window 2 can also display a text-type page element.

[0581] Alternatively, the second page element includes page elements of multiple element types. The second device generates multiple fourth target windows, and the multiple fourth target windows correspond one-to-one to the multiple element types in the second page element. The element types of the page elements displayed in different fourth target windows are different.

[0582] In some examples, as shown in (a) of Figure 5C , the second device may display a portion of the contents of the target file in a display area on the display interface of the second device. After the second device determines the second page element based on the identified page element in the target file, the second device may generate multiple fourth target windows in the display area on the display interface of the second device, as shown in (b) of Figure 5C .

[0583] The second device may classify the second page elements according to element type. For example, page elements with an image element type may be classified into a first element type set, i.e., a picture element set; page elements with a text element type may be classified into a second element type set, i.e., a text element set; and page elements with a table element type may be classified into a third element type set, i.e., a table element set.

[0584] Each element type set corresponds to a fourth target window in a one-to-one correspondence. One fourth target window corresponds to one element type set. One element type set corresponds to one fourth target window. The fourth target window can display page elements in the corresponding element type set.

[0585] After the second device categorizes each of the second page elements, it may generate at least one fourth target window of the element type in the display area of ​​the second device. Alternatively, after generating at least one fourth target window of the element type in the display area of ​​the second device, the second device may categorize each of the second page elements, i.e., determine a page element in the element type set corresponding to each fourth target window.

[0586] Optionally, the second device may generate a fourth target window of one element type, or generate a plurality of fourth target windows of multiple element types in the display area of ​​the second device.

[0587] As shown in (b) of Figure 5C , for example, the second device may generate a fourth target window 1 in the display area of ​​the second device. Fourth target window 1 is used to display image-type page elements. In other words, fourth target window 1 is used to display page elements in a set of image-type elements. For another example, the second device may generate a fourth target window 2. Fourth target window 2 is used to display text-type page elements. For another example, the second device may generate a fourth target window 3. Fourth target window 3 is used to display table-type page elements.

[0588] In some examples, after categorizing the page elements in the target file, the second device may generate a plurality of fourth target windows of different element types. Optionally, the plurality of fourth target windows may be displayed independently. Upon detecting a switching operation for any fourth target window, the second device may configure the plurality of fourth target windows to be displayed independently, switching only the page elements in the fourth target window.

[0589] Alternatively, multiple fourth target windows may be displayed in an associated manner. When the second device detects a switching operation of any fourth target window, it may be configured for associated display according to the display method of the multiple fourth target windows, and the page elements in each fourth target window may be switched, and the page elements in each fourth target window after switching may be associated with each other. For example, the second device generates a fourth target window 1 and a fourth target window 2. When the second device detects a switching operation of the fourth target window 1, it switches the page element in the fourth target window 1 to page element A, and switches the page element in the fourth target window 2 to page element B, wherein page element B and page element A have an associated relationship, such as the aforementioned positional association relationship or information association relationship.

[0590] Page element B and page element A having a positional association relationship can be understood as the positions of page element B and page element A satisfying a predetermined positional relationship condition, and thus the two page elements can be said to have a positional association relationship. For example, in the target file, the page elements surrounding page element A (within a predetermined range radiating from page element A) or the page elements adjacent to page element A.

[0591] The information association relationship between page element B and page element A can be understood as the fact that page element B and page element A have the same, similar, or similar corresponding semantic information. Therefore, the two page elements can be considered to have an information association relationship. These two page elements can be of the same element type or different element types. For example, page element A is "sun" and page element B is a picture containing a sun.

[0592] The display mode of the aforementioned plurality of fourth target windows may be a default setting or may be selected by the user. Regarding the determination method of the display mode of the plurality of fourth target windows, this application does not impose any additional restrictions on this.

[0593] Optionally, the user can also select some or all of the fourth target windows from multiple fourth target windows generated by the second device and project them onto the first device. The second device can also receive a window selection instruction, which can be used to indicate the selected fourth target window. The second device can generate display indication information of the page elements displayed in the selected fourth target window and send it to the first device. The first device can display the page elements displayed in the fourth target window based on the display indication information of the page elements displayed in the fourth target window.

[0594] It should be understood that the layout positions of the fourth target window 1 and the fourth target window 2 in (b) of FIG. 5C are for example only and are not intended to be a specific limitation on the actual layout positions of the fourth target window 1 and the fourth target window 2 during the display process.

[0595] In one possible design, when the second device generates the fourth target window, the second device can create the fourth target window through the window manager (WindowManager) of the Android system. Optionally, the fourth target window can be a floating window.

[0596] Optionally, the second device can record the fourth target window and send the video data of the recording to the first device so that the display interface of the first device displays the recorded content. In one possible design, the second device uses the fourth target window as a carrier in the operation of recording the generated fourth target window. The second device can use the virtual display device OverlayDisplayDevice created by SurfaceControl.createDisplay() in the Android system to register with the system as the second display device client. The SurfaceComposerClient component of the Android system creates a second display device client for connecting to SurfaceFlinger, thereby obtaining a Surface object. As a consumer, it can obtain the data rendered by the current display interface of the fourth target window, and perform video encoding on the data through the MediaCodec module to obtain the video data of the screen recording of the fourth target window. The second device can record the screen of the fourth target window.

[0597] The second device may generate a derivative page element of the second page element and display the derivative page element. Optionally, the second device may generate display instruction information for the derivative page element and send the display instruction information for the derivative page element to the first device. The first device displays the derivative page element according to the display instruction information for the derivative page element.

[0598] It is clear that the second page element is a page element in the target file. The derivative page element of the second page element may not belong to the target file, but is a page element of related content generated by the second device based on the second page element. Optionally, the second device can project the generated derivative page element to the first device.

[0599] For example, if the second page element includes a text page element, the derived page element generated by the second device may be a picture represented by the text page element. For another example, if the second page element includes a table page element, the derived page element generated by the second device may be a chart page element generated based on the table page element.

[0600] Alternatively, the display style of the derived page element of the second page element may not belong to the target file, and the second device may modify the display style of the second page element to obtain the derived page element of the second page element.

[0601] In one possible implementation, after the second device determines the second page element, the second device receives an eleventh instruction, wherein the eleventh instruction instructs the second device to modify the display style of the second page element. Based on the eleventh instruction, the second device modifies the display style of the second page element to obtain the derived page element.

[0602] In some possible application scenarios, the user can instruct the input device to trigger the eleventh instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the eleventh instruction by clicking a button. Alternatively, the user can trigger the eleventh instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the eleventh instruction by directly touching the display screen. Optionally, the user can trigger the eleventh instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the eleventh instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the eleventh instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the eleventh instruction by issuing a voice control instruction.

[0603] The second page element may include one page element or the second page element may include multiple page elements of the same element type.

[0604] In some examples, if the element type of the second page element is a text type, the second device can display multiple text modification methods that can modify the display style. Optionally, the second device can use a floating text box or a floating menu to display multiple text modification methods. The second device modifies the display style of the second page element, and the second device can modify the second page element according to the target text modification method. Among them, multiple text modification methods may include but are not limited to: adjusting the font, adjusting the font size, modifying the content in the page element, rearranging the text, adjusting the color, static display, dynamic display, etc. In this way, the second page element can be readjusted, or in other words, the second page element can be rewritten. Among them, (a) in Figure 2J shows a schematic diagram of modifying the content in the page element, and (b) in Figure 2J shows a schematic diagram of rearranging the text.

[0605] As shown in (a) of Figure 5D , the element type of page element 1 is text, and the second device determines that the second page element may be page element 1. The second device can modify page element 1 according to the text modification method selected by the user to obtain a derived page element of page element 1, and the second device can display the modified display style of page element 1. Optionally, the second device can generate a floating text box and display the modified display style of page element 1.

[0606] The second device displays the derived page elements of page element 1, that is, displays the modified display style of page element 1, as shown in (b) of Figure 5D. Optionally, the second device can generate display indication information of the derived page elements of page element 1 and send it to the first device.

[0607] In other examples, if the element type of the second page element is a picture type, the second device may display multiple picture modification methods. Optionally, the second device may use a floating text box or a floating menu to display multiple picture modification methods.

[0608] The second device modifies the display style of the second page element. The second device can modify the second page element according to the target image modification method. Among them, multiple image modification methods may include but are not limited to: adjusting resolution, adjusting size, adding or deleting image elements. Generally speaking, the content in a picture is composed of points and lines. Points and lines can form graphics, characters, text, etc. In the embodiment of the present application, the picture elements can be points or lines, etc., which are used to form the content of the picture. Figure 3B shows a schematic diagram of adding picture elements,

[0609] The second device may modify the second page element according to the image modification method selected by the user to obtain a derivative page element of the second page element. The second device may display the derivative page element of the second page element, that is, the modified display style of the second page element. Optionally, the second device may generate a floating display area and display the derivative page element of the second page element.

[0610] Optionally, the second device may generate display indication information of a derived page element of the second page element and send the information to the first device. The first device displays the derived page element of the second page element, that is, displays the modified display style of the second page element.

[0611] In other examples, if the element type of the second page element is a table type, the second device can identify the second page element and generate table data for the second page element. The second device can display multiple table modification methods. Optionally, the second device can use a floating text box or a floating menu to display multiple table modification methods.

[0612] When the second device detects the selection operation of the target table modification method, the second device can modify the second page element according to the target table modification method. Among them, multiple table modification methods may include but are not limited to: adjusting the number of rows, adjusting the number of columns, and modifying at least one data. The second device can modify the second page element according to the detected table modification method selected by the user, and obtain a derivative page element of the second page element. The second device can display the derivative page element of the second page element. Optionally, the second device can generate a floating display area and display the derivative page element of the second page element. The second device can generate display indication information of the derivative page element of the second page element and send it to the first device. The derivative page element of the second page element is displayed on the first device, that is, the modified display style of the second page element is displayed.

[0613] In one possible implementation, after the second device determines the second page element, the second device receives a twelfth instruction, wherein the twelfth instruction instructs the second device to modify the element type of the second page element to the second element type. Based on the twelfth instruction and the second page element, the second device generates a derivative page element, wherein the element type of the derivative page element is the fourth element type. This allows the element type of the second page element to be converted.

[0614] In some possible application scenarios, the user can instruct the input device to trigger the twelfth instruction. The instruction input device may include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the twelfth instruction by clicking a button. Alternatively, the user can trigger the twelfth instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the twelfth instruction by directly touching the display screen. Optionally, the user can trigger the twelfth instruction by performing a single click operation, a continuous click operation, or a long press operation on an object in the touch display screen (the object may be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the twelfth instruction by sliding on the touch screen to form a trajectory, or by sliding to form a graphic. Optionally, the user can trigger the twelfth instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the twelfth instruction by issuing a voice control instruction.

[0615] The element type of the second page element may be a third element type, and the third element type is any one of a text type, a table type, and a picture type.

[0616] In some examples, the third element type is a table type, and the fourth element type is a chart type. As shown in (a) of Figure 5E , in the display area of ​​the second device, the target file currently displays content including page element 1, page element 2, and page element 3. The element type of page element 1 is text type, the element type of page element 2 is image type, and the element type of page element 3 is table type.

[0617] Assuming that the second page element determined by the second device is page element 3, the twelfth instruction instructs to modify the element type of page element 3 to a chart type. Chart types may include multiple chart types, such as scatter plots, bar charts, pie charts, and the like.

[0618] Optionally, the second device may generate a chart corresponding to page element 3 based on a default chart type. Alternatively, the second device may display multiple chart types. Optionally, the second device may display multiple chart types using a floating text box or a floating menu, allowing the user to select a chart type, and the chart type selected by the user is recorded as the target chart type.

[0619] In some examples, the second device generates tabular data for page element 3 and, based on the tabular data, generates a first chart corresponding to page element 3, i.e., a derived page element of page element 3, where the chart type of the first chart is the target chart type. The second device displays the derived page element of page element 3, i.e., displays the first chart corresponding to page element 3, as shown in (b) of FIG5E . Optionally, the second device may generate display indication information for the derived page element of page element 3 and send it to the first device.

[0620] Optionally, the display instruction information for the derived page element of page element 3 may include the table data of page element 3, a drawing instruction, and a target chart type, and is sent to the first device. The first device may generate a first chart corresponding to page element 3 based on the table data of page element 3, that is, generate a derived page element of page element 3, and the chart type of the first chart is the target chart type.

[0621] In some other examples, the second device receives an eighth instruction. The eighth instruction includes location information, and the location information represents a range selected by the user in the display interface of the second device. As shown in (a) in Figure 5F, the user selects a portion of the content of page element 3 in the display interface. The second device identifies the page elements in the selection range according to the eighth instruction. The second device can identify that the page elements in the selection range are page elements of the table type, and can determine table Q in the selection range as the second page element. The second device can generate and display the derivative page elements of table Q. For example, the second device generates a second chart corresponding to table Q, as shown in (b) in Figure 5F. Optionally, the second device can project the derivative page elements of table Q onto the first device.

[0622] In one possible design, after the second device determines the second page element, the second device receives a twelfth instruction, wherein the twelfth instruction instructs to modify the element type of the second page element to a fourth element type. A large model may be pre-installed in the second device to implement the function of converting the element type.

[0623] In some examples, the third element type is a text type, and the fourth element type is a picture type. The second device can generate a picture related to the semantics or keywords based on the semantics of the second page element or the corresponding keywords.

[0624] As shown in (a) of Figure 5G, in the display area of ​​the second device, the current display content of the target file includes page element 1, page element 2, and page element 3. The element type of page element 1 is text type, the element type of page element 2 is picture type, and the element type of page element 3 is table type.

[0625] The second device determines that the second page element is page element 1, assuming that page element 1 is a circle. The second device can generate a derivative page element of page element 1, including an image of a circular pattern. The derivative page element of page element 1 is projected onto the second device, as shown in (b) of Figure 5G.

[0626] In some other examples, the third element type is a text type, and the fourth element type is a table type.

[0627] In some further examples, the third element type is a picture type, and the fourth element type is a text type. In some further examples, the third element type is a picture type, and the fourth element type is a table type. In some further examples, the third element type is a table type, and the fourth element type is a text type. In some further examples, the third element type is a table type, and the fourth element type is a picture type.

[0628] In addition, an embodiment of the present application provides another screen projection display system, which may include a first device and a second device. The first device may serve as a screen projection sending terminal, and the second device may serve as a screen projection receiving terminal. Optionally, the number of second devices in the screen projection display system may be one or more. This application does not impose too many restrictions on this.

[0629] In the screen projection display system provided in an embodiment of the present application, the first device obtains a target file. The first device opens the target file and displays at least a portion of the content of the target file in the display interface of the first device, that is, the content currently displayed by the target file, as shown in (a) in Figure 6A. The first device generates display indication information of the at least part of the content and sends the display indication information of the at least part of the content to the second device. The second device displays the page elements in the at least part of the content according to the display indication information of the at least part of the content, as shown in (b) in Figure 6A.

[0630] The second device receives a seventh instruction, the seventh instruction including location information, the location information representing a selection range in the display interface of the second device; the second device generates a target image corresponding to the selection range and displays the target image, as shown in (c) of Figure 6A. The target image accounts for a larger proportion of the display interface than the selection range. Optionally, the target image displayed by the second device may fully or partially cover the current display content of the target file displayed by the second device.

[0631] Optionally, the second device performs quality enhancement processing on the target image and displays the enhanced image.

[0632] In addition, an embodiment of the present application provides another screen projection display system, which may include a first device and a second device. The first device may serve as a screen projection sending terminal, and the second device may serve as a screen projection receiving terminal. Optionally, the number of second devices in the screen projection display system may be one or more. This application does not impose too many restrictions on this.

[0633] In the screen projection display system provided in an embodiment of the present application, the first device obtains a target file, opens the target file, and displays at least a portion of the target file's content, i.e., the currently displayed content of the target file, on the first device display interface, as shown in (a) of FIG6B .

[0634] In one example, a first device receives a screenshot casting instruction, which includes location information representing a selection range in the display interface of the first device. The first device generates a target image corresponding to the selection range, and generates display indication information for the target image, and sends it to the second device. The second device can display the target image according to the display indication information of the target image, as shown in (b) of Figure 6B. In some possible application scenarios, the user can trigger the screenshot casting instruction by using an instruction input device. The instruction input device can include but is not limited to input devices such as buttons, mice, and touch screens. The user can trigger the screenshot casting instruction by clicking a button. Alternatively, the user can trigger the screenshot casting instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the screenshot casting instruction by directly touching the display screen. Optionally, the user can trigger the screenshot casting instruction by performing a single click operation, a continuous click operation, or a long press operation on an object on the touch display screen (the object can be but is not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the screenshot casting instruction by sliding on the touch screen to form a trajectory, or sliding to form a graphic. Optionally, the user can trigger the screenshot casting instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the screenshot casting instruction by issuing a voice control instruction. The way in which the user sets the selection range can be found in the relevant introduction in the aforementioned embodiment and will not be repeated here.

[0635] Optionally, the second device performs quality enhancement processing on the target image and displays the enhanced image. The quality enhancement processing on the image may include but is not limited to adjusting the resolution, changing the image texture, and other operations.

[0636] In another example, the first device receives a screenshot projection instruction, which includes location information, and the location information represents a selection range in the display interface of the first device. The first device generates a target image corresponding to the selection range and performs instruction enhancement processing on the target image. The first device generates display indication information of the target image after quality enhancement and sends it to the second device. The second device can display the target image after quality enhancement according to the display indication information of the target image after quality enhancement.

[0637] Based on the screen projection display system provided by any of the aforementioned embodiments, the screen projection display system may further include a capture device. The capture device may capture the viewing range corresponding to the second device. The viewing range corresponding to the second device may refer to the range within which the content displayed by the second device can be seen. The capture device may capture images within the viewing range corresponding to the second device. In actual application scenarios, the capture device may capture images of viewers or listeners. The capture device may send the captured images to a target device, where the target device is one of the first device and the second device.

[0638] As shown in (a) of Figure 7, the content currently displayed in the target file in the display interface of the second device may include page element 1, page element 2, and page element 3. The acquisition device can acquire images within the viewing range corresponding to the second device in real time or periodically, and send them to the target device. The target device may have the ability to track or recognize line of sight. The target device can identify the image provided by the acquisition device in combination with the spatial physical relationship between the acquisition device and the viewing range, and determine the position of the eye gaze point of each viewer. The target device can determine the page elements of interest based on the position of the eye gaze point of each viewer. Optionally, the target device can use the distribution of the eye gaze point positions of each viewer as the page elements of interest, and the page elements in the area where the eye gaze point positions are concentrated can be used as page elements of interest.

[0639] If the target device is a second device, the second device can, after determining the page element of interest, increase the proportion of the page element of interest in the display area of ​​the second device. Assume that page element 2 is the page element of interest. As shown in (b) of Figure 7, the second device can increase the proportion of page element 2 in the display area. Optionally, the second device can reduce the proportion of other page elements other than the page element of interest in the display area of ​​the second device. If the target device is a first device, the first device can send the page element of interest and an increase proportion instruction to the second device. The second device can increase the proportion of the page element of interest in the display area of ​​the second device according to the increase proportion instruction. In some possible application scenarios, the user can trigger the increase proportion instruction using a command input device. The command input device may include, but is not limited to, input devices such as buttons, mice, and touch screens. The user can trigger the increase proportion instruction by clicking a button. Alternatively, the user can trigger the increase proportion instruction by clicking a button in the display interface with a mouse. Alternatively, the user can trigger the increase proportion instruction by directly touching the display screen. Optionally, the user can trigger the increase proportion instruction by performing a single click operation, a continuous click operation, or a long press operation on the object in the touch display screen (the object can be but not limited to a button, an input box, a floating box, a menu, a list, an icon, a page element, etc.). Optionally, the user can trigger the increase proportion instruction by sliding on the touch screen to form a trajectory, or sliding to form a graphic. Optionally, the user can trigger the increase proportion instruction by performing an air gesture within the image acquisition range of the first device. Optionally, the user can trigger the increase proportion instruction by issuing a voice control instruction.

[0640] Such a design can help the speaker to promptly understand the page elements that the audience is paying attention to, so that the speaker can adjust, explain or illustrate the content of the lecture in a timely manner.

[0641] In some examples, when the first device detects an operation to increase the proportion (or decrease the proportion) of any page element, the first device can send the target page element and the instruction to increase the proportion (or the instruction to decrease the proportion) to the second device. The second device can increase (or decrease) the proportion of any page element in the display area according to the instruction to increase the proportion (or the instruction to decrease the proportion). Alternatively, when the second device detects an operation to increase the proportion (or decrease the proportion) of any page element, the second device increases (or decreases) the proportion of any page element in the display area. Such a design can enable the speaker to guide the audience to pay attention to the page elements desired by the speaker in a timely manner when the audience does not pay attention to the page elements desired by the speaker.

[0642] Based on the screen projection display system provided by any of the above embodiments, data processing tasks involving page element recognition, image quality enhancement, element type conversion, and eye gaze point location determination, which require significant computing power, can be performed by a cloud server connected to the screen projection display system. The cloud server can send the executed data, information, images, and other execution results to the first device or the second device.

[0643] Based on the same inventive concept, the embodiments of the present application provide a variety of screen projection and display methods. This method can be applied to a screen projection and display system, which may include a first device and a second device. The first device is a sending terminal in the screen projection scenario, and the second device can be a receiving terminal in the screen projection scenario. Of course, the roles of the sending terminal and the receiving terminal can be interchanged, that is, the sending terminal can project the screen to the receiving terminal, and correspondingly, the receiving terminal can also project the screen to the sending terminal.

[0644] FIG8 is a flow chart of a screen projection display method provided in an embodiment of the present application, which can be applied to a screen projection display system. The method may include:

[0645] S801: The first device obtains a target file, where the target file includes at least one page element.

[0646] The target file includes a page element of at least one element type, and the at least one element type includes at least one type of text type, image type, and table type.

[0647] S802: The first device identifies page elements in the target file.

[0648] S803: The first device determines a first page element according to the identified page elements in the target file, and generates display indication information of the first page element.

[0649] Exemplarily, the target page element is the page element to be projected, wherein the target page element may include one page element or multiple page elements. Optionally, the target page element may include any page element in the target page currently displayed on the first device.

[0650] S804: The first device sends display indication information of the first page element to the second device.

[0651] S805: The second device displays the first page element according to the display instruction information of the first page element.

[0652] The execution process of each step in this method can be found in the relevant introduction of the above embodiments and will not be repeated here.

[0653] For example, the first device may generate display indication information for a first page element. For any page element, the display indication information may be used to display the page element. The second device may display the first page element on a display screen based on the received display indication information for the first page element.

[0654] Optionally, before the second device receives the first page element, it is displaying a projection task in the projection task queue. When the second device receives the display indication information of the first page element, it can add a first projection task to the projection task queue, where the first projection task is to display the first page element. In the projection task queue, after all projection tasks before the first projection task are completed, the second device executes the first projection task, that is, displays the first page element.

[0655] Optionally, the second device may also display the first page element on a background virtual display screen so as to perform other operations on the first page element, such as image quality enhancement, element type conversion, etc.

[0656] FIG9 is a flow chart showing a screen projection display method provided by an embodiment of the present application, which can be applied to a screen projection display system. The method may include:

[0657] S901: The first device obtains a target file, where the target file includes at least one page element.

[0658] S902: The first device identifies page elements in the target file.

[0659] S903: The first device determines a first page element according to the identified page elements in the target file.

[0660] S904: The first device generates a derived page element of the first page element, and generates display indication information of the derived page element.

[0661] S905: The first device sends display instruction information of the derived page element to the second device.

[0662] S906: The second device displays the derived page element according to the display instruction information of the derived page element.

[0663] The execution process of each step in this method can be found in the relevant introduction of the above embodiments and will not be repeated here.

[0664] FIG10 is a flow chart showing a method for screen projection and display according to an embodiment of the present application, which can be applied to a screen projection and display system. The method may include:

[0665] S1001: The first device obtains a target file.

[0666] S1002: The first device displays at least a portion of the content of the target file and generates display indication information of the at least a portion of the content.

[0667] S1003: The first device sends display instruction information of the at least part of the content to the second device.

[0668] S1004: The second device displays the at least part of the content according to the display instruction information of the at least part of the content.

[0669] S1005: The second device receives a seventh instruction, where the seventh instruction includes location information, and the location information represents a selection range in the display interface of the second device.

[0670] S1006: The second device generates a target image corresponding to the selection range and displays the target image, wherein the proportion of the target image in the display interface is greater than the proportion of the selection range in the display interface.

[0671] The execution process of each step in this method can be found in the relevant introduction of the above embodiments and will not be repeated here.

[0672] FIG11 is a flow chart showing a method for screen projection and display according to an embodiment of the present application. The method may include:

[0673] S1101: The first device obtains a target file.

[0674] S1102: The first device opens the target file and displays at least a portion of the content of the target file on a display interface of the first device.

[0675] S1103: The first device generates display indication information of the at least part of the content.

[0676] S1104: The first device sends display instruction information of the at least part of the content to the second device.

[0677] S1105: The second device identifies a page element in the at least a portion of the content according to the display indication information of the at least a portion of the content, determines a second page element according to the identified page element, and displays the second page element.

[0678] The execution process of each step in this method can be found in the relevant introduction of the above embodiments and will not be repeated here.

[0679] FIG12 is a flow chart showing a method for screen projection and display provided in an embodiment of the present application, which can be applied to a first device, i.e., a screen projection sending terminal. The method may include:

[0680] S1201: The first device obtains a target file, where the target file includes at least one page element.

[0681] S1202, the first device identifies page elements in the target file;

[0682] S1203: The first device determines a first page element according to the identified page elements in the target file, generates display indication information of the first page element, and sends the display indication information of the first page element to the second device.

[0683] The execution process of each step in this method can be found in the relevant introduction of the above embodiments and will not be repeated here.

[0684] FIG13 is a flow chart showing a method for screen projection and display provided in an embodiment of the present application, which can be applied to a first device, i.e., a screen projection sending terminal. The method may include:

[0685] S1301: The first device obtains a target file, where the target file includes at least one page element.

[0686] S1302: The first device identifies page elements in the target file;

[0687] S1303: The first device determines a first page element according to the identified page elements in the target file;

[0688] S1304: The first device generates a derivative page element of the first page element, generates display indication information of the derivative page element, and sends the display indication information of the derivative page element to the second device.

[0689] The execution process of each step in this method can be found in the relevant introduction of the above embodiments and will not be repeated here.

[0690] FIG14 is a flow chart showing a method for screen projection and display provided by an embodiment of the present application, which can be applied to a second device, i.e., a screen projection receiving terminal. The method may include:

[0691] S1401: The second device receives display instruction information of a first page element, where the first page element is a page element in a target file obtained by the first device;

[0692] S1402: The second device displays the first page element according to the display instruction information of the first page element.

[0693] The execution process of each step in this method can be found in the relevant introduction of the above embodiments and will not be repeated here.

[0694] FIG15 is a flow chart showing a method for screen projection and display provided by an embodiment of the present application, which can be applied to a second device, i.e., a screen projection receiving terminal. The method may include:

[0695] S1501: The second device receives display instruction information of a derived page element, where the derived page element is generated by the first device based on the first page element in the target file;

[0696] S1502: The second device displays the derived page element according to the display instruction information of the derived page element.

[0697] FIG16 is a flow chart showing a method for screen projection and display provided in an embodiment of the present application, which can be applied to a second device, i.e., a screen projection receiving terminal. The method may include:

[0698] S1601: The second device receives display instruction information of at least a portion of content of a target file, where the display instruction information of at least a portion of content is generated by the first device based on at least a portion of content of the target file displayed by the first device;

[0699] S1602: The second device displays the at least part of the content according to the display instruction information of the at least part of the content;

[0700] S1603: The second device receives a seventh instruction, where the seventh instruction includes location information, where the location information represents a selection range in a display interface of the second device;

[0701] S1604: The second device generates a target image corresponding to the selection range and displays the target image, wherein the proportion of the target image in the display interface is greater than the proportion of the selection range in the display interface.

[0702] FIG17 is a flow chart showing a method for screen projection and display provided by an embodiment of the present application, which can be applied to a second device, i.e., a screen projection receiving terminal. The method may include:

[0703] S1701: The second device receives display instruction information of at least a portion of content of a target file, where the display instruction information of at least a portion of content is generated by the first device based on at least a portion of content of the target file displayed by the first device;

[0704] S1702: The second device identifies a page element in the at least a portion of the content according to the display indication information of the at least a portion of the content, determines a second page element according to the identified page element, and displays the second page element.

[0705] On the other hand, an embodiment of the present application further provides an electronic device, which may include a display and a controller, wherein:

[0706] The display is used to display content;

[0707] The controller is configured to perform:

[0708] Acquire a target file, where the target file includes at least one page element;

[0709] Identifying page elements in the target file;

[0710] A first page element is determined according to the identified page elements in the target file, display indication information of the first page element is generated, and the display indication information of the first page element is sent to the second device.

[0711] In one possible design, the controller is further configured to execute:

[0712] The target file is opened, and at least a portion of the content of the target file is displayed on the display interface of the first device.

[0713] In one possible design, the controller is specifically configured to execute:

[0714] Based on the currently displayed content of the target file in the display interface of the first device, a page element of the currently displayed content is identified.

[0715] In one possible design, the controller is specifically configured to execute:

[0716] receiving a first instruction;

[0717] According to the first instruction, a page element in the at least a portion of the content is identified.

[0718] In one possible design, the first instruction may include position information, where the position information represents a range selected by the user in the display interface;

[0719] The controller is specifically configured to execute:

[0720] Page elements within the selection are identified.

[0721] In one possible design, the first instruction is given by the second device to the electronic device, or is triggered on the electronic device.

[0722] In one possible design, the controller is specifically configured to execute:

[0723] The at least a portion of the content is detected to identify a plurality of page elements.

[0724] In one possible design, the controller is specifically configured to execute:

[0725] The page elements in the at least a portion of the content are detected, a plurality of page elements are identified, and an element type of each page element is determined, where the element type is any one of a text type, a picture type, and a table type.

[0726] In one possible design, the controller is specifically configured to execute:

[0727] The target file is parsed to identify page elements in the target file and the element type of each page element.

[0728] In one possible design, the controller is specifically configured to execute:

[0729] The identified page element in the selection range is determined as the first page element.

[0730] In one possible design, the controller is specifically configured to execute:

[0731] receiving a second instruction, wherein the second instruction indicates a page element selected by a user in the at least a portion of the content;

[0732] Based on the second instruction, the selected page element is determined as the first page element.

[0733] In one possible design, the controller is specifically configured to execute:

[0734] receiving a third instruction, wherein the third instruction indicates at least one element type selected by the user from among all identified element types of the page elements;

[0735] Based on the third instruction, at least one page element of the selected at least one element type is determined as the first page element.

[0736] In one possible design, the controller is specifically configured to execute:

[0737] Receive search information;

[0738] Searching for page elements containing the search information, or searching for page elements whose element types are the search information, among the identified page elements in the target file;

[0739] The found page element is determined as the first page element.

[0740] In one possible design, the controller is specifically configured to execute:

[0741] Determining a target element type of the first page element;

[0742] The first device generates display indication information of the first page element according to the target data format corresponding to the target element type, and sends the display indication information of the first page element to the second device, wherein the display indication information of the first page element includes the target element data of the first page element in the target data format.

[0743] In one possible design, the controller is further configured to execute:

[0744] generating at least one first target window, where the at least one first target window is used to display the first page element;

[0745] The at least one first target window is recorded to generate first video data corresponding to the recorded content, and the display indication information of the first page element includes the first video data.

[0746] In one possible design, the controller is further configured to execute:

[0747] identifying an element type of each page element in the first page elements, wherein the first page elements include page elements of at least one element type;

[0748] At least one second target window is generated, where the at least one second target window is used to display the first page element, wherein one second target window is used to display a page element of one element type among the at least one element type.

[0749] In one possible design, the first page element includes page elements of multiple element types;

[0750] The at least one second target window is a plurality of second target windows, wherein different second target windows are used to display page elements of different element types.

[0751] In one possible design, the controller is further configured to execute:

[0752] receiving a fourth instruction, wherein the fourth instruction instructs switching a current page element in the first target window;

[0753] According to the fourth instruction, the page element in the first target window is switched to the next page element of the current page element.

[0754] In one possible design, the controller is specifically configured to execute:

[0755] In response to the identification instruction, identifying a page element in the target file;

[0756] The identification instruction is triggered by the user on the first device; or the identification instruction is triggered by the user on the second device and sent by the second device to the first device.

[0757] An embodiment of the present application further provides an electronic device, including a display and a controller, wherein:

[0758] The display is used to display content;

[0759] The controller is configured to perform:

[0760] Acquire a target file, where the target file includes at least one page element;

[0761] Identifying page elements in the target file;

[0762] Determining a first page element according to the identified page elements in the target file;

[0763] Generate a derivative page element of the first page element, generate display indication information of the derivative page element, and send the display indication information of the derivative page element to the second device.

[0764] An embodiment of the present application further provides an electronic device, including a display and a controller, wherein:

[0765] The display is used to display content;

[0766] The controller is configured to perform:

[0767] receiving display instruction information of at least a portion of content of a target file, where the display instruction information of at least a portion of content is generated by the first device based on at least a portion of content of the target file displayed by the first device;

[0768] According to the display instruction information of the at least part of the content, a page element in the at least part of the content is identified, and a second page element is determined according to the identified page element, and the second page element is displayed.

[0769] In one possible design, the controller is specifically configured to execute:

[0770] receiving an eighth instruction, wherein the eighth instruction may include position information, wherein the position information represents a range selected by a user in the display interface;

[0771] According to the eighth instruction, identifying page elements in the selection range;

[0772] The identified page element in the selection range is determined as the second page element.

[0773] In one possible design, the controller is specifically configured to execute:

[0774] receiving a ninth instruction, the ninth instruction indicating a page element selected by the user from among the identified page elements;

[0775] And based on the ninth instruction, the selected page element is determined as the second page element.

[0776] In one possible design, the controller is specifically configured to execute:

[0777] Identifying the element type of each identified page element;

[0778] receiving a tenth instruction, the tenth instruction instructing a user to select at least one element type from among all identified element types of the page elements;

[0779] Based on the tenth instruction, at least one page element of the selected at least one element type is determined as the second page element.

[0780] In one possible design, the controller is specifically configured to execute:

[0781] At least one third target window is generated, where the at least one third target window is used to display page elements in the at least one partial content.

[0782] In one possible design, the controller is specifically configured to execute:

[0783] identifying an element type of each page element in the second page elements, the second page elements including page elements of at least one element type;

[0784] At least one fourth target window is generated, where the at least one fourth target window is used to display the first page element, wherein one of the fourth target windows is used to display a page element of one element type among the at least one element type.

[0785] In one possible design, the second page element includes page elements of multiple element types;

[0786] The at least one fourth target window is a plurality of the second target windows, wherein different fourth target windows are used to display page elements of different element types.

[0787] In one possible design, the controller is further configured to execute:

[0788] receiving an eleventh instruction, wherein the eleventh instruction instructs to modify a display style of the second page element;

[0789] Based on the eleventh instruction, the display style of the second page element is modified, and the modified display style is displayed.

[0790] In one possible design, the second page element is of the third element type, and the first element type is any one of a text type, a table type, and a picture type;

[0791] The controller is further configured to perform:

[0792] receiving a twelfth instruction, wherein the twelfth instruction instructs to modify the element type of the second page element to a fourth element type;

[0793] Based on the twelfth instruction, a derived page element corresponding to the second page element is generated and displayed, wherein the element type of the derived page element is the fourth element type.

[0794] Based on the same concept, Figure 18 is a structural diagram of an electronic device 1800 provided in an embodiment of the present application. The electronic device 1800 may be the first device or the second device mentioned above. As shown in Figure 18, the electronic device 1800 may include: one or more processors 1801; one or more memories 1802; a communication interface 1803, and one or more computer programs 1804, and the above-mentioned devices may be connected via one or more communication buses 1805. The one or more computer programs 1804 are stored in the above-mentioned memory 1802 and are configured to be executed by the one or more processors 1801, and the one or more computer programs 1804 include instructions. For example, the above-mentioned instructions can be used to execute the relevant steps of the main device in the corresponding embodiment above. The communication interface 1803 is used to realize communication between the main device and other devices (such as the target device), for example, the communication interface can be a transceiver.

[0795] Based on the same concept, an embodiment of the present application also provides a screen projection display system. The screen projection display system includes a first device and a second device. Exemplarily, the first device can be a device with a display screen such as a mobile phone, a tablet computer, a PC, a watch, etc. The second device can be a large-screen device such as an all-in-one machine, a tablet computer, a PC, a television, etc. Among them, the structures of the first device and the second device can be shown in Figure 18. For example, when the electronic device 1800 shown in Figure 18 is the first device, when the instructions of one or more computer programs 1804 are executed by the processor, the first device performs the steps of the first device as described above. When the electronic device 1800 shown in Figure 18 is the second device, when the instructions of one or more computer programs 1804 are executed by the processor, the second device performs the steps of the second device as described above.

[0796] In the embodiments provided in the present application above, the method provided in the embodiment of the present application is introduced from the perspective of an electronic device (such as a first device or a second device) as an execution subject. In order to implement the various functions in the method provided in the embodiment of the present application above, the electronic device may include a hardware structure and / or a software module to implement the above functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether a function of the above functions is executed in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application and design constraints of the technical solution.

[0797] As used in the above embodiments, the terms “when…” or “after…” may be interpreted to mean “if…” or “after…” or “in response to determining…” or “in response to detecting…”, depending on the context. Similarly, the phrases “upon determining…” or “if (the stated condition or event) is detected” may be interpreted to mean “if determining…” or “in response to determining…” or “upon detecting (the stated condition or event)” or “in response to detecting (the stated condition or event)”, depending on the context. In addition, in the above embodiments, relational terms such as first and second are used to distinguish one entity from another, without limiting any actual relationship or order between these entities.

[0798] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0799] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0800] The present application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)). In the absence of conflict, the solutions of the above embodiments can be used in combination.

Claims

1. A screen projection display system, wherein: The device comprises a first device and a second device, wherein the first device and the second device are in communication connection, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal; The first device acquires a target file, where the target file includes at least one page element; The first device identifies a page element in the target file; The first device determines a first page element according to the identified page element in the target file, generates display indication information of the first page element, and sends the display indication information of the first page element to the second device; The second device displays the first page element according to the display indication information of the first page element.

2. The system of claim 1, wherein: The first device opens the target file and displays at least a portion of the content of the target file on a display interface of the first device.

3. The system of claim 2, wherein: The first device identifies a page element of the currently displayed content based on the currently displayed content of the target file in the display interface of the first device.

4. The system of claim 2, wherein: The first device receives a first instruction; The first device identifies page elements in the at least a portion of the content according to the first instruction.

5. The system of claim 4, wherein: The first instruction includes position information, and the position information represents a range selected by a user in a display interface; The first device identifies page elements in the selection range.

6. The system of claim 4, wherein: The first instruction is sent from the second device to the first device, or is triggered by the first device.

7. The system of claim 2, wherein: The first device detects the at least a portion of the content and identifies a plurality of page elements.

8. The system of claim 2, wherein: The first device detects page elements in the at least a portion of the content, identifies multiple page elements, and determines an element type of each page element, where the element type is any one of a text type, a picture type, and a table type.

9. The system of claim 1, wherein: The first device parses the target file, identifies page elements in the target file, and an element type of each page element.

10. The system of claim 5, wherein: The first device determines the page element identified in the selection range as the first page element.

11. The system of claim 3, wherein: The first device receives a second instruction, where the second instruction indicates a page element selected by a user in the at least a portion of the content; The first device determines the selected page element as the first page element based on the second instruction.

12. The system of claim 8 or 9, wherein: The first device receives a third instruction, wherein the third instruction indicates at least one element type selected by a user from among all identified element types of the page elements; The first device determines, based on the third instruction, at least one page element of the selected at least one element type as the first page element.

13. The system of claim 1, wherein: The first device receives search information; The first device searches for a page element containing the search information among the identified page elements in the target file, or searches for a page element whose element type is the search information; The first device determines the found page element as the first page element.

14. The system of claim 1, wherein: The first device determines a target element type of the first page element; The first device generates display indication information of the first page element according to the target data format corresponding to the target element type, and sends the display indication information of the first page element to the second device, wherein the display indication information of the first page element includes target element data of the first page element in the target data format; The second device parses the target element data and displays the first page element.

15. The system of claim 1, wherein: The first device generates at least one first target window, where the at least one first target window is used to display the first page element; The first device records the screen of the at least one first target window to generate video data corresponding to the recorded content, and the display indication information of the first page element includes the video data; The second device performs display based on the first video data.

16. The system of claim 1, wherein: The first device identifies an element type of each page element in the first page elements, wherein the first page elements include page elements of at least one element type; The first device generates at least one second target window, and the at least one second target window is used to display the first page element, wherein one second target window is used to display a page element of one element type among the at least one element type.

17. The system of claim 16, wherein: The first page element includes page elements of multiple element types; The at least one second target window is a plurality of second target windows, wherein different second target windows are used to display page elements of different element types.

18. The system of claim 15, wherein: The first device receives a fourth instruction, where the fourth instruction instructs switching a current page element in the first target window; The first device switches the page element in the first target window to the next page element of the current page element according to the fourth instruction.

19. The system of claim 1, wherein: The first device identifies the page element in the target file in response to the identification instruction; The identification instruction is triggered by the user on the first device; or the identification instruction is triggered by the user on the second device The second device triggers the sending of the message to the first device.

20. The system of claim 7, wherein: The screen projection display system also includes: a collection device; The acquisition device acquires images within a viewing range corresponding to the second device; The target device in the first device and the second device determines the page element of interest in the first page element based on the image captured by the capture device; The second device increases the proportion of the element of interest in the display area.

21. A screen projection display system, wherein: The device comprises a first device and a second device, wherein the first device and the second device are in communication connection, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal; The first device acquires a target file, where the target file includes at least one page element; The first device identifies a page element in the target file; The first device determines a first page element according to the identified page element in the target file; The first device generates a derived page element of the first page element, generates display indication information of the derived page element, and sends the display indication information of the derived page element to the second device; The second device displays the derived page element according to the display indication information of the derived page element.

22. The system of claim 21, wherein: The first device opens the target file and displays at least a portion of the content of the target file on a display interface of the first device.

23. The system of claim 22, wherein: The first device receives a first instruction; The first device identifies page elements in the at least a portion of the content according to the first instruction.

24. The system of claim 23, wherein: The first instruction includes location information, where the location information represents a range selected by a user in a display interface of the first device; The first device identifies page elements in the selection range; The page element recognized by the first device in the selection range is determined as the first page element.

25. The system of claim 22, wherein: The first device receives a second instruction, where the second instruction indicates a page element selected by a user in the at least a portion of the content; And the first device determines the selected page element as the first page element based on the second instruction.

26. The system of claim 21, wherein: The first device receives a fifth instruction, where the fifth instruction instructs to modify a display style of the first page element; The first device modifies the display style of the first page element based on the fifth instruction to obtain the derived page element.

27. The system of claim 26, wherein: The element type of the first page element is a text type; The first device modifies the display style of the first page element in at least one of the following ways: Adjust fonts; Adjust font size; Modify the content of the first page element; Rearrange the text.

28. The system of claim 26, wherein: The element type of the first page element is a picture type; The first device modifies the display style of the first page element in at least one of the following ways: Adjust the resolution; Resize; Add or remove image elements.

29. The system of claim 26, wherein: The element type of the first page element is a table type; The display style of the first page element is modified in at least one of the following ways: Adjust the number of rows; Adjust the number of columns; Modify at least one data.

30. The system of claim 21, wherein: The element type of the first page element is a first element type, and the first element type is any one of a text type, a table type, and a picture type; The first device receives a sixth instruction, where the sixth instruction instructs to modify the element type of the first page element to a second element type; The first device generates a derived page element based on the sixth instruction and the first page element, and the element type of the derived page element is the second element type.

31. The system of claim 30, wherein: The first element type is a text type, and the second element type is a picture type; or, The first element type is a text type, and the second element type is a table type; or, The first element type is a picture type, and the second element type is a text type; or, The first element type is a picture type, and the second element type is a table type; or, The first element type is a table type, and the second element type is a text type; or, The first element type is a table type, and the second element type is a picture type; or, The first element type is a table type, and the second element type is a chart type.

32. A screen projection display system, wherein: The device comprises a first device and a second device, wherein the first device and the second device are in communication connection, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal; The first device obtains a target file; The first device displays at least a portion of the content of the target file, generates display indication information of the at least a portion of the content, and sends the display indication information of the at least a portion of the content to the second device; The second device displays the at least part of the content according to the display instruction information of the at least part of the content; The second device receives a seventh instruction, where the seventh instruction includes location information, where the location information represents a selection range in a display interface of the second device; The second device generates a target image corresponding to the selection range and displays the target image, wherein a proportion of the target image in the display interface is greater than a proportion of the selection range in the display interface.

33. The system of claim 32, wherein: The second device performs quality enhancement processing on the target image and displays the enhanced image.

34. A screen projection display system, wherein: The device comprises a first device and a second device, wherein the first device and the second device are in communication connection, the first device is a screen projection sending terminal, and the second device is a screen projection receiving terminal; The first device obtains a target file; The first device opens the target file and displays at least a portion of the content of the target file on a display interface of the first device; The first device generates display indication information of the at least one portion of the content, and sends the display indication information of the at least one portion of the content to the second device; The second device identifies a page element in the at least a portion of the content according to the display indication information of the at least a portion of the content, determines a second page element according to the identified page element, and displays the second page element.

35. The system of claim 34, wherein: The second device receives an eighth instruction, where the eighth instruction may include location information, where the location information indicates that a user selects a range in a display interface of the second device; The second device identifies page elements in the selection range according to the eighth instruction; The second device determines the recognized page element in the selection range as the second page element.

36. The system of claim 34, wherein: The second device receives a ninth instruction, wherein the ninth instruction indicates a page element selected by a user from among the identified page elements; The second device determines the selected page element as the second page element based on the ninth instruction.

37. The system of claim 34, wherein: The second device identifies an element type of each of the identified page elements; The second device receives a tenth instruction, wherein the tenth instruction instructs a user to select at least one element type from among all identified element types of the page elements; The second device determines, based on the tenth instruction, at least one page element of the selected at least one element type as the second page element.

38. The system of claim 34, wherein: The second device generates at least one third target window, where the at least one third target window is used to display the second page element.

39. The system of claim 34, wherein: The second device identifies an element type of each page element in the second page elements, the second page elements including page elements of at least one element type; The second device generates at least one fourth target window, where the at least one fourth target window is used to display the second page element, wherein one of the fourth target windows is used to display a page element of one element type among the at least one element type.

40. The system of claim 39, wherein: The second page element includes page elements of multiple element types; The at least one fourth target window is a plurality of the second target windows, wherein different fourth target windows are used to display page elements of different element types.

41. The system of claim 34, wherein: The second device receives an eleventh instruction, where the eleventh instruction instructs to modify a display style of the second page element; The second device modifies the display style of the second page element based on the eleventh instruction, and displays the modified display style.

42. The system of claim 41, wherein: The second page element is of a third element type, and the first element type is any one of a text type, a table type, and a picture type; The second device receives a twelfth instruction, where the twelfth instruction instructs to modify the element type of the second page element to a fourth element type; The second device generates a derived page element corresponding to the second page element based on the twelfth instruction, and displays the derived page element, where the element type of the derived page element is the fourth element type.

43. A screen projection display method, wherein: Applied to a first device, the method includes: The first device acquires a target file, where the target file includes at least one page element; The first device identifies a page element in the target file; The first device determines a first page element according to the identified page elements in the target file, generates display indication information of the first page element, and sends the display indication information of the first page element to the second device.

44. A screen projection display method, wherein: Applied to a first device, the method includes: The first device acquires a target file, where the target file includes at least one page element; The first device identifies a page element in the target file; The first device determines a first page element according to the identified page element in the target file; The first device generates a derivative page element of the first page element, generates display indication information of the derivative page element, and sends the display indication information of the first page element to the second device.

45. A screen projection display method, wherein: Applied to the second device, the method includes: The second device receives display indication information of a first page element, where the first page element is a page element in a target file acquired by the first device; The second device displays the first page element according to the display indication information of the first page element.

46. ​​A screen projection display method, wherein: Applied to the second device, the method includes: The second device receives display indication information of a derived page element, where the derived page element is generated by the first device based on the first page element in the target file; The second device displays the derived page element according to the display indication information of the derived page element.

47. A screen projection display method, wherein: Applied to the second device, the method includes: The second device receives display indication information of at least a portion of the content of the target file, where the display indication information of at least a portion of the content is generated by the first device based on at least a portion of the content of the target file displayed by the first device; The second device displays the at least part of the content according to the display instruction information of the at least part of the content; The second device receives a seventh instruction, where the seventh instruction includes location information, where the location information represents a selection range in a display interface of the second device; The second device generates a target image corresponding to the selection range and displays the target image, wherein a proportion of the target image in the display interface is greater than a proportion of the selection range in the display interface.

48. A screen projection display method, wherein: Applied to the second device, the method includes: The second device receives display indication information of at least a portion of the content of the target file, where the display indication information of at least a portion of the content is generated by the first device based on at least a portion of the content of the target file displayed by the first device; The second device identifies the page element in the at least a portion of the content according to the display indication information, determines the second page element according to the identified page element, and displays the second page element.

49. An electronic device, wherein: A first device applied to a screen projection display system, the screen projection display system further comprising a second device, the first device and the second device being communicatively connected, the first device being a screen projection sending terminal, and the second device being a screen projection receiving terminal; The electronic device comprises a display and a controller, wherein: The display is used to display content; The controller is configured to perform: Acquire a target file, wherein the target file includes at least one page element; Identifying page elements in the target file; A first page element is determined according to the identified page elements in the target file, display indication information of the first page element is generated, and the display indication information of the first page element is sent to the second device.

50. The apparatus of claim 49, wherein: The controller is further configured to perform: The target file is opened, and at least a portion of the content of the target file is displayed on a display interface of the first device.

51. The apparatus of claim 50, wherein: The controller is specifically configured to execute: Based on the currently displayed content of the target file in the display interface of the first device, a page element of the currently displayed content is identified.

52. The apparatus of claim 50, wherein: The controller is specifically configured to execute: receiving a first instruction; According to the first instruction, a page element in the at least a portion of the content is identified.

53. The apparatus of claim 52, wherein: The first instruction may include position information, where the position information represents a range selected by a user in a display interface; The controller is specifically configured to execute: Page elements within the selection are identified.

54. The apparatus of claim 52, wherein: The first instruction is sent by the second device to the electronic device, or is triggered on the electronic device.

55. The apparatus of claim 50, wherein: The controller is specifically configured to execute: The at least a portion of the content is detected to identify a plurality of page elements.

56. The apparatus of claim 50, wherein: The controller is specifically configured to execute: The page elements in the at least a portion of the content are detected, a plurality of page elements are identified, and an element type of each page element is determined, where the element type is any one of a text type, a picture type, and a table type.

57. The apparatus of claim 49, wherein: The controller is specifically configured to execute: The target file is parsed to identify page elements in the target file and the element type of each page element.

58. The apparatus of claim 53, wherein: The controller is specifically configured to execute: The identified page element in the selection range is determined as the first page element.

59. The apparatus of claim 51, wherein: The controller is specifically configured to execute: receiving a second instruction, wherein the second instruction indicates a page element selected by a user in the at least a portion of the content; Based on the second instruction, the selected page element is determined as the first page element.

60. The device of claim 55 or 56, wherein the controller is specifically configured to perform: receiving a third instruction, wherein the third instruction indicates at least one element type selected by a user from among all the identified element types of the page elements; Based on the third instruction, at least one page element of the selected at least one element type is determined as the first page element.

61. The apparatus of claim 49, wherein: The controller is specifically configured to execute: Receive search information; Searching for page elements containing the search information, or searching for page elements whose element type is the search information, among the identified page elements in the target file; The found page element is determined as the first page element.

62. The apparatus of claim 49, wherein: The controller is specifically configured to execute: Determining a target element type of the first page element; Generate display indication information of the first page element according to the target data format corresponding to the target element type, and send the display indication information of the first page element to the second device, wherein the display indication information of the first page element includes the target element data of the first page element in the target data format.

63. The apparatus of claim 49, wherein: The controller is further configured to perform: generating at least one first target window, wherein the at least one first target window is used to display the first page element; The at least one first target window is recorded to generate first video data corresponding to the recorded content, and the display indication information of the first page element includes the first video data.

64. The apparatus of claim 49, wherein: The controller is further configured to perform: identifying an element type of each page element in the first page elements, the first page elements including page elements of at least one element type; At least one second target window is generated, and the at least one second target window is used to display the first page element, wherein one second target window is used to display a page element of one element type among the at least one element type.

65. The apparatus of claim 64, wherein: The first page element includes page elements of multiple element types; The at least one second target window is a plurality of second target windows, wherein different second target windows are used to display page elements of different element types.

66. The apparatus of claim 63, wherein: The controller is further configured to perform: receiving a fourth instruction, wherein the fourth instruction instructs switching a current page element in the first target window; According to the fourth instruction, the page element in the first target window is switched to the next page element of the current page element.

67. The apparatus of claim 49, wherein: The controller is specifically configured to execute: In response to the identification instruction, identifying the page element in the target file; The identification instruction is triggered by a user on the first device; or the identification instruction is triggered by a user on the second device and sent by the second device to the first device.

68. An electronic device, wherein: A first device applied to a screen projection display system, the screen projection display system further comprising a second device, the first device and the second device being communicatively connected, the first device being a screen projection sending terminal, and the second device being a screen projection receiving terminal; The electronic device comprises a display and a controller, wherein: The display is used to display content; The controller is configured to perform: Acquire a target file, wherein the target file includes at least one page element; Identifying page elements in the target file; Determining a first page element according to the identified page element in the target file; Generate a derived page element of the first page element, generate display indication information of the derived page element, and send the display indication information of the derived page element to the second device.

69. An electronic device, wherein: A second device applied to a screen projection display system, the screen projection display system further comprising a first device, the first device and the second device being communicatively connected, the first device being a screen projection sending terminal, and the second device being a screen projection receiving terminal; The electronic device comprises a display and a controller, wherein: The display is used to display content; The controller is configured to perform: receiving display indication information of at least a portion of the content of a target file, where the display indication information is generated by the first device based on at least a portion of the content of the target file displayed by the first device; According to the display indication information of at least a part of the content, a page element in the at least a part of the content is identified, and a second page element is determined according to the identified page element, and the second page element is displayed.

70. The apparatus of claim 69, wherein: The controller is specifically configured to execute: receiving an eighth instruction, wherein the eighth instruction may include position information, wherein the position information indicates that a user selects a range in the display interface; According to the eighth instruction, identifying page elements in the selection range; The identified page element in the selection range is determined as the second page element.

71. The apparatus of claim 69, wherein: The controller is specifically configured to execute: receiving a ninth instruction, wherein the ninth instruction indicates a page element selected by a user from among the identified page elements; And based on the ninth instruction, the selected page element is determined as the second page element.

72. The apparatus of claim 69, wherein: The controller is specifically configured to execute: Identifying the element type of each of the identified page elements; receiving a tenth instruction, wherein the tenth instruction indicates at least one element type selected by the user from among all the identified element types of the page elements; Based on the tenth instruction, at least one page element of the selected at least one element type is determined as the second page element.

73. The apparatus of claim 69, wherein: The controller is specifically configured to execute: At least one third target window is generated, where the at least one third target window is used to display page elements in the at least one partial content.

74. The apparatus of claim 69, wherein: The controller is specifically configured to execute: identifying an element type of each of the second page elements, the second page elements including page elements of at least one element type; At least one fourth target window is generated, and the at least one fourth target window is used to display the first page element, wherein one of the fourth target windows is used to display a page element of one element type among the at least one element type.

75. The apparatus of claim 74, wherein: The second page element includes page elements of multiple element types; The at least one fourth target window is a plurality of the second target windows, wherein different fourth target windows are used to display page elements of different element types.

76. The apparatus of claim 69, wherein: The controller is also configured to perform: receiving an eleventh instruction, wherein the eleventh instruction instructs to modify a display style of the second page element; Based on the eleventh instruction, the display style of the second page element is modified, and the modified display style is displayed.

77. The apparatus of claim 76, wherein: The second page element is of a third element type, and the first element type is any one of a text type, a table type, and a picture type; The controller is also configured to perform: receiving a twelfth instruction, wherein the twelfth instruction instructs to modify the element type of the second page element to a fourth element type; Based on the twelfth instruction, a derived page element corresponding to the second page element is generated and displayed, and the element type of the derived page element is the fourth element type.

78. A computer-readable storage medium, wherein: The computer-readable storage medium is used to store a computer program, and when the computer program is run on a computer, the computer is caused to perform the method as claimed in claim 43.

79. A computer-readable storage medium, wherein: The computer-readable storage medium is used to store a computer program, which, when executed on a computer, causes the computer to execute the method as claimed in claim 44 .

80. A computer-readable storage medium, wherein: The computer-readable storage medium is used to store a computer program, and when the computer program is run on a computer, the computer is caused to execute the method according to claim 45 .

81. A computer-readable storage medium, wherein: The computer-readable storage medium is used to store a computer program, and when the computer program is run on a computer, the computer is caused to perform the method according to claim 46.

82. A computer-readable storage medium, wherein: The computer-readable storage medium is used to store a computer program, and when the computer program is run on a computer, the computer is caused to execute the method as claimed in claim 47.

83. A computer-readable storage medium, wherein: The computer-readable storage medium is used to store a computer program, which, when executed on a computer, causes the computer to perform the method according to claim 48.

Images