Data collection method and device and storage medium

Abstract

The invention discloses a data collection method and device and a storage medium, and relates to the technical field of terminals. According to the method, the electronic equipment displays a first interface, first selected data is obtained in response to a selection operation for a first material on the first interface, and the first selected data is in a picture-text mixed arrangement format. And in response to a first operation for the first selected data, retaining the distribution position and sequence of the first selected data in the page where the first material is located, and storing the first selected data in the instance storage unit in a rich text structure. The user drags the first selected data, the first selected data moves along with the first selected data, when the first selected data moves to an area corresponding to the collection space, it is indicated that the user wants to collect the first selected data, and the collection space obtains the rich text structure from the instance storage unit and collects the first selected data. Therefore, collection of the data in the image-text mixed arrangement format is realized, a user does not need to collect texts and images respectively, and the collection efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and particularly to a data collection method, device, and storage medium. Background Art

[0002] With the continuous development of terminal technology, users will come into contact with a large amount of information in their daily lives. Some information is presented in the form of a mixed text and image layout (including images and text) on a page. When collecting data, users will select a large area of the page, which includes both images and text.

[0003] However, for the collection of offline content in the mixed text and image format, there is no third-party application that supports the collection and viewing of such types of materials. Therefore, how to achieve the collection of data in the mixed text and image format is an urgent problem to be solved at present. Summary of the Invention

[0004] This application provides a data collection method, device, and storage medium, which can achieve the collection of data in the mixed text and image format and improve the data collection efficiency. The technical solutions are as follows:

[0005] In a first aspect, a data collection method is provided. The method includes: an electronic device displays a first interface, and the user performs a selection operation on a first material on the first interface. The first material is a rich text in the mixed text and image format, that is, it includes both an image and text. The electronic device, in response to the selection operation, obtains first selected data selected by the user on the first material. The first selected data includes a first image and a first text. The user performs a first operation on the first selected data. The first operation can be a long-press operation or a touch operation, indicating that the user may have the intention to collect the first selected data. The electronic device, in response to the first operation, retains the distribution positions and order of the first image and the first text in the page where the first material is located, and stores them in an instance storage unit in a rich text structure. The first selected data can be moved following the user's drag operation. When it is moved to the area corresponding to the collection space (which can be a hidden area or a floating area, and the area of the hidden area is smaller than that of the floating area), it indicates that the user wants to collect the first selected data. The collection space obtains the rich text structure from the instance storage unit and collects the first selected data according to the rich text structure.

[0006] In some embodiments of this application, the data collection method can be applied to an electronic device, which can be a device with a display screen such as a mobile phone, a tablet computer, a desktop computer, a portable computer, a notebook computer, a handheld computer, an e-book reader, a television, a smart wearable device, a network server, a wireless terminal device, a communication device, or an embedded device.

[0007] In some embodiments of the present application, the materials may include data of types such as text and images; among them, the images may be static images or dynamic images, and the present application does not limit this.

[0008] In some embodiments of the present application, the collection space is a management application (Application, APP), which has the function of collecting any type of materials and can be regarded as a storage container for collecting rich text structures. When the user performs a first operation on the first selected data, it indicates that the user may have the intention to collect the first selected data, which can trigger the startup of the collection space application. When the user performs a drag operation on the first selected data, it indicates that the user wants to collect the first selected data, which can trigger the startup of the collection space application. Of course, the user can also open the collection space application before collecting materials, and the embodiments of the present application do not limit this.

[0009] In some embodiments of the present application, the instance storage unit (Instance Storage) is used to store one or more instances, and these instances can be data of different types. For example, text, images, and Uniform Resource Identifiers (URIs), etc. A rich text structure is a way to describe and organize text content using Hyper Text Markup Language (HTML) tags or other markup language tags. Using HTML tags can preserve the original format, structure, and semantics of the rich text. Each HTML tag has its specific meaning and use, and together they constitute the structure and content of a web page. By using HTML tags, it can ensure that the browser correctly parses and renders the web page content. In the instance storage unit, various tags can be used to add different elements, such as headings, paragraphs, links, and pictures, etc. These elements are rich text content, and these elements can be stored as strings or other data types. In this way, the instance storage unit can realize storing the data after instantiating it into a rich text structure, and reflecting the positions of the images and text on the page through HTML tags, so as to facilitate subsequent processing, display, and operation of the rich text structure.

[0010] In some embodiments of the present application, when the user performs a drag operation on the first selected data and moves it to the area corresponding to the collection space, it triggers the collection space to obtain the rich text structure from the instance storage unit. The rich text structure includes the data and the data type of the data (for example, HTML tags). The collection space can collect the data in a scattered manner according to the data type, that is, collect text and images separately. The collection space can also generate an offline web page file according to the data type and the data. The distribution positions and orders of the first image and the first text on the page where the first material is located are retained in the offline web page file, and then, the offline web page file is collected.

[0011] In some embodiments of the present application, by encapsulating the first selected data into a rich text structure for storage, the key information in the format of mixed text and graphics can be retained to the greatest extent. Then, by means of the collection space for collection, the discrete data or offline web page files collected include images, texts, the relevance between images and texts, and the relevance between texts and texts, so as to realize the collection of data in the format of mixed text and graphics, without the user having to collect texts and images separately, thus improving the collection efficiency.

[0012] In a possible implementation manner, in response to a first operation on the first selected data, the first selected data is stored in the instance storage unit in a rich text structure. The method includes: the user performs a first operation on the first selected data, and the electronic device responds to the first operation. If the application to which the first material belongs itself supports dragging of graphic and text content, the first selected data is stored in the instance storage unit in a rich text structure.

[0013] In some embodiments of the present application, the user selects materials in the application. The application can be a native application (such as a phone application, a text message application, a browser, a file manager, etc.) that comes with the electronic device, or a third-party application (such as a third-party browser). Some applications have the function of supporting dragging of graphic and text content (such as the browser that comes with the electronic device, a third-party browser, etc.), and some applications do not have the function of supporting dragging of graphic and text content. If the application to which the first material belongs has the function of supporting dragging of graphic and text content, the first selected data is stored in the instance storage unit in a rich text structure. Then, when the user wants to collect the first selected data, the collection space obtains the rich text structure from the instance storage unit for data collection, without the user having to drag texts and images separately, thus improving the collection efficiency.

[0014] In a possible implementation manner, in response to a first operation on the first selected data, the first selected data is stored in the instance storage unit in a rich text structure. The method includes: the user performs a first operation on the first selected data, and the electronic device responds to the first operation. In the case where the application to which the first material belongs itself does not support dragging of graphic and text content, if the application inherits the function of selecting a content area through a cursor in the operating system of the electronic device, which means that the user can select the required graphic and text content area by moving the cursor, the view component (View) in the electronic device obtains the selected area corresponding to the first selected data; identifies the resource identifier of the first image and the first text in the selected area to obtain identification data; and stores the identification data in the instance storage unit in a rich text structure.

[0015] In some embodiments of the present application, the resource identifier of an image may refer to the Uniform Resource Identifier (URI) of the image. A URI is a string used to identify and locate an image resource. The URI is used to specify the location and access method of the image. The URI of an image can represent the download link of the image, and the image can be obtained from a certain location on the network through the URI of the image. The View component recognizes the text in the selected area to obtain the text, and the View component recognizes the image in the selected area to obtain the URI of the image. The recognition data includes the text and the URI of the image, and the recognition data is stored in the instance storage unit in a rich text structure. Further, when the user wants to collect the first selected data, the collection space obtains the rich text structure from the instance storage unit for data collection, without the user dragging the text and the image separately, improving the collection efficiency.

[0016] In some embodiments of the present application, for the case where the application itself does not support dragging of text and image content, if the application inherits the function of selecting a content area through a cursor in the operating system of the electronic device, the View component recognizes the selected area, and then stores the recognition data in the instance storage unit in a rich text structure. In this way, applications that do not support dragging of text and image content can also be applicable to the collection method of text and image content, improving the richness of application scenarios.

[0017] In a possible implementation manner, in response to a selection operation on a first piece of material, first selected data is obtained. The method includes: the user performs a first operation on the first selected data, and the electronic device, in response to the first operation, obtains second selected data, where the second selected data includes any data in the first piece of material; the user performs a second operation on the second selected data, and the second operation may be a long-press operation or a touch operation, indicating that the user may have the intention of collecting the second selected data. In the case where the application to which the first piece of material belongs does not support dragging of text and image content and the application does not inherit the function of selecting a content area through a cursor in the operating system of the electronic device, the View component obtains the page content of the page where the first piece of material is located, and generates an additional view above the page. The additional view includes the page content, and the additional view is generated by the View component and has the function of supporting the selection of a content area through a cursor. The user performs a selection operation on the data in the additional view, and the electronic device, in response to the selection operation, obtains the first selected data.

[0018] In some embodiments of the present application, the user performs a first operation on the first selected data. The first operation may be a long - press operation or a touch operation, indicating that the user may intend to collect the first selected data. If the application to which the first material belongs does not support dragging of graphic and text content and the application does not inherit the function of selecting a content area through a cursor in the operating system of the electronic device, the collection of graphic and text content cannot be directly achieved. Therefore, it is necessary for the view component to regenerate an additional view. The additional view is set above the page and is not blocked by the page, for the user to perform a selection operation on the additional view again.

[0019] In some embodiments of the present application, for the case where the application itself does not support dragging of graphic and text content and does not inherit the function of selecting a content area through a cursor in the operating system of the electronic device, the entire page where the first material is located is used as the target. The view component generates an additional view above the page according to the page content. The additional view reproduces the page content for the user to perform a selection operation again. In this way, an application that does not inherit the function of selecting a content area through a cursor in the operating system of the electronic device can also be applicable to the collection method of graphic and text content, improving the richness of application scenarios.

[0020] In a possible implementation manner, the user performs a third operation on the first interface. The third operation may be a screenshot operation. In response to the third operation, the view component of the electronic device obtains the page content of the entire page and generates an additional view above the page where the first material is located. The additional view includes the page content of the page where the first material is located. The additional view is generated by the view component and has the function of supporting the selection of a content area through a cursor. The data in the additional view is the same as the first material. The user performs a selection operation on the first material in the additional view, and the electronic device obtains the first selected data in response to the selection operation.

[0021] In some embodiments of the present application, the user performs a third operation on the interface. The third operation may be a screenshot operation. The screenshot operation is a three - finger swipe - down operation provided in the embodiments of the present application. The three - finger swipe - down operation is a convenient operation for taking a screenshot of the current interface. By using the three - finger swipe - down operation, a specific area or the entire screen on the interface can be captured and saved as a picture, and the steps for collecting receipts are triggered. In this scenario, the electronic device cannot directly achieve the collection of graphic and text content on the page. Therefore, it is necessary for the view component to regenerate an additional view. The additional view is set above the page and is not blocked by the page, for the user to perform a selection operation on the additional view again.

[0022] In some embodiments of the present application, for the scenario of taking a screenshot by swiping down with three fingers, the view component can be triggered by the three-finger down swipe operation to target the entire page where the first material is located. An additional view is generated above the page according to the page content. The additional view reproduces the page content for the user to perform a selection operation again. The present application provides various collection schemes for graphic and text content, improving the richness of application scenarios.

[0023] In a possible implementation manner, in response to a first operation on the first selected data, storing the first selected data in a rich text structure in an instance storage unit includes: the user performs a first operation on the first selected data, and in response to the first operation, the view component in the electronic device obtains the selected area corresponding to the first selected data; identifying the resource identifier of the first image and the first text in the selected area to obtain identification data; storing the identification data in a rich text structure in the instance storage unit.

[0024] In some embodiments of the present application, for the scenario where the application itself does not support dragging of graphic and text content and does not inherit the function of selecting a content area through a cursor in the operating system of the electronic device, as well as the scenario of taking a screenshot by swiping down with three fingers, the additional view is generated by the view component and has the function of supporting the selection of a content area through a cursor. The view component identifies the selected area and stores a rich text structure in the instance storage unit. In this way, applications that do not support dragging of graphic and text content and the scenario of taking a screenshot by swiping down with three fingers can also be applicable to the collection method of graphic and text content, improving the richness of application scenarios.

[0025] In a possible implementation manner, generating an additional view above the page where the first material is located includes: obtaining, by the view component, the resource identifier of the image set in the page where the first material is located and the positions of each image in the image set in the page; the image set includes the first image; performing optical character recognition on the screenshot corresponding to the page by an optical character recognition component to obtain a text set and the positions of each text in the text set in the page; the text set includes the first text; generating, by the view component, an additional view above the page according to the image set extracted based on the resource identifier of the image set, the text set, and the positions of each text in the text set in the page.

[0026] In some embodiments of the present application, the page where the first material is located includes a first image and a first text. The page may further include images other than the first image, and texts other than the first text, that is, the page includes at least one image and at least one text. Taking the entire page where the first material is located as the target, the view component recognizes the entire page to obtain the resource identifiers of the images in the image set and the positions of the images in the page. The view component sends the screenshot corresponding to the page to the text recognition component, and the text recognition component performs text recognition on the screenshot corresponding to the page to obtain each text in the text set and the position of each text in the page. The text recognition component returns the recognized texts and the positions of the texts in the page to the view component. Then, the view component obtains each image according to the resource identifier of each image, and generates an additional view above the page according to each image, the position of each image in the page, each text, and the position of each text in the page.

[0027] Among them, the text recognition component can be implemented by using Optical Character Recognition (OCR) technology.

[0028] In some embodiments of the present application, the view component recognizes the resource identifiers of the images in the page and the positions of the images in the page; the text recognition component performs text recognition on the screenshot corresponding to the page to obtain each text and its position in the page. Then, an additional view is generated above the page according to the results of image recognition and text recognition. The additional view re-presents the page content for the user to perform the selection operation again. In this way, applications that do not support dragging of graphic and text content and scenarios of taking screenshots by swiping down with three fingers can also be applicable to the collection method of graphic and text content, improving the richness of application scenarios.

[0029] In a possible implementation manner, if the second text is included in the first image, the text set further includes the second text.

[0030] In some embodiments of the present application, if the image includes text, when the text recognition component performs text recognition on the screenshot, the text recognition component also recognizes the text in the image and returns it to the view component. The additional view generated by the view component above the page also includes the text in the image, increasing the richness of the displayed content in the additional view.

[0031] In a possible implementation manner, the additional view at least includes the first text, the first image, and the second text in the first image; the second selected data or the third selected data includes the first text, the first image, the second text, and / or other objects, and the other objects are objects in the page other than the first text, the first image, and the second text.

[0032] In some embodiments of the present application, the additional view generated by the view component above the page further includes the text in the image. When the user performs a selection operation on the additional view, the user can select the same data as the first selected data, that is, the first text and the first image. The selected data can also include the text in the image, that is, the first text, the first image, and the second text, or can also include other objects, that is, the first text, the first image, and other objects. The selected data can also include the text in the image and other objects. The selected data can also be any other text and image. The embodiments of the present application do not limit this.

[0033] In some embodiments of the present application, the user can also select the text in the image and collect the text in the image, which enriches the data collection and improves the user experience.

[0034] In a possible implementation manner, after collecting the first selected data in the collection space according to the rich text structure, the method further includes: the electronic device displays a second interface; the second interface includes a floating region corresponding to the collection space, and a first card group corresponding to the first selected data is displayed in the floating region, and the first card group includes a first card corresponding to the first image and a second card corresponding to the first text.

[0035] In some embodiments of the present application, the collection space is a storage container for collecting rich text structures. The floating region corresponding to the collection space is displayed on the interface and is a convenient entry for the user to view and use the collected data.

[0036] The collection space obtains the rich text structure from the instance storage unit. The rich text structure includes data and the data type of the data. The collection space can collect the data in a scattered manner according to the data type, that is, the text and the image are collected separately. The first selected data includes the first text and the second image. These two cards are displayed in the floating region in the form of a card group, indicating that the image and the text come from the same collection, and the relevance between the image and the text is retained.

[0037] In some embodiments of the present application, when the user collects graphic and text content, the user usually wants to use the text or the image again. The collection space collects the data in a scattered manner according to the rich text structure, that is, the image and the text are collected separately, which can facilitate the user to use the collected image and text, improve the use efficiency of the data, and improve the user experience. And the collection space retains the relevance between the image and the text in the form of a card group, improving the visibility of the data collection.

[0038] In a possible implementation manner, the first card group further includes a first icon and / or a first name, and both the first icon and the first name indicate the application source identifier of the first selected data.

[0039] In some embodiments of the present application, by displaying the icons or names of applications on the cards to prompt the user about the application sources of each card group, the diversity and richness of the interface display are increased, and the fun of human-computer interaction is enhanced.

[0040] In a possible implementation manner, the first text includes at least one paragraph, and the first image includes at least one picture; the line spacing between any two paragraphs is greater than a preset spacing, or the first image is located between any two paragraphs; the first card group includes cards corresponding to each picture and cards corresponding to each paragraph; the cards corresponding to each picture and the cards corresponding to each paragraph are arranged in the order of the positions of each picture and each paragraph on the page where the first material is located.

[0041] In some embodiments of the present application, the first text may include one or more paragraphs, and the first image may include one or more pictures. In the above display manner, the text is all displayed through one card, and the image is all displayed through another card, that is, the image and the text are displayed separately. In this example, if the line spacing between any two paragraphs is greater than the preset spacing, the two paragraphs are respectively displayed through two cards. If the two paragraphs are separated by an image, the two paragraphs are respectively displayed through two cards. The pictures in the image are respectively displayed through cards. The above display manner can be appropriately selected by those skilled in the art according to the actual situation.

[0042] The first card group includes cards corresponding to each picture and cards corresponding to each paragraph; the sorting order of the cards can be arranged in the order of the positions of each picture and each paragraph on the page where the first material is located, retaining the relevance between the image and the text, the relevance between the images, and the relevance between the texts, and improving the visibility of data collection.

[0043] In a possible implementation manner, the first card includes the first image, or a thumbnail or partial screenshot corresponding to the first image; the second card includes all or part of the characters in the first text.

[0044] In some embodiments of the present application, due to the limited display area of the interface, the data displayed on the cards in the floating state area can be all or part of the complete data. For short texts, for example, the number of characters corresponding to the text is less than or equal to the maximum number of characters that the card can display, the card displayed in the floating state area includes all the characters of the text. For long texts, for example, the number of characters corresponding to the text is greater than the maximum number of characters that the card can display, the card displayed in the floating state area includes part of the characters of the text. For small-sized images, for example, the size of the image is less than or equal to the maximum size of the card, the card displayed in the floating state area includes the image. For large-sized images, for example, the size of the image is greater than the maximum size of the card, the card displayed in the floating state area includes a thumbnail or a partial screenshot of the image.

[0045] Display as much data related to the data as possible in the limited area, which is convenient for the user to know the general information of each card and improves the user experience.

[0046] In a possible implementation manner, after the second interface is displayed, the method further includes: the user performs a triggering operation on the first card group, and the electronic device displays the first card and the second card in the floating state area in response to the triggering operation.

[0047] In some embodiments of the present application, the triggering operation may be a click operation, a touch operation, or a press operation. The user can open the first card group through the triggering operation to display the first card and the second card in the first card group. This is convenient for the user to further view and reuse each card and improves the user experience.

[0048] In a possible implementation manner, after the first card and the second card are displayed in the floating state area, the method further includes: the user performs a dragging operation on any card in the first card group, and the electronic device displays the third interface in response to the dragging operation; the third interface includes a first editing area, and the editing data corresponding to the card is displayed in the first editing area.

[0049] In some embodiments of the present application, the dragging operation may be a touch action or gesture such as dragging, pulling, dragging, towing, or traction.

[0050] In some embodiments of the present application, the data collected in the collection space is complete data. Due to the limited display area of the interface, the data displayed on the cards in the floating state area can be all or part of the complete data. When a card is selected, the complete data in the collection space is called, so that the complete data is displayed in the editing area, and the edited data represents the complete data. The user can reuse the collected data through a drag operation. Compared with the path of reusing data from a web page, the data reuse path of this method is shorter, improving the usage efficiency of the collected data. Moreover, the user can continue to perform operations such as deleting and changing the edited data, improving the flexibility of reusing data.

[0051] In a possible implementation manner, after the second interface is displayed, the method further includes: the user performs a drag operation on any card group, and the electronic device responds to the drag operation and displays a fourth interface; the fourth interface includes a second editing area, and the editing data corresponding to the multiple cards included in the card group is displayed in the second editing area.

[0052] In some embodiments of the present application, the user can reuse the data in any card or the data in any card group. When reusing the data in a card group, by calling the complete data of each card in the card group in the collection space, the editing data of the card group is displayed in the editing area, and the editing data of the card group includes the complete data corresponding to each card.

[0053] By reusing the data in the card group, the number of drag operations can be reduced, the data reuse efficiency can be improved, and the user experience can be improved.

[0054] In a possible implementation manner, after the second interface is displayed, the method further includes: the user performs a merging operation on any card group towards another card group, and the electronic device responds to the merging operation and displays the merged card group in the floating state area, and the merged card group includes the cards included in these two card groups.

[0055] In some embodiments of the present application, the user can merge any two card groups to reduce the number of card groups currently displayed in the floating state area, so as to display other card groups in the floating state area, improving the visibility of data collection.

[0056] In a possible implementation manner, after the first selected data is collected in the collection space according to the rich text structure, the method further includes: the electronic device displays a fifth interface; the fifth interface includes a floating state area corresponding to the collection space, and a single card corresponding to the first selected data is displayed in the floating state area, and the single card includes an offline web page file corresponding to the first selected data, and the offline web page file is generated by the collection space according to the rich text structure.

[0057] In some embodiments of the present application, the collection space obtains a rich text structure from the instance storage unit. The rich text structure includes data and the data type of the data. The collection space can generate an offline web page file according to the data type and the data. The distribution positions and order of the first image and the first text on the page where the first material is located are retained in the offline web page file, and the offline web page file is collected. And the offline web page file is displayed in the floating area in the form of a single card, retaining the original format of the first selected data on the page.

[0058] In some embodiments of the present application, when the user collects the content with mixed graphics and text, in some scenarios, the user wants to view the content with mixed graphics and text again. The collection space collects the data in the form of an offline web page according to the rich text structure, that is, the original format of the image and text on the page is retained, which is convenient for the user to view the content with mixed graphics and text again, improves the intuitiveness of the data, and improves the user experience.

[0059] In a possible implementation manner, after the fifth interface is displayed, the method further includes: the user performs a trigger operation on any single card in the floating area, and the electronic device responds to the trigger operation and displays at least one option; the option indicates a browsing engine or an application supporting web browsing; the user performs a trigger operation on any option, and the electronic device responds to the trigger operation and displays the web page corresponding to the single card.

[0060] In some embodiments of the present application, the trigger operation may be a click operation, a touch operation or a press operation. The electronic device responds to the trigger operation of the user on any single card and displays one or more options. The option may be a browsing engine or an application supporting web browsing. The user selects an option and opens the web page corresponding to the single card through the browsing engine or the application. The user can view the content with mixed graphics and text in the web page again, improving the reproduction efficiency of the content with mixed graphics and text.

[0061] In a possible implementation manner, the method further includes: the user performs a first operation on the first selected data, and the electronic device responds to the first operation and opens the collection space, and displays the collection space in a hidden state on the first interface; or, the user performs a drag operation on the first selected data, and the electronic device responds to the drag operation and opens the collection space, and displays the collection space in a hidden state on the first interface.

[0062] In some embodiments of the present application, when a user performs a first operation on the first selected data, it indicates that the user may have the intention to collect the first selected data, which can trigger the launch of the collection space application. When the user performs a drag operation on the first selected data, it indicates that the user wants to collect the first selected data, which can trigger the launch of the collection space application. The collection space is displayed in a hidden state on the first interface, which will not affect the user's normal viewing of the displayed content. By launching the collection space application at different triggering times, the richness of application scenarios is improved.

[0063] In a possible implementation, the upper limit of the number of displayed area elements in the floating state area is a preset number; the area elements are a card group or a single card; the method further includes: if the number of displayed area elements in the floating state area reaches the preset number, the area element with the earliest acquisition timestamp among the displayed area elements is hidden.

[0064] In some embodiments of the present application, due to the limited display area of the interface, the number of area elements displayed in the floating state area is limited. The preset number can be a positive integer, for example, 3, 4, 5, 6, etc. When a new area element needs to be displayed, if the number of displayed area elements in the floating state area reaches the upper limit, the area element that was displayed first needs to be hidden, and at the same time, the new area element is displayed.

[0065] The display order of the area elements in the floating state area can be sorted according to the acquisition timestamps of the area elements. Therefore, the area element with the earliest acquisition timestamp among the currently displayed area elements can be hidden here. The acquisition timestamp of the area element refers to the acquisition timestamp of the first selected data corresponding to the area element on the electronic device.

[0066] By displaying new area elements and hiding the area element with the earliest acquisition timestamp, the latest collected graphic and text content is presented to the user, improving the user experience.

[0067] When a new area element is displayed, the area element with the earliest acquisition timestamp can also be hidden at the same time. The two can be synchronized to achieve a scenario where one area element gradually hides and the other area element gradually shows. It can be understood that the two can also be one after the other, and after one area element is completely hidden, the other area element is displayed.

[0068] In a possible implementation, the floating state area includes a display progress bar, and the display progress bar is used to adjust the progress of the area elements displayed in the floating state area; the method further includes: in response to a fourth operation on the display progress bar, the hidden area elements in the floating state area are displayed.

[0069] In some embodiments of the present application, when the number of displayed regional elements in the suspended state area reaches the upper limit, the earliest displayed regional element is also hidden. There is also a display progress bar in the first suspended state area, which is used to adjust the display progress of the regional elements in the suspended state area. The user can perform a drag operation or a scroll operation on the display progress bar of the regional elements, so as to display other hidden regional elements. This is convenient for the user to view all regional elements and improves the user experience.

[0070] In a second aspect, a computer device is provided. The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the method involved in the first aspect or any implementation manner of the first aspect.

[0071] In a third aspect, a computer-readable storage medium is provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, the computer is caused to execute the method involved in the first aspect or any implementation manner of the first aspect.

[0072] In a fourth aspect, an embodiment of the present application provides a chip system. The chip system can be applied to an electronic device. The chip system includes one or more processors, and the processor is used to call computer instructions to cause the electronic device to execute the method described in the first aspect or any implementation manner of the first aspect.

[0073] In a fifth aspect, a computer program product containing instructions is provided. When it runs on a computer, the computer is caused to execute the data collection method described in the first aspect above.

[0074] The technical effects obtained in the second aspect, the third aspect, the fourth aspect, and the fifth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be elaborated here. Description of the Drawings

[0075] Figure 1 is a schematic structural diagram of a terminal provided by an embodiment of the present application;

[0076] Figure 2 is a block diagram of a software system of a terminal provided by an embodiment of the present application;

[0077] Figure 3 is a schematic diagram of a collection space provided by the present application;

[0078] Figure 4 is the state and display style of the collection space provided by the present application;

[0079] Figure 5 is a schematic diagram of collecting the text and picture mixed content in the interface provided by the present application;

[0080] Figure 6 A schematic diagram for collecting the mixed text and image content in the additional view provided by this application;

[0081] Figure 7 A schematic diagram for a screenshot operation provided by this application;

[0082] Figure 8 A schematic diagram for selecting the mixed text and image content in the additional view provided by this application;

[0083] Figure 9 A schematic diagram for a card group provided by this application;

[0084] Figure 10 A schematic diagram for opening a card group provided by this application;

[0085] Figure 11 Another schematic diagram for opening a card group provided by this application;

[0086] Figure 12 A schematic diagram for reusing the data in a card provided by this application;

[0087] Figure 13 A schematic diagram for reusing the data in a card group provided by this application;

[0088] Figure 14 A schematic diagram for merging card groups provided by this application;

[0089] Figure 15 A schematic diagram for the hidden display of a card group provided by this application;

[0090] Figure 16 A schematic diagram for a single card provided by this application;

[0091] Figure 17 A schematic diagram for viewing the favorite content in a web page provided by this application;

[0092] Figure 18 An exemplary flowchart of a cross - end interaction method provided by an embodiment of this application;

[0093] Figure 19 Another exemplary flowchart of a cross - end interaction method provided by an embodiment of this application. Detailed implementation manners

[0094] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe the implementation manners of this application in detail with reference to the accompanying drawings.

[0095] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means "or". For example, A / B can mean A or B. The "and / or" in this article is just a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to clearly describe the technical solution of this application, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the words such as "first" and "second" do not limit the quantity and execution order, and the words such as "first" and "second" do not necessarily limit to be different.

[0096] Currently, for the offline content in the format of mixed text and graphics, most applications neither support the collection and viewing of such types of materials, nor support the reuse of such types of materials. Although some applications such as browsers can support the collection of the entire web page, for the mixed text and graphics content in a local area of the entire web page, browsers and other applications still cannot achieve the collection of the mixed text and graphics content selected by the user.

[0097] In response to the above problems, this application provides a data collection method that supports the collection of offline content in the format of mixed text and graphics. Through this method, the user can perform a selection operation on the mixed text and graphics materials displayed on the interface, and the electronic device responds to the selection operation to obtain the selected data. When the user performs a long press operation or a touch operation on the selected data, it indicates that the user may have the intention to collect the selected data. The electronic device responds to the long press operation or the touch operation, retains the distribution position and order of the image and text in the page where the mixed text and graphics material is located, and stores the selected data in the instance storage unit in a rich text structure. When the user performs a drag operation on the selected data, it triggers the collection space to obtain the rich text structure from the instance storage unit. The rich text structure includes the data and the data type of the data. By storing it in a rich text structure through encapsulation, the key information of the mixed text and graphics format can be retained to the greatest extent. With the help of the collection space for collection, the collection space can collect the mixed text and graphics content in the form of discrete data or offline web page files. Both the discrete data and the offline web page files include images, texts, the association between images and texts, and the association between texts, so as to realize the collection of data in the mixed text and graphics format, without the user having to collect texts and images separately, improving the collection efficiency.

[0098] In the electronic device of this application, the collection function of the mixed text and graphics content in the local area is extended, enabling the user to retain the key information to the greatest extent with a finer granularity when collecting the mixed text and graphics content, while retaining the association between the information, improving the comprehensiveness of the information in the collected data.

[0099] Before explaining the data collection method provided in the embodiments of the present application in detail, the terminals involved in the embodiments of the present application will be described first.

[0100] Figure 1 It is a schematic structural diagram of a terminal provided in the embodiments of the present application. Refer to Figure 1 , the terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0101] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0102] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0103] Among them, the controller can be the nerve center and command center of the terminal 100. The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0104] A memory can also be set in the processor 110 to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0105] The wireless communication function of the terminal 100 can be implemented by the antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor, and baseband processor, etc.

[0106] The antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the terminal 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0107] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the terminal 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be set in the same device.

[0108] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0109] The wireless communication module 160 may provide solutions for wireless communications applied to the terminal 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive the signals to be transmitted from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0110] In some embodiments, the antenna 1 of the terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technologies.

[0111] The terminal 100 realizes the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change the display information.

[0112] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 100 may include one or N display screens 194, where N is an integer greater than 1.

[0113] The terminal 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, the application processor, etc.

[0114] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0115] The internal memory 121 can be used to store computer-executable program code, and the computer-executable program code includes instructions. The processor 110 executes various functional applications and data processing of the terminal 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the terminal 100 (such as audio data, a phone book, etc.). In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0116] The terminal 100 can implement audio functions, such as music playback, recording, etc., through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor, etc.

[0117] The pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates having conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the terminal 100 detects the intensity of the touch operation according to the pressure sensor 180A. The terminal 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0118] The touch sensor 180K, also known as the "touch panel". The touch sensor 180K may be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as the "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K may also be disposed on the surface of the terminal 100, at a different position from the display screen 194.

[0119] Next, the software system of the terminal 100 will be described.

[0120] The software system of the terminal 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the Android system with a layered architecture is taken as an example to exemplarily describe the software system of the terminal 100.

[0121] Figure 2 It is a block diagram of a software system of a terminal 100 provided by the embodiments of the present application. The software system of the terminal may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, the Android system with a layered architecture is taken as an example to exemplarily illustrate the software structure of the terminal.

[0122] See Figure 2, The layered architecture divides software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android Runtime and the system layer, and the kernel layer.

[0123] The application layer may include a series of application packages. As Figure 2 shown in Figure A, the application packages may include a collection space application, a social application, a shopping application, a news application, a game application, a communication application, etc. The application packages may also include applications such as a camera, a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, a music, a video, a short message, etc.

[0124] Among them, the following operations can be completed in the application layer of the electronic device: open a certain application, for example, a social application, a shopping application, a news application, a game application, a communication application, etc., display it on the interface, and the user performs a selection operation on the graphic and text materials in the application and a drag operation on the selected data.

[0125] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. As Figure 2 shown in Figure A, the application framework layer may include an Input Manager, view components, drag components, text recognition components, global collection and configuration management. The global collection in the application framework layer can support the collection space application in the application layer to implement the global collection ability. As Figure 2 shown in Figure B, the collection space in the application framework layer is a data collection implemented on the basis of the global collection ability. Figure 2 The view components are required for the global collection in Figure B. The collection space in the application framework layer is used to provide data support for the collection space application in the application layer, and the collection space application in the application layer is used to display the graphic and text mixed content collected in the collection space in the application framework layer in the form of a card group or a single card.

[0126] Figure 2 The application in Figure B may be any application that can display graphic and text materials. For example, the above Figure 2Social applications, shopping applications, news applications, game applications, communication applications, etc. shown in Figure A in []. The View framework in the application includes a View Tree, a View Recognizer, and an Activity Thread. The Text View under the root node (Root) of the View Tree is used to display and edit text, and the Image View under the root node (Root) of the View Tree is used to display images.

[0127] The System Server in this application includes a Config Manager, which is used to perform whitelist configuration (config) on whether the application supports dragging and dropping of text and image content, and on whether the application inherits the function of selecting the content area through the cursor in the operating system. The System Server can obtain the whitelist from the cloud. When the user selects text and image materials on the interface displayed by the application and long-presses or touches the text and image materials, it can be judged through the whitelist configuration whether the application supports dragging and dropping of text and image content, and on whether the application inherits the function of selecting the content area through the cursor in the operating system, so as to perform data collection in different situations according to different situations.

[0128] The System Server also constructs a drag component through the Warehouse Management Software (WMS) to detect the user's drag and drop. When the application where the text and image materials come from itself supports dragging and dropping of text and image content, the user drags the selected text and image mixed content and places the text and image mixed content at the collection space application. The drag framework (i.e., the drag component) collects the selected text and image mixed content in the collection space application.

[0129] The embodiments of the present application also add a view recognition module during the application process. The view recognition module is used to identify whether the current page includes a text view (Text View) and an image view (Image View); the view recognition module is also used to identify whether the application from which the graphic and text materials are sourced uses the native content selection implementation method of the operating system, that is, the view recognition module is also used to determine whether content extraction can be performed after identifying the Text View and the Image View. When the application from which the graphic and text materials are sourced does not support the drag-and-drop of graphic and text content and uses the native content selection implementation method of the operating system, the view recognition module obtains the selected area, and based on the local configuration policy (Local Strategies), traverses the text and images in the selected area based on the text View and the image View under the root node (Root) of the View tree in the application, and recognizes (Recognizer) the text and images. A communication connection is established between the view recognition module and the drag-and-drop framework through stubs. The view recognition module stores the recognition data (URIs of text and images) in the instance storage unit of the drag-and-drop framework.

[0130] The present application can add an interface to the global collection on the basis of the Android native application management service (Application Management Service, AMS) framework. Through this interface, the global collection can request the node information (Request Content Node) of the application in the application process. When the application process receives the request, it will copy the node of the application on the current page (copy Node), and pass the node information of the application on the current page to the global collection through the update token. The node information of the application will carry the instance (Activity) of the application on the current page. The instance (Activity) is composed of the views of the entire current page, including the Text View and the Image View. The global collection can traverse all the Views in the current entire page of the application from the instance, so as to perform content extraction (Content Extract Activity) through the View framework in the global collection. In this example, the global collection relies on the View traversal ability of the view framework to find the Text View and the Image View.

[0131] When the application from which the graphic and text materials are sourced does not support the drag-and-drop of graphic and text content and does not inherit the native content selection implementation method of the operating system, the View framework (i.e., the view component) in the global collection obtains the entire current page where the graphic and text materials are located from the application and traverses the entire current page View. The global collection provides an image extraction module (Picture ExtractEngine) and a text extraction module (Text Extract Client). The image extraction module extracts images from the entire current page to obtain all extractable images in the entire current page, temporarily stores the images, and obtains the URIs of the images. The images are extracted in the form of bitmaps (bitmap) and saved as images. If it is a web view, the images can be downloaded. The text extraction module includes a Software Development Kit (SDK). An Artificial Intelligence (AI) plugin (Plugin) can be installed through the SDK. The AIPlugin (i.e., the text recognition component) is used to recognize the text in the screenshot (including the text in the image). When the text extraction module performs text recognition, it can pass the screenshot of the entire current page to the AIPlugin in the form of a bitmap through an interface (Init). The AIPlugin includes a Vision Service, a Text Configuration, and an OCR recognition engine (OCRDetect Engine). The AIPlugin can select any one to perform text recognition on the text in the screenshot of the entire current page to obtain the recognition result. The recognition result includes the text and the position of the text in the entire current page. The recognition result can include the text in the image. The recognition result is returned to the global collection through an interface (on Result). The View framework in the global collection adds an additional View above the entire current page, extracts the images according to the URIs of the images, and draws the images and the text recognized by the AIPlugin on the additional View according to the position of the text in the entire current page and the position of the image in the entire current page. The additional View is the same as the content in the entire current page and is provided to the user for the user to select again. Then, the View framework in the global collection obtains the selected area, obtains the URIs of the images and the text in the selected area, and stores the recognition data (text and image URIs) in the instance storage unit of the drag-and-drop framework.

[0132] The global collection also provides a Collection Service and a SensorManage service. The Collection Service is used to collect and store materials, and the SensorManage service is used to call the sensors on the electronic device for image acquisition, etc.

[0133] As shown in Figure B of Figure 2 When the input manager includes a three-finger swipe-down detection module and stubs as shown in Figure B of Figure 2 . The three-finger swipe-down detection module is used to detect finger swipes and finger presses (Hw Fingers Snap shooter). When a three-finger press is detected, it indicates that the user intends to select the current page, and the global collection is launched through the Bind Service. When a three-finger press followed by a swipe-down is detected, the global collection responds to this operation, thereby invoking (on collectionInvoked) the global collection to obtain the entire current page from the application. The input manager and the global collection are communicatively connected through stubs, and the stubs are equivalent to an interface for the communication connection.

[0134] It should be noted that if the mixed text and image content is collected through a three-finger swipe-down operation, the URI of the image can be stored in the global collection. If the mixed text and image content is collected by dragging, the URI of the image can be stored in the memory of a temporary process.

[0135] The collection space application obtains the URIs of the text and images from the instance storage unit for collecting the mixed text and image content. It can be collected in a hashed form, that is, the text and images are formed into a set of materials for collection, which is convenient for users to use the collected images and text and improves the data usage efficiency. The collection space application can also collect the mixed text and image content in the form of discrete web pages, improving the visibility of data collection.

[0136] The application framework layer may also include a window manager, a content provider, a view system, a telephone manager, a resource manager, a notification manager, etc. The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data, and make this data accessible to application programs. This data may include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The view system includes visible controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build the display interface of an application program. The display interface can be composed of one or more views. For example, it includes a view for displaying a text message notification icon, a view for displaying text, and a view for displaying pictures. The telephone manager is used to provide the communication function of the terminal 100, such as the management of call status (including answering, hanging up, etc.). The resource manager provides various resources for application programs, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables application programs to display notification information in the status bar. It can be used to convey message notifications of the informing type. It can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform the completion of a download, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as the notification of a background-running application program. The notification manager can also be a notification that appears on the screen in the form of a dialog window, such as prompting text information in the status bar, emitting a prompt sound, vibrating the electronic device, flashing the indicator light, etc.

[0137] Android Runtime includes a core library and a virtual machine. Android Runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core library of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as the management of object life cycles, stack management, thread management, security and exception management, and garbage collection.

[0138] The system library can include multiple functional modules, such as: surface manager, Media Libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The Media Libraries support the playback and recording of multiple common audio and video formats, as well as static image files, etc. The Media Libraries can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is the drawing engine for 2D drawing.

[0139] The kernel layer is the layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers, sensor drivers, etc.

[0140] The following combines Figure 3 and Figure 4 to illustrate the collection space application provided by this application.

[0141] The collection space application is a management application (Application, APP), which has the function of collecting any type of material, that is, users can meet the need of collecting materials through the collection space application. Further, the collection space application can be built into a personal learning space, work space, etc. The data retention rule in the collection space application can be permanent retention.

[0142] The materials collected in the collection space application include various types of information such as text (for example, web links, plain text, etc.), graphics, images, videos, web pages, etc. Images can be static images (such as pictures), or dynamic images (such as animations or videos, etc.).

[0143] The data collected in the collection space in this application can be displayed in the floating state area. The floating state area is a floating window, and the floating window only occupies a small part of the entire screen of the electronic device. The floating window can float on the side above the page corresponding to the application without affecting the normal use of the application.

[0144] Users can enter the collection space in the following two ways. Users open the collection space APP, or the electronic device responds to the user's long-press operation or drag operation on the material, recognizes that the user has the intention to collect, and starts the collection space APP. The collection space APP is displayed in the floating state area, and the upper right corner of the floating state area can include three dots used to indicate the "more" control. As Figure 3As shown in Figure A in [the context], the user can click or touch the "More" control. In response to the user's operation, such as Figure 3 As shown in the display interface of Figure B in [the context], the electronic device can display options below the "More" control; the options include but are not limited to "Enter Full Screen", "Add to Desktop", "Settings", and "Close" options. That is, by clicking or touching the "More" control at the top in the floating state of the Favorite Space APP, the electronic device, in response to this click operation, displays more functions to the user.

[0145] During the use of the Favorite Space APP, the user can present in multiple forms to distinguish and indicate that the electronic device is in different processing stages or performing different processes. For example, the form of the Favorite Space can include a hidden state and a floating state, etc.; these are only for illustrative naming. For example, the hidden state can also be called the minimized state, and the floating state can also be called the distribution state. This application does not make any limitations in this regard.

[0146] Among them, the hidden state of the Favorite Space is used to indicate that the Favorite Space has been called in the foreground or is running in the background; the floating state is used to display the data collected in the Favorite Space in the floating area on the current display interface.

[0147] Optionally, the shape of the hidden state of the Favorite Space (that is, the hidden state area of the Favorite Space) can be a vertical strip. For example, when the Favorite Space is in the hidden state, the Favorite Space can be presented by being adsorbed on the inner side of the screen edge, as Figure 4 the icon a of the hidden state shown in Figure A in [the context]. At this time, when the icon a of the hidden state is at the inner side position of the screen edge, it will not affect the user's normal viewing of the displayed content. The icon a of the hidden state can be a vertical strip, and visually, it will not attract the user's attention and will not interfere with the user's normal use.

[0148] It should be noted that the position on the inner side of the screen edge described in the embodiments of this application can specifically be located on the upper side, lower side, left side, or right side of the inner edge. This application does not make any limitations in this regard.

[0149] It should be noted that the Favorite Space can be switched from the hidden state to the floating state, and vice versa, it can also be switched from the floating state to the hidden state. Optionally, this switch can be an animation effect such as scrolling or stretching. For example, the icon a of the hidden state of the Favorite Space can be switched to the floating state b with a gradually stretching animation effect from a vertical strip. This is only an example here, and this application does not make any limitations in this regard.

[0150] When the user needs to view the content displayed on the interface of the electronic device, the user can click or touch other areas of the interface except the floating area to switch the favorite space from the floating state to the hidden state, and display it at the inner side of the screen edge of the electronic device. When the user needs to view or use the data collected in the favorite space, the user can click or touch the icon of the hidden state of the favorite space to switch the favorite space from the hidden state to the floating state.

[0151] Optionally, each piece of data collected in the favorite space is displayed in the floating area in the form of a card. The floating state of the favorite space can be displayed in different areas of the screen in different styles in combination with the size and shape of the electronic device. For example, the shape of the floating state of the favorite space (i.e., the floating area of the favorite space) can specifically include two display styles. One is a long strip shape, such as the floating state b1 shown in Figure 4 ; the other is a horizontal strip shape, such as the floating state b2 shown in Figure 4 . The floating state b1 and the floating state b2 can be presented in the effect of adsorbing on the inner side of the display screen edge when displayed, so as to avoid interfering with the currently displayed content. The floating state b1 and the floating state b2 also include a display progress bar, and the display progress bar is used to indicate and adjust the display progress of the card.

[0152] It should be noted that during the use of the favorite space, although it will switch between multiple forms, the favorite space in each form is at the top layer (or called on top) of the multiple display layers included in the display interface, and is not covered and affected by the content displayed below.

[0153] The following describes the data collection method provided by the present application in combination with the favorite space, and illustrates the application scenarios shown in Figures 5 - 8 on the electronic device.

[0154] Figure 5 Figure A in shows the graphic and text materials in the current interface of the mobile phone. The graphic and text materials are arranged in a mixed manner of images and texts. The favorite space is displayed in the hidden state at the inner side of the screen edge. To facilitate the distinction between images and texts, Figure 5 in Figure A in the image is placed in a graphic and a background different from the text is set. The user selects the text and the image through the cursor (including the starting cursor and the ending cursor), and is filled in gray in Figure 5 Figure A in . As shown in Figure 5 Figure B in , when the user wants to collect this piece of graphic and text mixed content (i.e., the text and image selected by the cursor), a long press operation or a touch operation can be initiated. The mobile phone recognizes that the user has the intention to collect, stores the selected graphic and text mixed content in the instance storage unit in a rich text structure, and a reduced version of the graphic and text mixed content is displayed on the interface. The reduced version of the graphic and text mixed content can move in the direction of the user's finger drag. As shown in Figure 5As shown in Figure C, when the user drags the scaled-down text-and-graphic mixed content to the sidebar where the collection space is located, the collection space detects new data and displays a card for adding data. The collection space obtains the rich text structure from the instance storage unit and collects the text-and-graphic mixed content. As Figure 5 shown in Figure D, after the collection space successfully collects the text-and-graphic mixed content, a card indicating successful collection is displayed in the floating area, thus completing the collection of the text-and-graphic mixed content.

[0155] It should be noted that Figure 5 the above is described by taking the case where the user has already enabled the collection space as an example. It can be understood that the timing of enabling the collection space can also be triggered by Figure 5 the long-press operation or touch operation in Figure B, or can be triggered by Figure 5 the drag operation in Figure C. After the collection space is enabled, it is displayed in the sidebar of the interface in a hidden state and will not affect the user's normal viewing of the displayed content. The embodiments of the present application do not limit this. Starting the collection space application through different triggering timings improves the richness of application scenarios.

[0156] It can be understood that the above Figure 5 is only an exemplary description of an interface display. Optionally, Figure D may not be displayed to the user either. Figure 5

[0157] The present application can complete data collection in the following manner. Taking the View framework to represent the view component, the drag framework to represent the drag component, and the OCR detection engine (OCR Detect Engine) to represent the text recognition component as an example for illustration.

[0158] In one implementation, the application from which the text-and-graphic materials are sourced itself supports dragging of text-and-graphic content. In this case, the application responds to the user's long-press operation or touch operation, places the selected text-and-graphic mixed content in the instance storage unit in the style of a rich text structure (for example, HTML text), and starts the drag framework. The instance storage unit is the ClipData of the drag framework, and ClipData is a class used to transfer text and URI data between Views. When the user starts the drag operation, the drag framework responds to this drag operation and encapsulates the selected text-and-graphic mixed content into a scaled-down text-and-graphic mixed content. The scaled-down text-and-graphic mixed content moves in the interface along the dragging direction of the finger until it is moved to the collection space and the text-and-graphic mixed content is dropped. The collection space obtains the rich text structure from the ClipData and collects the text-and-graphic mixed content in the collection space. The interface display of this implementation can refer to the above Figure 5 description and will not be elaborated here.

[0159] ​In another implementation, the application where the graphic and text materials are sourced from does not support dragging of graphic and text content. However, this application is a native application of the electronic device, or this application inherits a class provided natively by the Android operating system that specifically provides a content area selection function. This class uses the native Android Clipboard Manager class to implement the cursor selection function. Usually, when selecting a content area in the application, there is a cursor, and the selected content can be obtained by pulling down the cursor. If the application inherits a class provided natively by the operating system that specifically provides a content area selection function and implements content selection based on this, or if the application directly uses a class provided natively by the operating system that specifically provides a content area selection function to implement content selection. In this case, after the user selects the content area, the operating system sets the selected content to the corresponding class, and then the URIs of the text and images are obtained or read from these classes, which is equivalent to being able to obtain the selected content.

[0160] In this case, the application inherits the function of selecting the content area through the cursor, and the operating system can use its native content selection implementation method to identify the selected area. When the user performs a long press or touch operation on the selected graphic and text mixed content on the page, the View framework recognizes the long press event or touch event, and according to the system native content selection implementation method inherited by the application, obtains the selected area corresponding to the selected graphic and text mixed content. Among them, the long press operation can refer to pressing on the View where the graphic and text mixed content is located for a preset time. The View framework respectively recognizes the URI of the image in the selected area and the text in the selected area. The URI of the image can be the download link of a picture or a video. The URI of the image and the text are encapsulated as a rich text structure (for example, HTML text) and placed in the instance storage unit, and the drag framework is started. Next, the user starts the drag operation, and the drag framework responds to this drag operation and drags the selected graphic and text mixed content to the favorite space. The favorite space obtains the rich text structure from the instance storage unit and collects the graphic and text mixed content in the favorite space. The interface display of this implementation method can refer to the description above Figure 5 and will not be elaborated here.

[0161] Among them, HTML text uses a markup language to describe the structure and content of a web page. HTML text is a form of rich text and can maintain the form of rich text. The style information in the HTML text describes how to render HTML elements. For example, color, font, size, layout, paragraph, line format, etc. When the browser parses the HTML text, the HTML elements are rendered into the web page that the user sees on the screen according to the style information of the HTML text.

[0162] In another implementation, the application where the graphic and text material source is located does not support dragging of graphic and text content, and the application does not inherit the class in the Android operating system that specifically provides the content area selection function. For example, the application is a third-party application, and the content selection method is a custom logic, that is, the application does not inherit the function of selecting the content area through the cursor. In this case, when the user performs a long-press operation or a touch operation on the selected graphic and text mixed content on the page, the View framework takes the entire current page where the graphic and text material is located as the target. On the one hand, the View framework traverses all the Views on the entire current page, obtains all the extractable images (pictures or videos) on the entire current page through the Views, and the positions of the images on the entire current page, and temporarily stores the images to obtain the URIs of the images. On the other hand, the View framework passes the bitmap corresponding to the screenshot of the entire current page to the OCR detection engine. The bitmap can also be called a raster image or a dot matrix image, which is an image represented by a pixel array or a dot matrix. The OCR detection engine recognizes the text in the screenshot of the entire current page, obtains the text and the position of the text on the entire current page, and returns the text and the position of the text on the page to the View framework, which can be returned in the format of paragraphs and lines. The View framework adds an additional View above the entire current page, extracts the images according to the URIs of the images, and draws the images and the text returned by the OCR detection engine on the additional View according to the position of the text on the entire current page and the position of the image on the entire current page. That is to say, the View framework redraws the entire current page as an additional View without changing the layout of the text and images on the entire current page, and provides it to the user for the user to select again. Since this View is drawn by the View framework itself, the View framework can obtain the selected area corresponding to the graphic and text mixed content and recognize the content in the selected area.

[0163] The user reselects the required graphic and text mixed content in the additional view and performs a long-press operation or a touch operation on the selected graphic and text mixed content. The View framework recognizes the long-press event or the touch event and obtains the selected area corresponding to the selected graphic and text mixed content. The View framework respectively recognizes the URI of the image in the selected area and the text in the selected area, constructs ClipData in the style of HTML text with the URI of the image and the text (retaining paragraph information), and starts the drag framework. Next, the user starts the drag operation, and the drag framework responds to the drag operation and drags the selected graphic and text mixed content to the collection space. The collection space obtains the HTML text from the ClipData and collects the graphic and text mixed content in the additional view in the collection space.

[0164] Exemplarily, as Figure 6 shownFigure 6 Figure A in [reference] shows an additional view that is generated by the View framework when the user performs a long - press operation or a touch operation on the text - and - image content selected in Figure A of [reference]. The text - and - image materials shown in this additional view are the content of the entire current page. When the user performs a selection operation on the text - and - image materials in the additional view again, as Figure 5 shown in Figure B of [reference], the user selects text and images in the additional view through the cursor (including the start cursor and the end cursor), which is shown filled in gray in Figure B of [reference]. As Figure 6 shown in Figure B of [reference], when the user wants to collect the text - and - image content that is selected again (i.e., the text and images selected by the cursor in the additional view), the user can perform a long - press operation or a touch operation. The mobile phone recognizes the user's intention to collect and stores the selected text - and - image content in the additional view in the instance storage unit in a rich - text structure, and a reduced - size text - and - image content is displayed on the interface. The reduced - size text - and - image content can move in the direction of the user's finger drag. As Figure 6 shown in Figure B of [reference], when the user drags the reduced - size text - and - image content to the sidebar where the collection space is located, the collection space detects the new data and displays a card for adding data. The collection space obtains the rich - text structure from the instance storage unit and collects the text - and - image content. Figure 6 shown in Figure C of [reference], when the user drags the reduced - size text - and - image content to the sidebar where the collection space is located, the collection space detects the new data and displays a card for adding data. The collection space obtains the rich - text structure from the instance storage unit and collects the text - and - image content. Figure 6 shown in Figure D of [reference], when the user drags the reduced - size text - and - image content to the sidebar where the collection space is located, the collection space detects the new data and displays a card for adding data. The collection space obtains the rich - text structure from the instance storage unit and collects the text - and - image content.

[0165] It can be understood that after Figure D of [reference], a card indicating successful collection as shown in Figure D of [reference] above can also be displayed to complete the collection of the text - and - image content. However, this application embodiment does not show it. Figure 6 After Figure D of [reference], a card indicating successful collection as shown in Figure D of [reference] above can also be displayed to complete the collection of the text - and - image content. However, this application embodiment does not show it. Figure 5 After Figure D of [reference], a card indicating successful collection as shown in Figure D of [reference] above can also be displayed to complete the collection of the text - and - image content. However, this application embodiment does not show it.

[0166] Figure 6 The additional view in Figure C of [reference] can no longer be displayed after the user selects the text - and - image content, or it can be always displayed for the user to collect again. The user can cancel the display of the additional view by clicking or pressing on the area other than the additional view, or the additional view includes a close button, and the user can cancel the display of the additional view by clicking or pressing the close button. This application embodiment does not limit this.

[0167] It should be noted that the text - and - image content selected by the user in the additional view can be different from the text - and - image content selected for the first time. Figure 5 Figure A in [reference] shows the text - and - image content selected by the user for the first time. Figure 6Figure B in the following shows the text and image mixed content re - selected by the user, and the two are not exactly the same. For some applications, after using them multiple times, the user knows that the application does not support direct dragging and needs to regenerate an additional view. When the user first selects data, they can perform a selection operation and a long - press operation on any data (which may not be the data they really want to collect), and wait for the regenerated additional view. Then, perform a selection operation on the text and image materials in the additional view to select the data they really want to collect. The user does not need to waste time in the first selection, which can improve the collection efficiency.

[0168] In another implementation, the user performs a three - finger swipe - down operation on the interface to take a screenshot, triggering the View framework to use the entire current page as the target and draw an additional View above the entire current page and provide it to the user for the user to select again. The user can collect the text and image materials on the additional view. The drawing process and the collection process can refer to the description above for the case where the application does not support dragging of text and image content and does not inherit the function of selecting the content area through the cursor, and will not be elaborated here.

[0169] Exemplarily, as Figure 7 shown, Figure 7 in the following, taking a three - finger swipe - down as an example to implement the screenshot of the entire current page. After taking the screenshot, the description of collecting the text and image mixed content above can be referred to, and will not be elaborated here. Figure 6 in the following

[0170] For the case where the image also includes text, such as Figure A in the above Figure 5 and Figure 7 . The View framework passes the bitmap corresponding to the screenshot of the entire current page to the OCR detection engine. The OCR detection engine recognizes the text in the screenshot of the entire current page, including the text in the image, to obtain the text (including the text in the image) and the position of the text in the entire current page (including the position of the text in the image), and returns the text and the position of the text in the entire current page to the View framework. In this way, the View framework extracts the image according to the URI of the image, and generates an additional view according to the image and the text (including the text in the image) returned by the OCR detection engine, according to the position of the text in the entire current page (including the position of the text in the image) and the position of the image in the entire current page. The additional view generated by the View framework includes text, images, and the text in the images.

[0171] Figure 6 In the additional view shown in Figure A in the following, it includes text, images, and the text in the images. When the user selects the text and image mixed content in the additional view, they can also select the text in the image. As Figure 8 shown, Figure 8 is inFigure 6 is selected in the additional view of Figure A in Figure 8 the selected mixed text and image content in Figure 6 Figure B in Figure 6 is selected in the additional view of Figure A in Figure 6 the mixed text and image content selected in Figure B in

[0172] Based on the above Figures 9 - 17 the display form of the collected data in the floating state area will be introduced.

[0173] The mixed text and image content collected in the collection space can be shown in the form of a card group. The collection space collects the image and text in a hash form, that is, the image and text are collected separately, and the image and text correspond to cards respectively, and are displayed in the floating state area in the form of a card group, indicating that the image and text come from the same collection, and the relevance between the image and text is retained. The collection space obtains the rich text structure from the instance storage unit. The rich text structure includes data and the data type of the data (for example, HTML tags), and the HTML tags reflect the positions of the image and text on the page. The collection space downloads or copies the image and text, and at the same time forms a set of materials with the image and text in their original relative positions on the page as the order, and collects them in a card group. The card group includes cards corresponding to the image and text respectively. The card group can display the application source identifier of the source of the graphic and text materials. The application source identifier can be the icon of the application or the application name, indicating which application the card group is from. Among them, the icon can be a general logo representing the application, and this application does not limit this. When collecting the mixed text and image content in the same application multiple times, the card group can be distinguished by using a numbered form or marking a timestamp.

[0174] Exemplarily, as Figure 9 shown in Figure A in Figure 9 Figure A in Figure 5 is a floating state area shown based on Figure D in the above Figure 5 the mixed text and image content collected in Figure D in Figure 9 Figure A in Figure 9 shows the card group in the floating state area. The card group displays the application name (Application a) and the number of cards in the card group (3 items). The card group can also be as

[0175] For cases where additional views need to be generated, when the user selects the text-wrapped content in the additional view, the text in the image can also be selected. The collected card group also includes cards corresponding to the text in the image, such as Figure 9 shown in Figure C of Figure 9 Figure C in Figure 8 shows the floating state area based on the above

[0176] It should be noted that the application source identifier can be set at any one of the upper left corner, lower left corner, upper right corner or lower right corner of the card group, and this application does not limit this.

[0177] The selected text-wrapped content includes at least an image and text. The card group corresponding to the text-wrapped content includes at least two cards, one card displays the text and the other card displays the image. The number of cards in the card group is related to the division rule of the text and image materials. For example, the division rule is that all the images in the text-wrapped content can be divided into one part, and all the text in the text-wrapped content can be divided into another part. In this way, the card group includes two cards, one card displays the text and the other card displays the image. Another example is that the division rule is based on the image as the dividing line. If the image is embedded in the text and divides the text into two paragraphs, then the text-wrapped content is divided into Paragraph 1, image, Paragraph 2, as shown in Figure A of Figure 9 and Figure B of Figure 9 above. If the image is at the beginning or end of the text-wrapped content, then the text-wrapped content is divided into image, text, or text, image. Another example is that the division rule is based on the line spacing between the paragraphs included in the text. If the text includes at least two paragraphs with a large line spacing (e.g., greater than the preset spacing), then the text in the text-wrapped content is divided into paragraphs.

[0178] When collecting the text-wrapped content, any combination of the above division rules can also be used. For example, the text-wrapped content includes paragraphs with a line spacing greater than the preset spacing, and the image is located between any two paragraphs. The paragraphs and images in the text-wrapped content respectively correspond to their own cards.

[0179] Generally speaking, the purpose of the user to collect text and image materials is to use these materials to edit documents or make files, etc. The hash collection method can facilitate the user to directly access the data. For example, accessing a paragraph of text or an image, improving the flexibility of collecting data. Compared with the path of reusing data from the web page, the data reuse path of this method is shorter, improving the data usage efficiency and the user experience.

[0180] It should be noted that Figure 9 One card group shown in Figure 9 is the card group for displaying the content of the mixed text and pictures in this collection. It can be understood that before the content of the mixed text and pictures in this collection, the content of the mixed text and pictures has been collected in the collection space, and the floating area can also display the previously collected card group. Of course, after the content of the mixed text and pictures in this collection, the content of the mixed text and pictures can continue to be collected in the collection space, and the floating area can also display the card groups collected later. Figure 9 It is only illustrated by taking one card group as an example, and does not constitute a limitation of this application.

[0181] As Figure 10 shown in Figure A in Figure 10 , the user can open the card group by clicking, pressing or touching the card group. The electronic device responds to the user's click, press or touch operation on the card group and displays as Figure 10 shown in Figure B in Figure 10 . The card group includes 3 cards. At the same time, in Figure 10 Figure B in Figure 10 , a back control for retreating to the upper level of the card group is displayed in the floating area. Among them, the upper level of the card group is the collection space APP. This is convenient for users to further view and reuse each card, improving the user experience.

[0182] The display area in the interface of the electronic device is limited. Therefore, the data displayed on the card is limited. One card in the floating area may display part of the content of a piece of mixed text and pictures. The user can enlarge the card by triggering the operation on the card to view the complete data of the mixed text and pictures. As Figure 10 shown in Figure C in Figure 10 , the user can click, press or touch any card to enlarge the card and view the complete data in the card. The electronic device responds to the user's click, press or touch operation on the card group and displays the complete data of the card as Figure 10 shown in Figure D in Figure 10 . The enlarged card may include: the complete data of the paragraph, the acquisition timestamp of the paragraph, and the total number of words in the paragraph, etc.

[0183] For the situation where the user has also collected the text in the image, the card group in the collection space also includes a card corresponding to the text in the image, as shown in Figure C above Figure 9 Based on the card group collected in Figure C above Figure 9 as Figure 11 shown in Figure A in Figure 11 , the user can open the card group by clicking, pressing or touching the card group. The electronic device responds to the user's click, press or touch operation on the card group and displays as Figure 11 shown in Figure B in Figure 11 . The card group includes 4 cards, and one of the cards corresponds to the text in the image.

[0184] For the case of collecting data in the form of a card deck, for the collected text or paragraph, the card in the card deck displays all or part of the text. For example, the card can display all the characters of the text, or the first few characters, keywords, middle few characters, or last few characters in the text, etc. As shown in Figure B of Figure 10 , the card displays part of the text: "Translation: The bright moonlight shines on the railing of the well, as if a layer of white has risen on the ground", and the full text is: "Translation: The bright moonlight shines on the railing of the well, as if a layer of white frost has risen on the ground. I can't help but look up and see the bright moon in the sky outside the window, and can't help but lower my head and ponder, thinking of my hometown in the distance." For the collected images, the cards in the card deck display the image itself, or a thumbnail or partial screenshot of the image. The partial screenshot can be a screenshot of the upper left corner, upper right corner, lower left corner, lower right corner, or middle part of the image. The partial characters and partial images displayed in the floating state area in this application are for illustration purposes and may not be all of the text or image.

[0185] To facilitate the distinction between the text and image mixed content in the collection space and the data displayed in the card (which may be partial data), here, the text and image mixed content in the collection space is called the editing data, and the editing data is the complete data.

[0186] Exemplarily, taking the example of a user reusing the text and image mixed content collected on a mobile phone, the user opens the editing software, and an editing cursor is displayed in the editing area on the interface, as shown in Figure A of Figure 12 . For the card corresponding to the text or paragraph, the user can drag the card to drag the complete data of the text or paragraph corresponding to the card in the collection space to the editing area, as shown in Figure B of Figure 12 . After dragging to the editing area, the user releases the card, and all of the text is displayed in the editing area, as shown in Figure C of Figure 12 , and the editing cursor moves to the last position of the text to provide a starting position for the next paste. Further, after all of the text is displayed in the editing area, the collection space is displayed in a hidden state, and the collection space switches from the floating state to the hidden state, as shown in Figure D of Figure 12 , facilitating the user to view the pasted content.

[0187] Of course, the user can also perform a selection operation on the card by touching the card, long pressing the card, etc., and then paste the text in the editing area. This application does not limit this.

[0188] For the collected images, the same pasting method as that for text can be adopted. The difference between the two is that, usually, images occupy more memory than text, and the images need to be stored in the electronic device in the form of distributed files. What the collection space collects is the image information of the images, and the image information may include the thumbnail or partial screenshot of the image, the storage location information of the image, the application source identifier of the image, the acquisition timestamp of the image, etc. When the user retrieves the image, the user selects the card corresponding to the image, and obtains the image through the storage location information of the image in the image information, so as to display the image in the editing area.

[0189] For the case of collecting data in the form of a card group, the user can also directly reuse the card group. Exemplarily, taking the user's reuse of the text and image mixed content on the mobile phone as an example, the user opens the editing software, and the editing cursor is displayed in the editing area on the interface, as Figure 13 shown in Figure A. The user can drag the card group to drag the complete data of the text or paragraph corresponding to the card group in the collection space and the image into the editing area, as Figure 13 shown in Figure B. After dragging to the editing area, the user releases the card group, and all the text and the image are displayed in the editing area in the arrangement order of the cards in the card group, as Figure 13 shown in Figure C, and the editing cursor moves to the last position to provide a starting position for the next paste. Further, after all the text and the image are displayed in the editing area, the collection space is displayed in a hidden state, and the collection space switches from the floating state to the hidden state, as Figure 13 shown in Figure D, which is convenient for the user to view the pasted content.

[0190] For the case of collecting data in the form of a card group, the user can also perform a merging operation on the card group, as Figure 14 shown in Figure A. In the floating state area of the collection space, card group 1, card group 2, card group 3, and card group 4 are displayed. The user selects any card group (for example, card group 1) to perform a merging operation. The user can drag card group 1 to drag the card group 1 (card group 1 includes 2 cards) along the dragging direction into card group 3 (card group 3 includes 4 cards), as Figure 14 shown in Figure B. To facilitate the distinction of the merged card groups, Figure 4 in Figure B, the merged card group is shown filled with gray. After dragging to the position corresponding to card group 3 (which can be a preset area around card group 3), the user releases card group 1, and card group 1 and card group 3 are merged into one card group, as Figure 14As shown in Figure C in []. In the floating state area, a new card group is displayed. The new card group includes Card Group 1 and Card Group 3. The new card group can also display the application name of the merged card group (for example, Card Group 3) or the application name of the merged card group (for example, Card Group 1). The new card group can also display the number of cards in the new card group (for example, 6 items). Of course, it can also be that the user edits a new name for the new card group, and the embodiments of this application do not limit this.

[0191] After the above Card Group 1 and Card Group 3 are merged, Card Group 1 is no longer displayed in the floating state area. Correspondingly, if the new card group and other card groups move up or down, the number of card groups currently displayed in the floating state area is reduced, so as to display other hidden card groups, improving the visibility of data collection. The above Figure 14 In Figure C in [], an example is given with card groups that are not hidden, which does not constitute a limitation to this application. For the display method of card groups in the floating state area after card groups are merged, the embodiments of this application do not make any limitations.

[0192] Since the display area in the interface of the electronic device is limited, therefore, in the interface of the electronic device, the collection space collects multiple text and picture mixed content, and a limited number of text and picture mixed content is displayed in the floating state area, that is, a limited number of card groups are displayed, and other card groups are hidden. For the hidden card groups, the card groups can be displayed alternately by dragging or scrolling the progress bar. Each text and picture mixed content is displayed as multiple objects (each object can be an image or a paragraph) in the form of a combination of multiple cards in the floating state area. Each card displays all or part of each object. The multiple cards maintain the corresponding order in the card group. The multiple cards can be displayed vertically, side by side, or stacked, etc., and the embodiments of this application do not make any limitations to this.

[0193] When multiple text and picture mixed content is displayed in the form of card groups in the floating state area of different electronic devices, they can be sorted according to the acquisition timestamp of this text and picture mixed content.

[0194] It should be noted that if the display effect of the card group appears from the bottom of the floating state area, the text and picture mixed content with the acquisition timestamp closest to the current moment is displayed in the form of a card group at the bottom. Correspondingly, other card groups located at the top are hidden. If the display effect of the card group appears from the right side of the floating state area, the text and picture mixed content with the acquisition timestamp closest to the current moment is displayed in the form of a card group on the rightmost side. Correspondingly, other card groups located on the leftmost side are hidden.

[0195] Due to the limited display area of the interface, the number of card groups displayed in the floating state area is limited. The upper limit of the number of card groups displayed in the floating state area can be appropriately set by those skilled in the art according to the size of the electronic device. When a new card group needs to be displayed, if the number of card groups already displayed in the floating state area reaches the upper limit, the card group that was displayed first needs to be hidden, and at the same time, the new card group is displayed. Since the display order of the card groups in the floating state area can be sorted according to the acquisition timestamp of the card groups, therefore, the card group with the earliest acquisition timestamp can be hidden here.

[0196] The form of hiding and displaying the card group can be that the collection space directly shows the interface to the user with the final new card group, without showing the process of hiding and displaying. It can also show the user the process of hiding and displaying. For example, the card group at the top of the floating state area gradually hides, and at the same time, the new card group gradually shows at the bottom of the floating state area until the card group at the top of the floating state area is completely hidden. At this time, the new card group is completely shown at the bottom of the floating state area. Of course, it can also be that when the upper boundary of the topmost card group reaches the upper edge of the floating state area, it directly completely hides, and the middle card groups and the new card group move upward at the same time, and the new card group gradually shows at the bottom of the floating state area until it is completely shown. Of course, it can also be that when the upper boundary of the topmost card group reaches the upper edge of the floating state area, it directly completely hides, and the middle card groups move upward at the same time until they reach the upper edge, and the new card group is directly completely shown at the bottom of the floating state area. The present application does not limit the form of hiding and displaying the card group.

[0197] When the number of card groups already displayed in the floating state area reaches the upper limit, the card group that was displayed first will also be hidden. There will be a display progress bar in the floating state area. This display progress bar is used to adjust the progress of the card groups displayed in the floating state area. The user can perform a drag operation or a scroll operation on this display progress bar to display other hidden card groups. As Figure 15 shown in Figure A of [reference], the mobile phone responds to the user's drag operation, determines the drag displacement according to the drag direction corresponding to the drag operation, and moves the card groups in the floating state area according to the drag displacement to realize the rotation display of the card groups. As Figure 15 shown in Figure B of [reference], the card group at the top of the floating state area gradually hides, and at the same time, the hidden card group gradually shows at the bottom of the floating state area. As the finger continues to drag the display progress bar, the card group at the top of the floating state area is completely hidden, and at the same time, the hidden card group is completely shown at the bottom of the floating state area, as Figure 15 shown in Figure C of [reference].

[0198] It should be noted that the card group can be hidden or displayed as a whole. For example, when the top card group reaches the upper edge of the suspended area, it skips gradual hiding and is completely hidden directly. At the same time, the middle card group moves upward until it moves to the upper edge, and the hidden card group is directly and completely displayed at the bottom of the suspended area. Of course, it is also possible that the top card group is completely hidden when it reaches the upper edge of the suspended area, and the middle card group and the hidden card group move upward at the same time, and the hidden card group is gradually displayed at the bottom of the suspended area until it moves to be fully displayed. This application does not limit the form of hiding and displaying the card group.

[0199] The mixed text and image content collected by the collection space can be displayed in the form of an offline web page file. The collection space obtains a rich text structure from the instance storage unit. The rich text structure includes data and data types of the data (for example, HTML tags). The HTML tags reflect the position of the image and text in the page. The collection space downloads or copies the image and text, and at the same time, the image and text are arranged in a standard HTML text style with their original relative positions in the page to generate an offline web page file. After the collection space obtains the rich text structure, certain data types in the HTML tags can also be changed. However, in the present application, the collection space retains the original tag style as much as possible, typesets the data according to the original information (the original relative position and data type of the data), and generates an offline web page file in the style of the HTML text in the web page.

[0200] For example, Figure 16 As shown, Figure 16 is in the above Figure 5 The suspension region shown in Figure D is based on the Figure 5 The mixed text and image content in Figure D comes from application a. Figure 16 A single card in the suspended area is shown, which displays the icon of application a and the web page title corresponding to the mixed text and image content. The icon of application a indicates that the offline web page file in the single card is from the collection of application a. Of course, this is only an example, and it can also be displayed in other forms, which is not limited by this application. The single card can also display the size of the offline web page file (for example, 544.18KB) and the generation timestamp of the offline web page file (for example, 2023.1.30).

[0201] It should be noted that Figure 16One single card shown in the figure is the single card for displaying the mixed text and image content in this collection. It can be understood that before this collection of the mixed text and image content, the mixed text and image content has been collected in the collection space, and the suspended area can also display the previously collected single card. Of course, after this collection of the mixed text and image content, the mixed text and image content can continue to be collected in the collection space, and the suspended area can also display the single cards collected later. Figure 16 It is only illustrated by taking one single card as an example, and does not constitute a limitation of this application.

[0202] After the single card is displayed in the suspended area, the user can view it through a browser, a file manager, or other applications that can browse HTML styles, such as Figure 17 as shown in Figure A in. The user can open the single card by clicking, pressing, or touching the single card. In response to the user's click, press, or touch operation on the single card, the electronic device displays an option for opening an offline web page file in the interface, such as Figure 17 as shown in Figure B in. The option includes but is not limited to "third-party application", "browser", and "more" controls. The third-party application in the option refers to an application that can browse HTML styles. The user can also click, press, or touch the "more" control to make the electronic device display more third-party applications that can open the offline web page file. As Figure 17 shown in Figure C in, the user clicks, presses, or touches the browser to select the browser as the way to open the offline web page file. Figure 17 As shown in Figure D in, the offline web page file is opened in the browser.

[0203] It should be noted that when the collected mixed text and image content is displayed in the form of a single card in the suspended area, the suspended area includes a display progress bar, which is used to adjust the progress of the single card displayed in the suspended area. The user can perform a drag operation or a scroll operation on the display progress bar to display other hidden single cards. Reference can be made to the above description of Figure 15 and will not be elaborated here.

[0204] The collection space collects the mixed text and image content in the form of an offline web page file, which is beneficial for the user to view the mixed text and image content of interest again and improves the data intuitiveness. Compared with the way of collecting the entire page, it can reduce the steps for the user to rummage through the entire web page to find the place of interest. The collected offline web page file only includes the mixed text and image content of interest and retains the original layout of the mixed text and image content, improving the user experience of viewing the collected data.

[0205] The collection space can adopt a suspended area as shown in Figures 9 - 15 to collect the mixed text and image content in a hash form, or can also adopt such asFigure 16 and Figure 17 For the floating state region shown, the text and image mixed content is collected in the form of an offline web page. The implementation form of the collection in the collection space can be appropriately set by those skilled in the art according to the actual situation, and the embodiments of this application do not limit this.

[0206] It can be understood that the collection space can also collect the text and image mixed content in the form of hash and offline web page respectively, that is, collect the text and image mixed content into a card group and a single card. In this way, the user can reuse the collected text and images through the card group, and can view the collected text and image mixed content through the single card, improving the diversity of the data collection form, meeting the personalized needs of the user, and improving the user experience.

[0207] Figure 18 It is an exemplary flowchart of a data collection method provided by an embodiment of this application.

[0208] In some possible cases, the exemplary process involved in the electronic device collecting text and image materials can refer to the following description of S101 - S104.

[0209] S101. Display a first interface, where the first interface includes a first material.

[0210] It can be understood that the first interface can be the interface showing text and images involved in the foregoing content.

[0211] In one possible implementation, the first interface can be the Figure 5 display interface shown in Figure A above. At this time, the first material includes text and images.

[0212] S102. In response to a selection operation on the first material, obtain first selected data, where the first selected data includes a first image and a first text in the first material.

[0213] In one possible implementation, the selection operation on the first material can be completed through the Figure 5 starting cursor and ending cursor in Figure A above, and the first selected data can be the Figure 5 image and text shown in gray filling in Figure A above.

[0214] S103. In response to a first operation on the first selected data, store the first selected data in the instance storage unit in a rich text structure, where the first operation refers to an operation with a collection intention.

[0215] In one possible implementation, the first operation can be a long - press operation or a touch operation.

[0216] S104. In response to a drag operation on the first selected data, collect the first selected data in the collection space according to the rich text structure.

[0217] In a possible implementation, the drag operation can be Figure 5 the drag operation in Figure C of

[0218] For the relevant content involved in S101 - S104, reference can be made to the above description of collecting the content with text and pictures mixed, which will not be elaborated here. The exemplary interfaces involved in S101 - S104 can be referred to the above description of Figures 5 - 17 the relevant content, which will not be elaborated here.

[0219] In this application, the user performs a selection operation on the text and picture materials displayed on the interface. The electronic device responds to the selection operation and obtains the selected data. The user performs a first operation on the selected data, indicating that the user may have the intention to collect the selected data. The electronic device responds to the first operation, retains the distribution positions and orders of the image and text on the page where the text and picture materials are located, and stores the selected data in the instance storage unit in the rich text structure. The user performs a drag operation on the selected data, triggering the collection space to obtain the rich text structure from the instance storage unit. The rich text structure includes the data and the data type of the data. By storing in the form of a packaged rich text structure, the key information of the text and picture mixed format can be retained to the greatest extent. With the help of the collection space for collection, the collected data or offline web page files include images, texts, the relevance between images and texts, and the relevance between texts, so as to realize the collection of the data in the text and picture mixed format, without the need for the user to collect the text and image separately, improving the collection efficiency.

[0220] Next, in combination with Figure 19 introduce the data collection process of the collection space of the electronic device.

[0221] Taking the View framework to represent the view component, the drag framework to represent the drag component, and the OCR detection engine (OCR DetectEngine) to represent the text recognition component as an example for illustration.

[0222] S11. The user performs a selection operation on the text and picture materials in the application on the interface displayed by the electronic device.

[0223] Taking the electronic device as a mobile phone as an example, as shown in Figure A of the above Figure 5 the user selects text and images in the text and picture materials in the application through the start cursor and the end cursor.

[0224] S12. In response to the long - press operation performed by the user on the selected text and picture mixed content, the application stores the text and picture mixed content in the ClipData in the style of HTML text.

[0225] The application can be a native application in an electronic device or a third-party application. Some applications have the function of supporting the drag and drop of text and image content (for example, the browser built into the electronic device, third-party browsers, etc.). In this case, when the user wants to collect this mixed text and image content, a long-press operation can be initiated. The application recognizes the user's intention to collect and stores the selected mixed text and image content in the form of HTML text in ClipData. At the same time, a reduced version of the mixed text and image content is displayed in the interface, as shown in Figure B in the above Figure 5 As shown in Figure 3, the user drags the reduced version of the mixed text and image content.

[0226] The reduced version of the mixed text and image content can move along with the direction of the user's finger drag.

[0227] S13. In response to the drag operation performed by the user on the mixed text and image content, the application starts the drag framework.

[0228] The user starts to drag and drop (Start Drag And Drop) the mixed text and image content. In response to the drag operation, the application starts the drag framework.

[0229] After S13, S18 - S19 can be executed.

[0230] S14. If the application itself does not support the drag and drop of text and image content and inherits the native content selection implementation method of the operating system, the application starts the View framework.

[0231] If the application inherits the function of selecting the content area through the cursor, the application can use the native content selection implementation method of the operating system to identify the selected area. When the user wants to collect this mixed text and image content, a long-press operation can be performed on this mixed text and image content to start the View framework. The View framework recognizes the long-press event and obtains the selected area corresponding to the selected mixed text and image content according to the native content selection implementation method inherited by the application.

[0232] S15. The View framework recognizes the long-press event and obtains the selected area corresponding to the mixed text and image content selected by the user according to the native content selection implementation method inherited in the application.

[0233] S16. The View framework respectively obtains the text and the URI of the image of the selected area from the View, encapsulates the obtained data as HTML text, and stores it in ClipData.

[0234] Among them, the image includes pictures and videos. The URI of the image represents the download link of the image, and the image can be obtained from a certain location on the network through the URI of the image. The HTML text is in a rich text structure, and ClipData is a storage container in the drag-and-drop framework, which can also be called an instance storage unit. The View framework recognizes the text and images in the selected area.

[0235] S17. In response to the drag-and-drop operation performed by the user on the content with text and images mixed, the View framework starts the drag-and-drop framework.

[0236] As shown in Figure 5 Figure C above, a reduced version of the content with text and images mixed is displayed in the interface, and the user drags the reduced version of the content with text and images mixed.

[0237] After S17, S30 - S31 can be executed.

[0238] S18. The user performs a screenshot operation on the interface.

[0239] As shown above Figure 7 The user can use a three-finger swipe-down operation to take a screenshot of the current interface. If the user directly performs a three-finger swipe-down operation on the interface, it triggers the View framework to use the entire page where the image material is located as the target. The text and image materials in the application are displayed in the interface. For the case where the user takes a screenshot of the interface, regardless of whether the application itself supports dragging of text and images and whether it inherits the content selection implementation method of the operating system native, the View framework uses the entire page where the image material is located in the interface as the target.

[0240] S19. In response to the user's screenshot operation, the application starts the View framework.

[0241] In the embodiments of the present application, a three-finger swipe-down operation can trigger the start of the View framework.

[0242] In this example, in response to the user's three-finger swipe-down operation, a screenshot is taken, thereby starting the View framework.

[0243] After S19, S21 - S31 can be executed.

[0244] S20. In response to the long-press operation performed by the user on the selected content with text and images mixed, if the application itself does not support dragging of text and images and does not inherit the content selection implementation method of the operating system native, the application starts the View framework.

[0245] After S11 above, if the application is a third-party application and its content selection method is a custom logic, that is, the application does not inherit the function of selecting the content area through the cursor, then the user initiates a long press operation on the selected text and image mixed content, and the View framework is started. The View framework takes the entire page of the page where the image material is located as the target.

[0246] After S20, S21 - S31 can be executed.

[0247] S21. The View framework traverses the entire current page View, obtains all the extractable images in the entire current page through the View and the positions of the images in the entire current page, temporarily stores the images, and obtains the URIs of the images.

[0248] Among them, the images include pictures and videos. The View framework recognizes the text and images in the entire current page.

[0249] S22. The View framework sends the bitmap corresponding to the screenshot of the entire current page to the OCR detection engine.

[0250] The View framework passes the bitmap corresponding to the screenshot of the entire current page to the OCR detection engine.

[0251] S23. The OCR detection engine recognizes the text in the screenshot of the entire current page, and obtains the text and the position of the text in the entire current page.

[0252] S24. The OCR detection engine sends the text and the position of the text in the entire page to the View framework.

[0253] The OCR detection engine recognizes the text in the screenshot of the entire current page, obtains the text and the position of the text in the entire current page, and returns the text and the position of the text in the entire page to the View framework, which can be returned in the format of paragraphs and lines.

[0254] S25. The View framework adds an additional view above the entire current page, and draws the images and text on the additional view according to the positions of the text and images in the entire current page.

[0255] As shown in Figure A above Figure 6 The View framework generates an additional view above the entire current page, and the content displayed in the additional view is the same as the page content of the page where the text and image materials are located.

[0256] Extract the images according to the URIs of the images, draw the images and the text returned by the OCR detection engine on the additional View according to the positions of the text in the entire current page and the positions of the images in the entire current page, and provide them to the user for the user to select again.

[0257] S26. The user re - executes the selection operation in the additional view.

[0258] As described above Figure 6 shown in Figure B above, the user re - selects the required text - and - image mixed content on the additional view shown in Figure A above. Figure 6 shown in Figure A above.

[0259] When the above - mentioned OCR detection engine recognizes a screenshot, the recognized text also includes the text in the image. Therefore, the generated additional view also includes the text in the image. Based on this, the text selected by the user can also include the text in the image. As Figure 8 shown, the text selected by the user can include the text in the image.

[0260] S27. In response to the long - press operation performed by the user on the re - selected text - and - image mixed content, the View framework recognizes the long - press event and obtains the selected area corresponding to the text - and - image mixed content selected by the user in the additional view.

[0261] As described above Figure 6 shown in Figure C above, when the user wants to collect the text - and - image mixed content selected in the additional view, a long - press operation can be initiated, and a reduced - size version of the text - and - image mixed content is displayed in the interface. The reduced - size version of the text - and - image mixed content can move in the direction of the user's finger drag.

[0262] Since the additional view is generated by the View framework, the View framework can obtain the selected area corresponding to the text - and - image mixed content selected by the user in the additional view.

[0263] S28. The View framework obtains the URIs of the text and images in the selected area, encapsulates the obtained data as HTML text, and stores it in ClipData.

[0264] The View framework respectively recognizes the URI of the image in the selected area (i.e., the download link of the image) and the text in the selected area, constructs ClipData in the style of HTML text for the text and the image, and starts the drag - and - drop framework. Among them, the text retains the paragraph information and the image places the URI.

[0265] S29. In response to the drag - and - drop operation performed by the user on the text - and - image mixed content in the additional view, the View framework starts the drag - and - drop framework.

[0266] As described above Figure 6 shown in Figure D above, the user drags the reduced - size version of the text - and - image mixed content.

[0267] The user starts to perform drag and drop on the content with text and pictures (Start Drag And Drop). In response to the drag operation, the electronic device starts a drag framework.

[0268] S30. In response to the user's operation of placing the content with text and pictures in the area corresponding to the collection space, the drag framework starts the collection space.

[0269] After the user drags the content with text and pictures to the hidden area corresponding to the collection space and drops it, it means that the user wants to collect the content with text and pictures. In this case, the drag framework starts the collection space.

[0270] The hidden area corresponding to the collection space is displayed on the side bar of the terminal. When collecting data, the collection space is adsorbed on the side bar in a hidden state, which does not affect the user's viewing and operation of the interface. Of course, the collection space is displayed in a floating state, and this application does not make any restrictions on this. As shown in Figure D above, when the user drags the scaled-down content with text and pictures to the hidden area corresponding to the collection space (which can be the side bar where the collection space is located), the collection space detects new data and displays a card for adding data. Figure 6 As shown in Figure D above, when the user drags the scaled-down content with text and pictures to the hidden area corresponding to the collection space (which can be the side bar where the collection space is located), the collection space detects new data and displays a card for adding data.

[0271] S31. The collection space obtains the HTML text from ClipData and collects the content with text and pictures.

[0272] As above Figure 5 As shown in Figure D above, the collection space collects the content with text and pictures.

[0273] The HTML text carries HTML tags, and the HTML tags reflect the positions of the images and text on the page. The collection space obtains the HTML text from ClipData and downloads or copies the images and text. In one way, the collection space collects the images and text in a scattered manner according to their original relative positions on the page, and displays them in the form of a card group as shown above in the floating area corresponding to the collection space. In another way, the collection space combines the images and text in their original relative positions on the page to form the style of a standard HTML text, generates an offline web page file, and displays it in the form of a single card as shown above in the floating area corresponding to the collection space. Figure 9 As above Figure 16 As shown in Figure D above, the collection space collects the content with text and pictures.

[0274] In this example, the collection of text and image mixed content through the collection space can be carried out in the form of an offline web page. The image (such as a picture or video) can be downloaded or copied according to the URI of the image. At the same time, the text and the image are combined into the style of a standard HTML text in their original relative positions, and the original tag style is retained as much as possible. Subsequently, the user can view it through a browser or other applications that can view the style of HTML text, which is convenient for the user to view the text and image mixed content again, improves the data intuitiveness, and enhances the user experience.

[0275] In this example, the collection of text and image mixed content through the collection space can be carried out in the form of a hash. The image (such as a picture or video) can be downloaded or copied according to the URI of the image. At the same time, the text and the image are taken as the order in their original relative positions, and multiple text and image contents form a set of materials. That is, the image and the text are collected separately, which is convenient for the user to use the collected image and text, improves the data usage efficiency, and enhances the user experience. Moreover, the collection space retains the relevance between the image and the text in the form of a card group, improving the visibility of data collection. Further, the card group can display the applications corresponding to the text and image materials, increasing the diversity and richness of the interface display.

[0276] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a Digital Versatile Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)), etc.

[0277] The above are the optional embodiments provided by the present application, which are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the technical scope disclosed in the present application shall be included within the protection scope of the present application.

Claims

1. A data collection method, characterized in that, Applied to an electronic device, the method includes: Display a first interface, where the first interface includes a first material; In response to a selection operation on the first material, obtain first selection data, where the first selection data includes a first image and a first text in the first material; In response to a first operation on the first selection data, store the first selection data in an instance storage unit in a rich text structure, where the first operation refers to an operation with a collection intention; In response to a drag - and - drop operation on the first selection data, when the first selection data is dragged and dropped into the area corresponding to the collection space in the first interface, collect the first selection data in the collection space according to the rich text structure.

2. The method according to claim 1, wherein The step of storing the first selection data in the instance storage unit in a rich text structure in response to the first operation on the first selection data includes: In response to the first operation on the first selection data, if the application to which the first material belongs supports dragging of graphic and text content, store the first selection data in the instance storage unit in the rich text structure.

3. The method according to claim 1, characterized in that The step of storing the first selection data in the instance storage unit in a rich text structure in response to the first operation on the first selection data includes: In response to the first operation on the first selection data, if the application to which the first material belongs does not support dragging of graphic and text content and the application inherits the function of selecting a content area through a cursor in the operating system of the electronic device, obtain the selected area corresponding to the first selection data; Identify the resource identifier of the first image and the first text in the selected area through a view component to obtain identification data; Store the identification data in the instance storage unit in a rich text structure.

4. The method according to claim 1, wherein The step of obtaining first selection data in response to a selection operation on the first material includes: In response to a selection operation on the first material, obtain second selection data, where the second selection data includes any data in the first material; In response to a second operation on the second selection data, if the application to which the first material belongs does not support dragging of graphic and text content and the application does not inherit the function of selecting a content area through a cursor in the operating system of the electronic device, generate an additional view above the page where the first material is located, where the additional view includes the page content of the page where the first material is located and supports selecting a content area through a cursor; In response to a selection operation on the data in the additional view, obtain the first selection data.

5. The method according to claim 1, wherein Before obtaining the first selection data in response to a selection operation on the first material, it further includes: In response to a third operation on the first interface, generate an additional view above the page where the first material is located, where the additional view includes the page content of the page where the first material is located and supports selecting a content area through a cursor; The step of obtaining first selection data in response to a selection operation on the first material includes: In response to a selection operation on the first material on the additional view, obtain the first selection data.

6. The method according to claim 4 or 5, characterized in that, In response to a first operation on the first selected data, storing the first selected data in a rich text structure in an instance storage unit includes: In response to the first operation on the first selected data, obtaining a selected area corresponding to the first selected data; Identifying a resource identifier of a first image and first text in the selected area through a view component to obtain identification data; Storing the identification data in the instance storage unit in a rich text structure.

7. The method according to any one of claims 4 to 6, characterized in that, Generating an additional view above the page where the first material is located includes: Obtaining, through a view component, a resource identifier of an image set in the page where the first material is located and positions of each image in the image set in the page; the image set includes the first image; Performing optical character recognition on a screenshot corresponding to the page through an optical character recognition component to obtain a text set and positions of each text in the text set in the page; the text set includes the first text; Generating the additional view above the page according to the image set extracted based on the resource identifier of the image set, the text set, and positions of each text in the text set in the page.

8. The method according to claim 7, wherein if the first image includes second text, the text set further includes the second text.

9. The method according to claim 8, wherein the additional view at least includes the first text, the first image, and the second text in the first image; The second selected data or the third selected data includes the first text, the first image, the second text, and / or other objects, and the other objects are objects in the page other than the first text, the first image, and the second text.

10. The method according to any one of claims 1-9, characterized in that, After collecting the first selected data in the collection space according to the rich text structure, further includes: Displaying a second interface; the second interface includes a suspended area corresponding to the collection space, and a first card group corresponding to the first selected data is displayed in the suspended area, and the first card group includes a first card corresponding to the first image and a second card corresponding to the first text.

11. The method according to claim 10, wherein the first card group further includes a first icon and / or a first name, and both the first icon and the first name indicate an application source identifier of the first selected data.

12. The method according to claim 10, wherein the first text includes at least one paragraph, and the first image includes at least one picture; a line spacing between any two paragraphs is greater than a preset spacing, or the first image is located between any two paragraphs; the first card group includes a card corresponding to each picture and a card corresponding to each paragraph; the cards corresponding to each picture and the cards corresponding to each paragraph are arranged in the order of positions of each picture and each paragraph in the page where the first material is located.

13. The method according to claim 10, wherein The first card includes the first image, or a thumbnail or partial screenshot corresponding to the first image; The second card includes all or part of the characters in the first text.

14. The method according to claim 10, wherein After displaying the second interface, it further includes: In response to a trigger operation on the first card group, the first card and the second card are displayed in the floating area.

15. The method according to claim 14, wherein, After displaying the first card and the second card in the floating area, it further includes: In response to a drag operation on a first target card, a third interface is displayed; the third interface includes a first editing area, and the editing data corresponding to the first target card is displayed in the first editing area, where the first target card is any card in the first card group.

16. The method according to claim 10, characterized in that, After displaying the second interface, it further includes: In response to a drag operation on a second card group, a fourth interface is displayed; the fourth interface includes a second editing area, and the editing data corresponding to each of the multiple cards included in the second card group is displayed in the second editing area, where the second card group is any card group in the floating area.

17. The method according to claim 10, wherein After displaying the second interface, it further includes: In response to a merge operation on a third card group and a fourth card group, a fifth card group is displayed in the floating area, and the fifth card group includes the cards included in the third card group and the cards included in the fourth card group, where the third card group and the fourth card group are any two card groups in the floating area.

18. The method according to any one of claims 1-9, characterized in that, After collecting the first selected data in the collection space according to the rich text structure, it further includes: Displaying a fifth interface; the fifth interface includes a floating area corresponding to the collection space, and a single card corresponding to the first selected data is displayed in the floating area. The single card includes an offline web page file corresponding to the first selected data, and the offline web page file is generated by the collection space according to the rich text structure.

19. The method according to claim 18, wherein After displaying the fifth interface, it further includes: In response to a trigger operation on a second target card, at least one option is displayed; the second target card is any single card in the floating area, and the option indicates a browsing engine or an application supporting web browsing; In response to a trigger operation on a target option, the web page corresponding to the second target card is displayed, where the target option is any option among the at least one option.

20. The method according to any one of claims 1-19, characterized in that, The method further includes: In response to a first operation on the first selected data, the collection space is opened, and the collection space is displayed in a hidden state in the first interface; Alternatively, in response to a drag operation on the first selected data, the collection space is opened, and the collection space is displayed in a hidden state in the first interface.

21. The method according to any one of claims 9-19, characterized in that, The upper limit of the display area elements in the floating area is a preset number; the area elements are card groups or single cards; The method further includes: If the number of area elements already displayed in the floating area reaches the preset number, the area element with the earliest acquisition timestamp among the already displayed area elements is hidden.

22. The method according to claim 21, wherein The suspended state area includes a display progress bar, and the display progress bar is used to adjust the progress of the area elements displayed in the suspended state area; The method further includes: In response to a fourth operation on the display progress bar, displaying the area elements hidden in the suspended state area.

23. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the method described in any one of claims 1-22 is implemented.

24. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium, and when it runs on a computer, the computer is caused to execute the method described in any one of claims 1-22.

25. A computer program product comprising instructions, characterized in that, When it runs on a computer, the computer is caused to execute the method described in any one of claims 1-22.

Images