Data collection method, device and storage medium

Abstract

The application discloses a data collection method and device and a storage medium, and relates to the technical field of terminals. According to the method, an electronic device displays a first interface, in response to a selection operation on a first material on the first interface, acquires first selected data, and the first selected data is in a picture-text mixed arrangement format. In response to a first operation on the first selected data, the distribution position and sequence of the first selected data in a page where the first material is located are retained, and the first selected data is stored in an instance storage unit in a rich text structure. A user performs a drag operation on the first selected data, and the first selected data moves along with the drag operation. When the first selected data moves to a region corresponding to a collection space, it is indicated that the user wants to collect the first selected data. The collection space acquires the rich text structure from the instance storage unit and collects the first selected data. In this way, the collection of data in the picture-text mixed arrangement format is realized, and the user does not need to collect text and images respectively, so that the collection efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a data collection method, device and storage medium. Background Technology

[0002] With the continuous development of terminal technology, users encounter a large amount of information in their daily lives. Some of this information is presented on the page in the form of mixed text and images (including images and text). When users collect data, they select a large area on the page, which includes both images and text.

[0003] However, no third-party applications currently support the collection and viewing of offline content in mixed text and image formats. Therefore, how to achieve the collection of mixed text and image data is a problem that urgently needs to be solved. Summary of the Invention

[0004] This application provides a data collection method, device, and storage medium, which can realize the collection of data with mixed text and image formats, thereby improving data collection efficiency. The technical solution is as follows:

[0005] Firstly, a data collection method is provided, comprising: an electronic device displaying a first interface; a user performing a selection operation on a first material on the first interface; the first material being rich text in a mixed image and text format, including both images and text; the electronic device responding to the selection operation by acquiring first selected data selected by the user on the first material, the first selected data including a first image and first text; the user performing a first operation on the first selected data, the first operation being a long press or a touch operation, indicating that the user may intend to collect the first selected data; the electronic device responding to the first operation by retaining the distribution position and order of the first image and first text on the page where the first material is located, and storing them in an instance storage unit in a rich text structure; the first selected data can be moved following the user's drag operation, and when it moves to the area corresponding to the collection space (which can be a hidden area or a floating area, the area of ​​the hidden area being smaller than the area of ​​the floating area), it indicates that the user wants to collect the first selected data, the collection space retrieves the rich text structure from the instance storage unit, and collects the first selected data according to the rich text structure.

[0006] In some embodiments of this application, the data collection method can be applied to electronic devices, which can be devices with displays such as mobile phones, tablets, desktop computers, portable computers, laptops, PDAs, e-book readers, televisions, smart wearable devices, web servers, wireless terminal devices, communication devices, or embedded devices.

[0007] In some embodiments of this application, the material may include data of types such as text and images; wherein, the images may be static images or dynamic images, and this application does not limit them.

[0008] In some embodiments of this application, the collection space is a management application (APP) with the function of collecting materials of any type. It can be regarded as a storage container for collecting rich text structures. When a user performs a first operation on the first selected data, it indicates that the user may intend to collect the first selected data, which can trigger the launch of the collection space application. When a user performs a drag-and-drop operation on the first selected data, it indicates that the user wants to collect the first selected data, which can trigger the launch of the collection space application. Of course, the user can also open the collection space application before collecting materials, and this embodiment of the application does not limit this.

[0009] In some embodiments of this application, the instance storage unit is used to store one or more instances, which can be different types of data, such as text, images, and Uniform Resource Identifiers (URIs). Rich text structure is a way of describing and organizing text content using Hyper Text Markup Language (HTML) tags or other markup language tags. Using HTML tags preserves the original format, structure, and semantics of rich text. Each HTML tag has its specific meaning and purpose, and together they constitute the structure and content of a webpage. By using HTML tags, it is ensured that the browser correctly parses and renders the webpage content. In the instance storage unit, various tags can be used to add different elements, such as headings, paragraphs, links, and images. These elements are rich text content and can be stored as strings or other data types. In this way, the instance storage unit can instantiate data into a rich text structure for storage, and use HTML tags to reflect the position of images and text on the page, facilitating subsequent processing, display, and manipulation of the rich text structure.

[0010] In some embodiments of this application, the user performs a drag-and-drop operation on the first selected data. When the data is moved to the area corresponding to the collection space, the collection space is triggered to retrieve a rich text structure from the instance storage unit. The rich text structure includes data and the data's data type (e.g., HTML tags). The collection space can perform hash-based collection of data according to the data type, that is, text and images are collected separately. The collection space can also generate an offline webpage file based on the data type and the data itself. The offline webpage file retains the distribution position and order of the first image and the first text on the page where the first material is located, and then the offline webpage file is collected.

[0011] In some embodiments of this application, by encapsulating the first selected data into a rich text structure for storage, the key information of the mixed text and image format can be preserved to the greatest extent. Furthermore, by utilizing a collection space for collection, the collected discrete data or offline web page files include images, text, the relationships between images and text, and the relationships between texts, thereby enabling the collection of mixed text and image format data without requiring users to collect text and images separately, thus improving collection efficiency.

[0012] In one possible implementation, in response to a first operation on the first selected data, the first selected data is stored in an instance storage unit in a rich text structure. The method includes: a user performing a first operation on the first selected data; an electronic device responding to the first operation; and if the application to which the first material belongs supports drag-and-drop of text and image content, then storing the first selected data in an instance storage unit in a rich text structure.

[0013] In some embodiments of this application, the user selects materials within an application. The application can be a native application built into the electronic device (e.g., a phone app, SMS app, browser, file manager, etc.) or a third-party application (e.g., a third-party browser). Some applications support drag-and-drop functionality for text and images (e.g., the browser built into the electronic device, third-party browsers, etc.), while others do not. If the application to which the first material belongs supports drag-and-drop functionality for text and images, the first selected data is stored in the instance storage unit as a rich text structure. Then, when the user wants to save the first selected data, the save space retrieves the rich text structure from the instance storage unit to save the data, eliminating the need for the user to drag and drop text and images separately, thus improving save efficiency.

[0014] In one possible implementation, in response to a first operation on the first selected data, the first selected data is stored in an instance storage unit in a rich text structure. The method includes: a user performing a first operation on the first selected data; the electronic device responding to the first operation; if the application to which the first material belongs does not support dragging and dropping of text and image content, and if the application inherits the function of selecting a content area by cursor from the operating system of the electronic device, meaning that the user can select the desired text and image content area by moving the cursor, then the view component in the electronic device obtains the selected area corresponding to the first selected data; identifies the resource identifier of the first image and the first text in the selected area to obtain identification data; and stores the identification data in an instance storage unit in a rich text structure.

[0015] In some embodiments of this application, the resource identifier of an image can refer to a Uniform Resource Identifier (URI), which is a string used to identify and locate image resources. The URI specifies the location and access method of the image. An image URI can represent a download link for the image, allowing the image to be retrieved from a specific location on the network. The view component recognizes the text in the selected area to obtain the text, and the view component recognizes the image in the selected area to obtain the image URI. The recognized data includes the URIs of both text and images, and is stored in the instance storage unit in a rich text structure. Therefore, when a user wants to save the first selected data, the save space retrieves the rich text structure from the instance storage unit for data saving, eliminating the need for the user to drag and drop text and images separately, thus improving saving efficiency.

[0016] In some embodiments of this application, for applications that do not support dragging and dropping of text and images, if the application inherits the function of selecting a content area with a cursor from the operating system of an electronic device, the selected area is identified through a view component, and then the identified data is stored in the instance storage unit in a rich text structure. In this way, applications that do not support dragging and dropping of text and images can still be used for collecting text and images, thus increasing the richness of application scenarios.

[0017] In one possible implementation, in response to a selection operation on the first material, first selected data is acquired. The method includes: a user performing a first operation on the first selected data; the electronic device responding to the first operation acquiring second selected data, the second selected data including any data from the first material; the user performing a second operation on the second selected data, the second operation being a long press or a touch operation, indicating that the user may intend to save the second selected data. If the application containing the first material does not support dragging and dropping text and images, and the application does not inherit the functionality of selecting content areas with a cursor from the electronic device's operating system, the view component acquires the page content of the page containing the first material and generates an additional view above the page. The additional view includes the page content and is generated by the view component, supporting the functionality of selecting content areas with a cursor. The user performs a selection operation on data in the additional view, and the electronic device responding to the selection operation acquires the first selected data.

[0018] In some embodiments of this application, the user performs a first operation on the first selected data. The first operation can be a long press or a touch operation, indicating that the user may intend to save the first selected data. If the application to which the first material belongs does not support dragging and dropping of text and images, and the application does not inherit the function of selecting content areas by cursor in the operating system of the electronic device, then saving the text and images cannot be achieved directly. Therefore, the view component needs to regenerate an additional view, which is placed at the top of the page and is not obscured by the page, so that the user can re-perform the selection operation on the additional view.

[0019] In some embodiments of this application, for applications that do not support dragging and dropping text and images and do not inherit the function of selecting content areas by cursor in the operating system of electronic devices, the entire page containing the first material is used as the target. An additional view is generated above the page based on the page content using a view component. This additional view re-reflects the page content, allowing the user to re-perform the selection operation. In this way, applications that do not inherit the function of selecting content areas by cursor in the operating system of electronic devices can also be applied to the method of collecting text and images, increasing the richness of application scenarios.

[0020] In one possible implementation, the user performs a third operation on the first interface, which could be a screenshot. In response to this third operation, the electronic device's view component acquires the entire page content and generates an additional view above the page containing the first material. This additional view includes the page content of the first material and is generated by the view component, allowing selection of content areas using a cursor. The data in the additional view is identical to the first material. When the user selects the first material in the additional view, the electronic device responds by acquiring the first selected data.

[0021] In some embodiments of this application, the user performs a third operation on the interface, which can be a screenshot operation. The screenshot operation is a three-finger swipe down operation provided in this application embodiment, a convenient operation for taking a screenshot of the current interface. The three-finger swipe down operation can capture a specific area or the entire screen of the interface, save it as an image, and trigger the step of saving it to favorites. In this scenario, electronic devices cannot directly save text and image content on the page; therefore, the view component needs to regenerate an additional view. This additional view is placed at the top of the page and is not obscured by the page, allowing the user to re-perform the selection operation on the additional view.

[0022] In some embodiments of this application, for scenarios where a screenshot is taken by swiping down with three fingers, the three-finger swipe operation can trigger a view component to take the entire page containing the first material as the target and generate an additional view above the page based on the page content. The additional view re-reflects the page content, allowing the user to re-perform the selection operation. This application provides various methods for collecting text and image content, enhancing the richness of application scenarios.

[0023] In one possible implementation, in response to a first operation on the first selected data, storing the first selected data in an instance storage unit in a rich text structure includes: a user performing a first operation on the first selected data; an electronic device responding to the first operation; a view component in the electronic device acquiring the selected area corresponding to the first selected data; identifying the resource identifier of a first image and the first text in the selected area to obtain identification data; and storing the identification data in an instance storage unit in a rich text structure.

[0024] In some embodiments of this application, for scenarios where the application itself does not support dragging and dropping of text and images, and does not inherit the function of selecting content areas with a cursor from the operating system of the electronic device, as well as scenarios where screenshots are taken by swiping down with three fingers, the additional view is generated by a view component and has the function of selecting content areas with a cursor. The selected area is identified by the view component, and the rich text structure is stored in the instance storage unit. In this way, applications that do not support dragging and dropping of text and images and scenarios where screenshots are taken by swiping down with three fingers can also be applied to the method of collecting text and images, improving the richness of application scenarios.

[0025] In one possible implementation, generating an additional view above the page containing the first material includes: obtaining the resource identifier of the image set in the page containing the first material and the position of each image in the image set on the page through a view component; the image set includes the first image; performing text recognition on the screenshot corresponding to the page through a text recognition component to obtain a text set and the position of each text in the text set on the page; the text set includes the first text; and generating an additional view above the page through the view component based on the image set resource identifier, the text set, and the position of each text in the text set.

[0026] In some embodiments of this application, the page containing the first material includes a first image and first text. The page may also include images other than the first image and text other than the first text; that is, the page includes at least one image and at least one text. Taking the entire page containing the first material as the target, the view component identifies the entire page to obtain the resource identifiers of each image in the image set and the position of each image on the page. The view component sends a screenshot of the page to the text recognition component, which performs text recognition on the screenshot to obtain each text in the text set and its position on the page. The text recognition component returns the identified text and its position on the page to the view component. Then, the view component obtains each image based on its resource identifier and generates an additional view above the page based on each image, its position on the page, and the text and its position on the page.

[0027] The text recognition component can be implemented using Optical Character Recognition (OCR) technology.

[0028] In some embodiments of this application, a view component identifies the resource identifiers of each image on the page and the position of each image on the page; a text recognition component performs text recognition on the corresponding screenshot of the page to obtain each text and its position on the page. Then, an additional view is generated at the top of the page based on the results of image recognition and text recognition. The additional view re-reflects the page content, allowing the user to re-perform the selection operation. In this way, applications that do not support dragging and dropping of text and images, and scenarios where screenshots are taken by swiping down with three fingers, can also be applied to the method of collecting text and images, improving the richness of application scenarios.

[0029] In one possible implementation, if the first image includes the second text, then the text set also includes the second text.

[0030] In some embodiments of this application, if the image includes text, when the text recognition component performs text recognition on the screenshot, it also recognizes the text in the image and returns it to the view component. The additional view generated by the view component at the top of the page also includes the text in the image, increasing the richness of the content displayed in the additional view.

[0031] In one possible implementation, the additional view includes at least first text, first image, and second text in the first image; the second selected data or third selected data includes the first text, first image, second text, and / or other objects, which are objects on the page other than the first text, first image, and second text.

[0032] In some embodiments of this application, the additional view generated above the page by the view component also includes text in an image. When a user performs a selection operation on the additional view, they can select the same data as the first selected data, i.e., the first text and the first image. The selected data can also include text in an image, i.e., the first text, the first image, and the second text, or other objects, i.e., the first text, the first image, and other objects. The selected data can also include text in an image and other objects. The selected data can also be any other text and image. This application does not limit this aspect.

[0033] In some embodiments of this application, users can also select text in an image and save the text in the image, which increases the richness of data collection and improves the user experience.

[0034] In one possible implementation, after the first selected data is stored in the collection space according to the rich text structure, the method further includes: the electronic device displays a second interface; the second interface includes a floating area corresponding to the collection space, and the floating area displays a first card group corresponding to the first selected data, the first card group including a first card corresponding to the first image and a second card corresponding to the first text.

[0035] In some embodiments of this application, the collection space is a storage container for collecting rich text structures. The floating area corresponding to the collection space is displayed on the interface and serves as a convenient entry point for users to view and use the collected data.

[0036] The collection space retrieves rich text structures from instance storage units. These structures include data and its data type. The collection space can then collect data in a hashed manner based on its data type; that is, text and images are collected separately. The first selected data includes the first text and the second image. These two cards are displayed as a card group in a floating area, indicating that the image and text originated from the same collection, thus preserving the association between them.

[0037] In some embodiments of this application, when users save text and image content, they typically intend to reuse the text or images. The save space uses a hash-based save structure based on rich text, meaning images and text are saved separately. This facilitates user access to the saved images and text, improves data utilization efficiency, and enhances the user experience. Furthermore, the save space preserves the correlation between images and text through card groups, improving the visibility of the saved data.

[0038] In one possible implementation, the first card group also includes a first icon and / or a first name, both of which indicate the application source identifier of the first selected data.

[0039] In some embodiments of this application, displaying the application's icon or name on the card prompts the user about the application's source for each card group, increasing the diversity and richness of the interface display and enhancing the fun of human-computer interaction.

[0040] In one possible implementation, the first text includes at least one paragraph, and the first image includes at least one picture; the line spacing between any two paragraphs is greater than a preset spacing, or the first image is located between any two paragraphs; the first card group includes a card corresponding to each picture and a card corresponding to each paragraph; the cards corresponding to each picture and the cards corresponding to each paragraph are arranged in the order of their positions on the page where the first material is located.

[0041] In some embodiments of this application, the first text may include one or more paragraphs, and the first image may include one or more pictures. The above display method involves displaying all text through one card and all images through another card; that is, images and text are displayed separately. In this example, if the line spacing between any two paragraphs is greater than a preset spacing, the two paragraphs are displayed through two separate cards. If any two paragraphs are separated by an image, the two paragraphs are displayed through two separate cards. Pictures within the image are displayed through separate cards. The above display method can be appropriately selected by those skilled in the art based on the actual situation.

[0042] The first card set includes a card for each image and a card for each paragraph. The cards can be sorted according to the position of each image and each paragraph on the page where the first material is located, preserving the relationship between images and text, between images, and between texts, thus improving the visibility of the data collection.

[0043] In one possible implementation, the first card includes a first image, or a thumbnail or partial screenshot of the first image; the second card includes all or part of the characters in the first text.

[0044] In some embodiments of this application, due to the limited display area of ​​the interface, the data displayed on the card in the floating area can be all or part of the complete data. For shorter text, for example, if the number of characters corresponding to the text is less than or equal to the maximum number of characters displayed on the card, the card displayed in the floating area includes all the characters of the text. For longer text, for example, if the number of characters corresponding to the text is greater than the maximum number of characters displayed on the card, the card displayed in the floating area includes part of the characters of the text. For smaller images, for example, if the size of the image is less than or equal to the maximum size of the card, the card displayed in the floating area includes the image. For larger text, for example, if the size of the image is greater than the maximum size of the card, the card displayed in the floating area includes a thumbnail or a partial screenshot of the image.

[0045] Display as much relevant data as possible within a limited area to help users understand the general information of each card and improve the user experience.

[0046] In one possible implementation, after displaying the second interface, the method further includes: the user performing a trigger operation on the first card group, and the electronic device responding to the trigger operation by displaying the first card and the second card in the floating area.

[0047] In some embodiments of this application, the triggering operation can be a click operation, a touch operation, or a press operation. The user can open the first card group and display the first and second cards within the first card group through the triggering operation. This facilitates further viewing and reuse of each card, improving the user experience.

[0048] In one possible implementation, after displaying the first card and the second card in the floating area, the method further includes: the user performing a drag operation on any card in the first card group, and the electronic device responding to the drag operation by displaying a third interface; the third interface includes a first editing area, in which the editing data corresponding to the card is displayed.

[0049] In some embodiments of this application, the drag operation can be a touch action or gesture such as dragging, pulling, dragging, pulling, or traction.

[0050] In some embodiments of this application, the data collected in the collection space is complete data. Due to the limited display area of ​​the interface, the data displayed on the cards in the floating area can be all or part of the complete data. When a card is selected, the complete data in the collection space is invoked, thereby displaying the complete data in the editing area; the edited data represents the complete data. Users can reuse collected data through drag-and-drop operations. Compared to reusing data from a webpage, this method has a shorter data reuse path, improving the efficiency of using collected data. Moreover, users can continue to perform operations such as deletion and modification on the edited data, increasing the flexibility of data reuse.

[0051] In one possible implementation, after displaying the second interface, the method further includes: the user performing a drag operation on any card group, and the electronic device responding to the drag operation by displaying a fourth interface; the fourth interface includes a second editing area, which displays the editing data corresponding to the multiple cards included in the card group.

[0052] In some embodiments of this application, users can reuse data from any card or from any card group. When reusing data from a card group, the edit data of the card group is displayed in the editing area by calling the complete data of each card in the card group from the collection space. The edit data of the card group includes the complete data corresponding to each card.

[0053] By reusing data in card groups, the number of drags and drop operations can be reduced, data reuse efficiency can be improved, and the user experience can be enhanced.

[0054] In one possible implementation, after displaying the second interface, the method further includes: the user performing a merge operation toward another card group on either card group, and the electronic device responding to the merge operation by displaying the merged card group in a floating area, the merged card group including the cards contained in both card groups.

[0055] In some embodiments of this application, users can merge any two card groups to reduce the number of card groups currently displayed in the floating area, thereby displaying other card groups in the floating area and improving the visibility of data collection.

[0056] In one possible implementation, after the first selected data is stored in the collection space according to the rich text structure, the method further includes: the electronic device displays a fifth interface; the fifth interface includes a floating area corresponding to the collection space, and the floating area displays a single card corresponding to the first selected data, the single card including an offline web page file corresponding to the first selected data, the offline web page file being generated by the collection space according to the rich text structure.

[0057] In some embodiments of this application, the collection space obtains a rich text structure from the instance storage unit. The rich text structure includes data and the data's data type. Based on the data type and the data, the collection space can generate an offline webpage file. The offline webpage file retains the distribution position and order of the first image and the first text on the page where the first material is located, and the offline webpage file is collected. The offline webpage file is then displayed as a single card in a floating area, preserving the original format of the first selected data on the page.

[0058] In some embodiments of this application, when users save mixed text and image content, they may want to view it again in certain scenarios. The save space saves data in an offline webpage format based on a rich text structure; that is, the images and text retain their original format on the page. This makes it easy for users to view the mixed text and image content again, improves data intuitiveness, and enhances the user experience.

[0059] In one possible implementation, after displaying the fifth interface, the method further includes: the user performing a trigger operation on any single card in the floating area, and the electronic device responding to the trigger operation by displaying at least one option; the option indicates a browsing engine or an application that supports web browsing; the user performing a trigger operation on any option, and the electronic device responding to the trigger operation by displaying the webpage corresponding to the single card.

[0060] In some embodiments of this application, the triggering operation can be a click operation, a touch operation, or a press operation. In response to the user's triggering operation on any single card, the electronic device displays one or more options. These options can be a browsing engine or an application that supports web browsing. The user selects an option, and the webpage corresponding to that single card is opened through the browsing engine or application. The user can then view the mixed text and image content on the webpage again, improving the reproduction efficiency of the mixed text and image content.

[0061] In one possible implementation, the method further includes: the user performing a first operation on the first selected data, and the electronic device responding to the first operation by opening the favorites space and displaying the favorites space in a hidden state on the first interface; or, the user performing a drag operation on the first selected data, and the electronic device responding to the drag operation by opening the favorites space and displaying the favorites space in a hidden state on the first interface.

[0062] In some embodiments of this application, when a user performs a first operation on the first selected data, it indicates that the user may intend to save the first selected data, which can trigger the launch of the Favorites Space application. When a user performs a drag-and-drop operation on the first selected data, it indicates that the user wants to save the first selected data, which can trigger the launch of the Favorites Space application. Displaying the Favorites Space in a hidden state on the first interface will not affect the user's normal viewing of the displayed content. Launching the Favorites Space application at different triggering times improves the richness of application scenarios.

[0063] In one possible implementation, the upper limit of the number of area elements displayed in the floating area is a preset number; the area elements are card groups or single cards; the method also includes: if the number of area elements already displayed in the floating area reaches the preset number, then the area element with the earliest timestamp among the displayed area elements is hidden.

[0064] In some embodiments of this application, due to the limited display area of ​​the interface, the number of elements displayed in the floating area is limited. The preset number can be a positive integer, such as 3, 4, 5, 6, etc. When a new element needs to be displayed, if the number of elements already displayed in the floating area reaches the upper limit, the first element displayed needs to be hidden, and the new element needs to be displayed.

[0065] The display order of area elements in a floating area can be sorted according to their acquisition timestamps. Therefore, the currently displayed area element with the earliest acquisition timestamp can be hidden. The acquisition timestamp of an area element refers to the timestamp when the first selected data corresponding to the area element was acquired on the electronic device.

[0066] By displaying new area elements and hiding the area element with the earliest timestamp, the latest saved image and text content is shown to the user, improving the user experience.

[0067] While displaying new area elements, the area element with the earliest timestamp can also be hidden. These two actions can be synchronized, allowing one area element to gradually hide while the other gradually appears. Alternatively, they can occur sequentially, with one area element completely hidden before the other appears.

[0068] In one possible implementation, the floating region includes a display progress bar for adjusting the progress of the region elements displayed in the floating region; the method further includes: displaying hidden region elements in the floating region in response to a fourth operation on the display progress bar.

[0069] In some embodiments of this application, when the number of displayed area elements in the floating area reaches its upper limit, the first displayed area element is also hidden. A display progress bar is also present in the first floating area to adjust the display progress of the area elements. Users can drag or scroll the display progress bar to reveal other hidden area elements. This facilitates viewing all area elements and improves the user experience.

[0070] In a second aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the methods as described in the first aspect or any implementation thereof.

[0071] Thirdly, a computer-readable storage medium is provided, which stores instructions that, when executed on a computer, cause the computer to perform the methods as described in the first aspect or any implementation thereof.

[0072] Fourthly, embodiments of this application provide a chip system that can be applied to an electronic device. The chip system includes one or more processors that invoke computer instructions to cause the electronic device to perform the method described in the first aspect or any implementation thereof.

[0073] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to perform the data collection method described in the first aspect.

[0074] The technical effects achieved by the second, third, fourth, and fifth aspects mentioned above are similar to those achieved by the corresponding technical means in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application;

[0076] Figure 2 This is a block diagram of a terminal software system provided in an embodiment of this application;

[0077] Figure 3 A schematic diagram of the collection space provided for this application;

[0078] Figure 4 The status and display style of the collection space provided for this application;

[0079] Figure 5 This application provides a schematic diagram for collecting mixed text and graphics content on an interface;

[0080] Figure 6 This application provides a schematic diagram for collecting mixed text and graphics content in an additional view;

[0081] Figure 7 A schematic diagram illustrating a screenshot operation provided in this application;

[0082] Figure 8 A schematic diagram illustrating the selection of mixed text and graphics content in an additional view, as provided in this application;

[0083] Figure 9 A schematic diagram of a card set provided in this application;

[0084] Figure 10 This application provides a schematic diagram of opening a card deck;

[0085] Figure 11 Another schematic diagram of opening a card deck is provided for this application;

[0086] Figure 12 A schematic diagram of data reused in a card provided in this application;

[0087] Figure 13 A schematic diagram of data in a reusable card set provided in this application;

[0088] Figure 14 A schematic diagram of a merged card set provided in this application;

[0089] Figure 15 A schematic diagram illustrating the hidden display of a card deck provided in this application;

[0090] Figure 16 A schematic diagram of a single card provided for this application;

[0091] Figure 17 This application provides an illustration of viewing saved content on a webpage.

[0092] Figure 18 An exemplary flowchart of a cross-platform interaction method provided in an embodiment of this application;

[0093] Figure 19 An exemplary flowchart of another cross-platform interaction method provided in this application embodiment. Detailed Implementation

[0094] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0095] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0096] Currently, most applications neither support the collection and viewing of offline content in a mixed text and image format, nor do they support the reuse of this type of material. Although some browsers and other applications can support the collection of entire web pages, they cannot yet support the collection of selected mixed text and image content within a specific area of ​​a web page.

[0097] To address the aforementioned issues, this application provides a data collection method that supports the collection of offline content in a mixed text and image format. This method allows users to select displayed text and image materials on an interface. The electronic device responds to the selection operation by retrieving the selected data. A long-press or touch operation on the selected data indicates the user's intention to collect it. The electronic device responds to the long-press or touch operation, preserving the distribution and order of images and text within the page containing the text and image materials, and storing the selected data in an instance storage unit as a rich text structure. A drag-and-drop operation on the selected data triggers the collection space to retrieve the rich text structure from the instance storage unit. The rich text structure includes the data and its data type. By encapsulating the data in a rich text structure for storage, key information in the mixed text and image format can be preserved to the greatest extent. Collection can be achieved through a collection space, where mixed text and image content can be collected as discrete data or offline web page files. Both discrete data and offline web page files include images, text, the relationships between images and text, and the relationships between text elements, thus enabling the collection of mixed text and image format data without requiring users to collect text and images separately, improving collection efficiency.

[0098] The electronic device of this application extends the function of collecting mixed text and image content in local areas, allowing users to retain key information to the greatest extent with more granularity when collecting mixed text and image content, while preserving the correlation between information and improving the comprehensiveness of information in the collected data.

[0099] Before providing a detailed explanation of the data collection method provided in the embodiments of this application, the terminal involved in the embodiments of this application will be described first.

[0100] Figure 1 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. See also... Figure 1 Terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0101] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal 100. In other embodiments of this application, the terminal 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0102] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0103] The controller can serve as the central nervous system and command center of the terminal 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0104] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0105] The wireless communication function of terminal 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0106] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0107] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on the terminal 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0108] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0109] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0110] In some embodiments, the antenna 1 of the terminal 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal 100 can communicate with the network and other devices through wireless communication technology.

[0111] Terminal 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0112] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal 100 may include one or N displays 194, where N is an integer greater than 1.

[0113] Terminal 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0114] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions, such as saving music, video, and other files on the external storage card.

[0115] Internal memory 121 can be used to store computer-executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of terminal 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created by terminal 100 during use (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0116] Terminal 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D and application processor.

[0117] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Terminal 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, terminal 100 detects the touch operation intensity based on pressure sensor 180A. Terminal 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than the pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0118] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of terminal 100, in a different position than display screen 194.

[0119] The software system of terminal 100 will be described next.

[0120] The software system of terminal 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered Android system as an example to illustrate the software system of terminal 100.

[0121] Figure 2 This is a block diagram of a software system for a terminal 100 provided in an embodiment of this application. The terminal's software system can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the application uses the layered architecture Android system as an example to illustrate the software structure of the terminal.

[0122] See Figure 2A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android Runtime, the system layer, and the kernel layer.

[0123] The application layer can include a series of application packages. For example... Figure 2 As shown in Figure A, the application package can include applications for bookmarking, social networking, shopping, news, games, and communication. The application package can also include applications for camera, gallery, calendar, calling, maps, navigation, WLAN, Bluetooth, music, video, and SMS.

[0124] In the application layer of an electronic device, the following operations can be performed: opening an application, such as a social application, shopping application, news application, game application, communication application, etc., displaying it on the interface, and allowing the user to select text and image materials in the application and drag and drop the selected data.

[0125] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 2 As shown in Figure A, the application framework layer can include an Input Manager, view components, drag-and-drop components, text recognition components, global favorites, and configuration management. The global favorites layer within the application framework layer enables the application layer's favorites space to implement global favorites capabilities. For example... Figure 2 As shown in Figure B, the collection space in the application framework layer is a data collection system implemented on the basis of global collection capabilities. Figure 2 In Figure B, the global collection requires the use of view components. The collection space in the application framework layer is used to provide data support for the collection space application in the application layer. The collection space application in the application layer is used to display the mixed text and image content collected in the collection space of the application framework layer in the form of card groups or single cards.

[0126] Figure 2 The application in Figure B can be any application that can display graphic and textual materials, such as the one mentioned above. Figure 2The diagram in Figure A shows social networking apps, shopping apps, news apps, gaming apps, and communication apps. The View framework within these apps includes a View Tree, a View Recognizer, and an Activity Thread. The Text View, located at the root node of the View Tree, is used to display and edit text, while the Image View, also located at the root node, is used to display images.

[0127] The system server in this application includes a configuration manager, used to configure whitelists for whether applications support drag-and-drop of text and images, and whether applications inherit the operating system's functionality of selecting content areas with a cursor. The system server can obtain the whitelist from the cloud. When a user selects text or images on the application's interface and long-presses or touches them, the whitelist configuration can determine whether the application supports drag-and-drop of text and images, and whether the application inherits the operating system's functionality of selecting content areas with a cursor, thus allowing for different data collection methods under different circumstances.

[0128] The system server also utilizes Warehouse Management Software (WMS) to build drag-and-drop components, which detect user dragging and dropping. When the application from which the text and images originate supports drag-and-drop, the user drags the selected mixed text and images to the Collections application and places it there. The drag-and-drop framework (i.e., the drag-and-drop component) then saves the selected mixed text and images to the Collections application.

[0129] This application embodiment also adds a view recognition module to the application process. The view recognition module is used to identify whether the current page includes a text view and an image view. It is also used to identify whether the application from which the text and image materials originate uses the operating system's native content selection method; that is, the view recognition module is also used to determine whether content extraction can be performed after recognizing the text view and image view. When the application from which the text and image materials originate does not support dragging and drop of text and image content and uses the operating system's native content selection method, the view recognition module obtains the selected area and, based on local strategies, traverses the text and image in the selected area according to the text view and image view under the root node of the application's view tree, performing recognition on the text and image. The view recognition module establishes a communication connection with the drag-and-drop frame through stubs. The view recognition module stores the recognized data (URIs of the text and images) in the instance storage unit of the drag-and-drop frame.

[0130] This application adds an interface to the global collection based on the native Android Application Management Service (AMS) framework. Through this interface, the global collection can request application node information (Request Content Node) from the application process. Upon receiving the request, the application process copies the application node of the current page and transmits this node information to the global collection via an update token. The application node information carries the application's instance (Activity) on the current page. An instance (Activity) consists of all the views on the current page, including Text Views and Image Views. The global collection can traverse through these instances to access all views on the current page, thus extracting content (Content Extract Activity) using the view framework within the global collection. In this example, the global collection leverages the view traversal capabilities of the view framework to locate the Text View and Image View.

[0131] When the application from which the image and text material originates does not support drag-and-drop of image and text content and does not inherit the operating system's native content selection implementation method, the View framework (i.e., the view component) in the global collection obtains the entire current page containing the image and text material from the application and traverses the entire current page View. The global collection provides an image extraction module (Picture ExtractEngine) and a text extraction module (Text Extract Client). The image extraction module extracts images from the entire current page, obtaining all extractable images, temporarily storing the images and obtaining their URIs. The images are then extracted as bitmaps and stored as images; if it's a web view, the images can be downloaded. The text extraction module includes a Software Development Kit (SDK), which allows the installation of an Artificial Intelligence (AI) plugin (AIPlugin, i.e., a text recognition component) to recognize text in screenshots (including text in images). When performing text recognition, the text extraction module can send a screenshot of the entire current page as a bitmap to AIPlugin via an interface (Init). AIPlugin includes a Vision Service, a Text Configuration Service, and an OCR Detection Engine. AIPlugin can select any of these services to perform text recognition on the text in a screenshot of the current page, obtaining the recognition result, which includes the text and its position within the current page. The result can also include text within an image. The recognition result is returned to the Global Collections instance via an API (on Result). The View framework in Global Collections adds an additional View above the current page. It extracts the image based on the image's URI and draws the image and the text recognized by AIPlugin onto this additional View, according to their respective positions within the current page. The additional View contains the same content as the current page and is provided to the user for further selection. Then, the View framework in Global Collections retrieves the selected area, obtains the image URI and text within that area, and stores the recognition data (text and image URIs) in the instance storage unit of the drag-and-drop framework.

[0132] Global Collection also provides a Collection Service and a SensorManage service. The Collection Service is used to collect and store materials, while the SensorManage service is used to access sensors on electronic devices for image acquisition, etc.

[0133] like Figure 2 As shown in Figure B, the input manager includes a three-finger swipe detection module and stubs. The three-finger swipe detection module is used to detect finger swipes and finger presses (Hw Fingers Snap shooter). When a three-finger press is detected, it indicates that the user intends to select the current page, and the global collection is activated through the Bind Service. When a swipe is made after a three-finger press, the global collection responds to this operation, thereby calling (on collectionInvoked) the global collection to retrieve the current entire page from the application. The input manager and the global collection communicate through stubs, which act as an interface for this communication connection.

[0134] It should be noted that if text and image content is saved via a three-finger swipe down, the image's URI can be stored in the global save list. If text and image content is saved via drag and drop, the image's URI can be stored in a temporary process's memory.

[0135] The Collection Space application retrieves text and image URIs from instance storage units to collect mixed text and image content. It can collect content in a hash format, grouping text and images together for easier user access and improved data utilization. The Collection Space application can also collect mixed text and image content as discrete web pages, enhancing the visibility of the collected data.

[0136] The application framework layer may also include a window manager, content providers, a view system, a phone manager, a resource manager, and a notification manager. The window manager manages window programs. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture the screen. The content provider stores and retrieves data, making this data accessible to the application. This data can include video, images, audio, made and received calls, browsing history and bookmarks, and a phone book. The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build the application's display interface, which can consist of one or more views, such as a view displaying SMS notification icons, a view displaying text, and a view displaying images. The phone manager provides communication functions for the terminal 100, such as managing call status (including connection and disconnection). The resource manager provides the application with various resources, such as localized strings, icons, images, layout files, and video files. The notification manager allows the application to display notification information in the status bar, which can be used to convey informational messages and can disappear automatically after a short pause without user interaction. For example, the notification manager is used to notify users of download completions and message alerts. The notification manager can also display notifications as icons or scrolling text in the system's top status bar, such as notifications from background applications. Furthermore, the notification manager can appear as dialog boxes on the screen, such as displaying text messages in the status bar, emitting sounds, causing electronic devices to vibrate, or flashing indicator lights.

[0137] The Android Runtime comprises the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that Java calls, and the other part is the core Android library itself. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0138] The system library can include multiple functional modules, such as a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics engines (e.g., SGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media libraries support playback and recording of various common audio and video formats, as well as still image files. The media libraries support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing libraries are used for 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.

[0139] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, and sensor drivers.

[0140] The following is combined Figure 3 and Figure 4 This application explains the use of the collection space provided.

[0141] The Collection Space application is a management application (APP) that allows users to collect materials of any type. Users can use the Collection Space application to fulfill their material collection needs. Furthermore, the Collection Space application can be developed into a personal study space, work space, etc. The data retention rules in the Collection Space application can be permanent.

[0142] The materials collected in the collection space application include various types of information such as text (e.g., web links, plain text), graphics, images, videos, and web pages. Images can be static images (such as pictures) or dynamic images (such as animations or videos).

[0143] In this application, the data collected in the collection space can be displayed in a floating area. The floating area is a floating window that occupies only a small portion of the electronic device's screen. The floating window can float on the side of the page corresponding to the application without affecting the normal use of the application.

[0144] Users can access their favorites space in two ways: either by opening the favorites space app, or by the electronic device responding to a long press or drag operation on a file, recognizing the user's intention to save it, and launching the favorites space app. The favorites space app is displayed as a floating area, and the upper right corner of this floating area may include three dots to indicate the "More" control. Figure 3As shown in Figure A, users can click or touch the "More" control, responding to the user's action, such as... Figure 3 As shown in Figure B, the electronic device can display options below the "More" control; these options include, but are not limited to, "Enter Full Screen," "Add to Desktop," "Settings," and "Off." In other words, when a user clicks or touches the "More" control at the top of the Favorites app's floating state, the electronic device responds to this click by displaying more functions to the user.

[0145] During the use of the Collection Space app, the device can exhibit various states to indicate that it is in different processing stages or undergoing different processes. For example, the Collection Space state may include a hidden state and a floating state; these are merely illustrative names. For instance, the hidden state may also be called the minimized state, and the floating state may also be called the distributed state. This application does not limit the scope of these terms.

[0146] The hidden state of the collection space indicates that the collection space has been invoked in the foreground or is running in the background; the floating state is used to display the data collected in the collection space in the floating area on the currently displayed interface.

[0147] Optionally, the shape of the hidden state of the collection space (i.e., the hidden area of ​​the collection space) can be a vertical strip. For example, when the collection space is in the hidden state, it can be displayed by attaching to the inner edge of the screen, such as... Figure 4 The icon 'a' in the hidden state is shown in Figure A. When the hidden icon 'a' is positioned inside the screen edge, it will not affect the user's normal viewing of the displayed content. The hidden icon 'a' can be a vertical strip, and it will not visually attract the user's attention or interfere with normal use.

[0148] It should be noted that the position of the inner edge of the screen described in the embodiments of this application can be located on the upper side, lower side, left side or right side of the inner edge, and this application does not limit it in this regard.

[0149] It should be noted that the collection space can switch from a hidden state to a floating state, and vice versa. Optionally, this switching can be an animation effect such as scrolling or scaling. For example, the hidden icon 'a' of the collection space can switch from a vertical bar with a gradually expanding animation effect to a floating icon 'b'. This is merely an example, and this application does not limit it.

[0150] When a user needs to browse the content displayed on the electronic device's interface, they can tap or touch an area other than the floating area to switch the Favorites space from floating to hidden, displaying it on the inside edge of the screen. When a user needs to view or use the data saved in the Favorites space, they can tap or touch the hidden icon to switch the Favorites space from hidden to floating.

[0151] Optionally, each piece of data in the collection space is displayed as a card in the floating area. The floating area of ​​the collection space can be combined with the size and shape of the electronic device to display in different styles in different areas of the screen. For example, the shape of the floating area of ​​the collection space (i.e., the floating area of ​​the collection space) can specifically include two display styles, one being a long strip, such as... Figure 4 The suspended state b1 shown in the image; another type is horizontal stripe-shaped, such as... Figure 4 The floating state b2 is shown in the image. Floating states b1 and b2 can be displayed with an effect of attaching to the inside edge of the display screen, avoiding interference with the currently displayed content. Floating states b1 and b2 also include a progress bar, which is used to indicate and adjust the display progress of the card.

[0152] It should be noted that although the collection space may switch between various forms during use, the collection space in each form is always on top (or pinned to the top) of the multiple display layers included in the display interface, and is not covered or affected by the content displayed below.

[0153] The following describes the implementation of the data collection method provided in this application on electronic devices, using a collection space as an example. Figures 5-8 The application scenarios shown are explained.

[0154] Figure 5 Figure A in the image shows the graphic and text elements on the current phone screen. These elements are arranged in a mixed format, with the "favorites" section hidden at the inner edge of the screen. This is to facilitate differentiation between images and text. Figure 5 In Figure A, the image is placed within a graphic and has a different background than the text. The user selects both the text and the image using the cursor (including the start and end cursors). Figure 5 In Figure A, it is shown filled with gray. For example... Figure 5 As shown in Figure B, when a user wants to save the text and image content (i.e., the text and image selected by the cursor), they can initiate a long press or touch operation. The phone recognizes the user's intention to save and stores the selected text and image content in an instance storage unit as a rich text structure. A smaller version of the text and image content is then displayed on the interface. The smaller version of the text and image content can be moved in the direction the user drags their finger. Figure 5As shown in Figure C, when a user drags a scaled-down version of the mixed text and image content to the sidebar where the Favorites space is located, the Favorites space detects new data and displays a card indicating that the data has been added. The Favorites space retrieves the rich text structure from the instance storage unit and saves the mixed text and image content. Figure 5 As shown in Figure D, after a text-image mixed content is successfully saved in the collection space, a card indicating successful saving is displayed in the floating area, thus completing the saving of the text-image mixed content.

[0155] It should be noted that, Figure 5 The example given is that the user has already enabled the collection space. It can be understood that the timing of enabling the collection space can also be determined by… Figure 5 The long press or touch operation of image B in the image can also be triggered by... Figure 5 The drag-and-drop operation of image C in the image triggers the Collection Space application. After the Collection Space is enabled, it is displayed in a hidden state in the sidebar of the interface, without affecting the user's normal viewing of the content. This embodiment of the application does not limit this. By launching the Collection Space application at different trigger times, the richness of application scenarios is improved.

[0156] Understandably, the above Figure 5 This is merely an illustrative example of an interface display; optionally, it may not be shown to the user. Figure 5 The D diagram in the image.

[0157] This application can complete data collection in the following ways. The following example illustrates how a View frame represents view components, a drag-and-drop frame represents drag-and-drop components, and an OCR Detect Engine represents text recognition components.

[0158] In one implementation, the application that provides the text and image content supports drag-and-drop. In this case, the application responds to the user's long press or touch operation, placing the selected text and image content into an instance storage unit with a rich text structure (e.g., HTML text), and then initiates the drag-and-drop framework. The instance storage unit is the ClipData of the drag-and-drop framework, a class used to pass text and URI data between views. When the user initiates a drag operation, the drag-and-drop framework responds by encapsulating the selected text and image content into a smaller version. The smaller version moves across the interface following the direction of the drag, until it reaches the collection space, where it is dropped. The collection space retrieves the rich text structure from the ClipData and stores the text and image content there. The interface display for this implementation can be referenced above. Figure 5 The description will not be repeated here.

[0159] In another implementation, the application that provides the text and image materials doesn't inherently support drag-and-drop of content. However, this application is a native application of the electronic device, or it inherits from a class native to the Android operating system that specifically provides content area selection functionality. This class uses Android's native Clipboard Manager class to implement cursor selection. Typically, when selecting content in an application, a cursor is displayed, and the selected content can be retrieved by dragging the cursor down. If the application inherits from the operating system's native class that specifically provides content area selection functionality and implements content selection accordingly, or if the application directly uses the operating system's native class that specifically provides content area selection functionality, then after the user selects a content area, the operating system sets the selected content to the corresponding class. Then, it retrieves or reads the URIs of the text and images from these classes, effectively allowing it to retrieve the selected content.

[0160] In this scenario, the application inherits the functionality of selecting content areas with the cursor, and the operating system can use its native content selection implementation method to identify the selected area. When the user performs a long press or touch operation on the selected text and image content on the page, the View framework recognizes the long press or touch event and, based on the system's native content selection implementation method inherited by the application, obtains the selected area corresponding to the selected text and image content. A long press operation can refer to pressing on the View containing the text and image content for a preset time. The View framework identifies the URI of the image in the selected area and the text within the selected area. The image URI can be a download link for an image or video. The image URI and text are encapsulated as rich text structure (e.g., HTML text) and placed in the instance storage unit, and the drag-and-drop framework is initiated. Next, the user begins a drag-and-drop operation, and the drag-and-drop framework responds by dragging the selected text and image content to the favorites space. The favorites space retrieves the rich text structure from the instance storage unit and saves the text and image content there. The interface display of this implementation can be referenced above. Figure 5 The description will not be repeated here.

[0161] HTML text uses a markup language to describe the structure and content of a webpage. It's a form of rich text that maintains the formatting characteristics of rich text. The style information in HTML text describes how HTML elements are rendered, such as colors, fonts, sizes, layouts, paragraphs, and line formats. When a browser parses HTML text, it renders the HTML elements into the webpage that the user sees on their screen based on this style information.

[0162] In another implementation, the application from which the text and image materials originate does not support dragging and dropping text and image content, and the application does not inherit the Android operating system's native class that specifically provides content area selection functionality. For example, if the application is a third-party application and its content selection method uses custom logic, meaning the application does not inherit the functionality to select content areas using a cursor. In this case, when the user performs a long press or touch operation on the selected mixed text and image content on the page, the View framework uses the entire current page containing the text and image materials as the target. On one hand, the View framework traverses the entire current page View, obtaining all extractable images (pictures or videos) and their positions within the current page, temporarily storing the images and obtaining their URIs. On the other hand, the View framework passes the bitmap corresponding to a screenshot of the entire current page to the OCR detection engine. A bitmap, also known as a raster image or bitmap, is an image represented using a pixel array or dot matrix. The OCR engine recognizes the text in a screenshot of the current page, obtaining the text and its position within the page. It then returns the text and its position to the View framework, either in paragraph or line format. The View framework adds an additional View above the current page, extracting the image from its URI. The image, along with the text returned by the OCR engine, is then drawn onto this additional View, according to their respective positions within the page. In other words, the View framework redraws the entire page as an additional View without altering the existing text and image layout, providing it to the user for further selection. Because this View is drawn by the View framework itself, it can identify the selected area corresponding to the mixed text and image content and recognize the content within that area.

[0163] The user reselects the desired text and image content in the attached view, performing a long press or touch operation on the selected content. The View framework recognizes the long press or touch event and obtains the selected area corresponding to the chosen text and image content. The View framework identifies the URI of the image and the text within the selected area, constructs ClipData using HTML text styles from the image URI and text (preserving paragraph information), and initiates the drag-and-drop framework. Next, the user begins the drag-and-drop operation, and the drag-and-drop framework responds by dragging the selected text and image content to the Favorites space. The Favorites space retrieves the HTML text from the ClipData and saves the text and image content from the attached view to the Favorites space.

[0164] For example, such as Figure 6 As shown, Figure 6 Figure A in the diagram illustrates an additional view, which is displayed on the user's view... Figure 5 In Figure A, a long-press or touch operation on the selected text and image content triggers the view frame to generate an additional view. This additional view displays the text and image content of the entire current page. If the user selects the text and image content in the additional view again, such as... Figure 6 As shown in Figure B, the user selects text and images in the supplementary view using the cursor (including the start and end cursors). Figure 6 In Figure B, it is shown filled with gray. For example... Figure 6 As shown in Figure C, when a user wants to save the selected text and image content (i.e., the text and image selected by the cursor in the supplementary view), they can initiate a long press or touch operation. The phone recognizes the user's intention to save and stores the selected text and image content in the supplementary view as a rich text structure in the instance storage unit. A smaller version of the text and image content is then displayed on the interface. The smaller version of the text and image content can be moved in the direction the user drags their finger. Figure 6 As shown in Figure D, when a user drags a scaled-down version of the mixed text and image content to the sidebar where the Favorites space is located, the Favorites space detects the new data and displays a card indicating that the data has been added. The Favorites space retrieves the rich text structure from the instance storage unit and saves the mixed text and image content.

[0165] Understandable, Figure 6 Following the D diagram in the image, the above can also be displayed. Figure 5 The card shown in Figure D indicates a successful collection, thus completing the collection of the mixed text and image content. This embodiment of the application does not show this.

[0166] Figure 6 The supplementary view of Figure C can be hidden after the user selects the mixed text and image content, or it can remain displayed for the user to bookmark again. The user can cancel the supplementary view by clicking or pressing an area other than the supplementary view itself, or the supplementary view includes a close button, which the user can click or press to cancel the supplementary view. This embodiment of the application does not limit this aspect.

[0167] It should be noted that the text and image content selected by the user in the additional view can be different from the text and image content selected initially. Figure 5 Image A in the middle shows the text and image content selected by the user for the first time. Figure 6The image in Figure B shows the text and image content selected by the user again; the two are not exactly the same. For some applications, after multiple uses, users realize that the application does not support direct drag-and-drop and requires regenerating the additional view. Users can, on the first selection of data, perform a selection and long-press operation on any data (which may not actually be the data they want to save), and wait for the regenerated additional view. Then, they can perform a selection operation on the text and image materials in the additional view to select the data they actually want to save. Users do not need to waste time on the first selection, which can improve the efficiency of saving.

[0168] In another implementation, the user performs a three-finger swipe down on the interface to take a screenshot. This triggers the View framework to draw an additional View above the current page, which is then provided to the user for further selection. The user can then save the text and image content on the additional view. The drawing and saving processes can be found in the description above for applications that do not support dragging text and image content and do not inherit the functionality of selecting content areas with the cursor, and will not be repeated here.

[0169] For example, such as Figure 7 As shown, Figure 7 The example shown is a three-finger swipe down to take a screenshot of the entire current page. After taking the screenshot, please refer to the above. Figure 6 The description of how to save content with mixed text and images is omitted here.

[0170] For cases where the image also includes text, as described above Figure 5 Figure A in the middle and Figure 7 The View framework passes the bitmap corresponding to a screenshot of the entire current page to the OCR detection engine. The OCR detection engine recognizes the text in the screenshot, including text within images, obtaining the text (including text within images) and its position on the current page (including its position within images), and returns the text and its position to the View framework. The View framework then extracts the image based on its URI, and, based on the image and the text returned by the OCR engine (including text within images), generates an additional view according to the text's position on the current page (including its position within images) and the image's position on the current page. The additional view generated by the View framework includes the text, the image, and the text within the image.

[0171] Figure 6 The additional view shown in Figure A includes text, images, and text within images. When a user selects mixed text and image content in the additional view, they can also select text within the image. For example... Figure 8 As shown, Figure 8 Is Figure 6 Select it in the additional view of Figure A. Figure 8 The selected mixed text and image content includes text, images, and text within images. (The above...) Figure 6 Figure B in the text is Figure 6 Select it in the additional view of Figure A. Figure 6 The selected text and image content in Figure B includes both text and images.

[0172] Based on the above Figures 9-17 This section introduces the display format of collected data in the floating area.

[0173] The image and text content collected in the collection space can be displayed in the form of a card set. The collection space collects images and text in a hash format, that is, images and text are collected separately, with each image and text corresponding to a card. These cards are displayed in a floating area, indicating that the images and text originated from the same collection, preserving the association between them. The collection space retrieves rich text structures from instance storage units. These rich text structures include data and data types (e.g., HTML tags), with the HTML tags reflecting the positions of the images and text on the page. The collection space downloads or copies the images and text, and then groups them together according to their original relative positions on the page, collecting them in a card set. The card set includes cards corresponding to each image and text. The card set can display an application source identifier for the image and text materials. The application source identifier can be an application icon or application name, indicating which application the card set originated from. The icon can be a general identifier for an application, and this application does not impose any limitations on this. When saving multiple sets of mixed text and image content from the same application, you can distinguish the card sets by numbering them or by marking them with timestamps.

[0174] For example, such as Figure 9 As shown in Figure A. Figure 9 Figure A in the above Figure 5 The suspended region is shown based on the D diagram in the figure. Figure 5 The mixed text and image content collected in image D comes from application a. Figure 9 Figure A in the diagram illustrates a card group in a floating area, displaying the application name (application a) and the number of cards in the group (3 items). The card group can also be... Figure 9 As shown in Figure B, the card set displays the icon of application a and the number of cards in the card set (3 items). The icon of application a indicates that the card set comes from application a's collection. Of course, this is just an example, and it can be displayed in other forms, which this application does not limit.

[0175] For situations requiring the generation of additional views, users can select text within an image when selecting mixed text and image content within the additional view. The saved card group also includes cards corresponding to the text within the image, such as... Figure 9 As shown in Figure C, Figure 9 Figure C in the above Figure 8 Based on the floating area shown, the card group displays the application name (application a) and the number of cards in the card group (4 items). Of course, the name of the card group can also be edited by the user, and this embodiment of the application does not limit this.

[0176] It should be noted that the application source identifier can be set in any of the top left, bottom left, top right, or bottom right corners of the card group; this application does not limit this.

[0177] The selected mixed text and image content must include at least an image and text. The corresponding card set must contain at least two cards: one displaying text and the other displaying an image. The number of cards in a card set depends on the division rules of the text and image materials. For example, the division rule might divide all images in the mixed text and image content into one part and all text into another part. In this case, the card set would contain two cards: one displaying text and the other displaying an image. Alternatively, the division rule might use images as the dividing line. If an image is embedded within text, dividing the text into two paragraphs, then the mixed text and image content would be divided into paragraph 1, image, and paragraph 2, as described above. Figure 9 Figure A in the above and Figure 9 As shown in Figure B, if an image is at the beginning or end of a mixed text / image content, the content is divided into image and text, or text and image. Alternatively, the division rule can be based on the line spacing between paragraphs within the text. If the text includes at least two paragraphs with relatively large line spacing (e.g., greater than a preset spacing), the text in the mixed text / image content is divided into paragraphs and paragraphs.

[0178] When saving mixed text and image content, any combination of the above division rules can be used. For example, mixed text and image content includes paragraphs with line spacing greater than the preset spacing, and images located between any two paragraphs. The paragraphs and images in this mixed text and image content correspond to their respective cards.

[0179] Typically, users collect text and image materials for document editing or file creation. Distributed collection allows users to directly access the data, such as a piece of text or an image, increasing the flexibility of the collected data. Compared to reusing data from web pages, this method has a shorter data reuse path, improving data utilization efficiency and enhancing the user experience.

[0180] It should be noted that, Figure 9 The card set shown is the one displayed for this collection of mixed text and image content. It's understandable that mixed text and image content was already collected in the collection space before this collection, and the floating area can still display previously collected card sets. Of course, after this collection, more mixed text and image content can be collected in the collection space, and the floating area can still display newly collected card sets. Figure 9 This is merely an example of a card deck and does not constitute a limitation of this application.

[0181] like Figure 10 As shown in Figure A, users can open card groups by clicking, pressing, or touching them. The electronic device responds to these actions by displaying... Figure 10 As shown in Figure B, the card set includes 3 cards. Meanwhile, in... Figure 10 In Figure B, a back control is displayed in the floating area for navigating to the previous level of the card group. The previous level of the card group is the Collection Space app. This allows users to further view and reuse individual cards, improving the user experience.

[0182] The display area on an electronic device's interface is limited; therefore, the data displayed on a card is also limited. A card in a floating area might display a portion of a text-and-image content; users can zoom in on the card by triggering it, thus viewing the complete text-and-image content. For example... Figure 10 As shown in Figure C, users can zoom in on any card by clicking, pressing, or touching it, thus viewing the complete data within the card. The electronic device responds to the user's click, press, or touch actions on the card group by displaying... Figure 10 The card shown in Figure D contains complete data. A magnified card can include: complete paragraph data, the paragraph's acquisition timestamp, and the paragraph's total word count, etc.

[0183] For users who have also saved text within images, the card set in the saved space also includes cards corresponding to the text within the images, as described above. Figure 9 As shown in Figure C. Based on the above. Figure 9 The card sets collected by C-picture, such as Figure 11 As shown in Figure A, users can open card groups by clicking, pressing, or touching them. The electronic device responds to these actions by displaying... Figure 11 As shown in Figure B, the card set includes four cards, one of which corresponds to the text in the image.

[0184] For data collected in the form of card groups, for the collected text or paragraphs, the cards in the card group display all or part of the text. For example, a card can display all the characters of the text, or the first few characters, keywords, middle characters, or last few characters, etc. Figure 10 As shown in Figure B, the card displays part of the text: "Translation: The bright moonlight shines on the railing above the well, as if a layer of white frost has appeared on the ground." The full text is: "Translation: The bright moonlight shines on the railing above the well, as if a layer of white frost has appeared on the ground. I couldn't help but look up at the bright moon in the sky outside the window, and then I couldn't help but lower my head in thought, thinking of my distant hometown." For collected images, the cards in the card set display the image itself, or a thumbnail or partial screenshot of the image. A partial screenshot can be a screenshot of the upper left, upper right, lower left, lower right, or center portion of the image. The partial characters and partial images displayed in the floating area in this application are for illustration purposes and may not be the entirety of the text or image.

[0185] To better distinguish between the mixed text and image content in the collection space and the data displayed in the cards (which may be partial data), the mixed text and image content in the collection space will be referred to as edited data, which is complete data.

[0186] For example, taking a user's saved text and image content on their mobile phone as an example, the user opens the editing software, and the editing area on the interface displays the editing cursor, such as... Figure 12 As shown in Figure A, for cards corresponding to text or paragraphs, users can drag and drop the complete data of the corresponding text or paragraph from the collection space to the editing area, as shown in Figure A. Figure 12 As shown in Figure B. After dragging and dropping the card into the editing area, the user releases the card, and the entire text is displayed in the editing area, as shown below. Figure 12 As shown in Figure C, the editing cursor is moved to the end of the text to provide a starting position for the next paste. Furthermore, after displaying all the text in the editing area, the favorites space is shown in a hidden state; the favorites space switches from a floating state to a hidden state, as shown... Figure 12 As shown in Figure D, this makes it easier for users to view the pasted content.

[0187] Of course, users can also select cards by touching or long-pressing them, and then paste the text into the editing area. This application does not limit this.

[0188] For saved images, the pasting method can be the same as for text. The difference is that images typically consume more memory than text, so they need to be stored on the electronic device using a distributed file storage method. The saved space contains image information, which may include image thumbnails or partial screenshots, image storage location information, application source identifier, and image acquisition timestamp. When a user retrieves an image, they select the corresponding card, and the image is retrieved using the image's storage location information, thus displaying the image in the editing area.

[0189] For data collected in card sets, users can directly reuse these card sets. For example, when a user reuses mixed text and image content on their phone, they open an editing app, and the editing area on the interface displays an editing cursor, such as... Figure 13 As shown in Figure A, users can drag and drop the complete data of the corresponding text or paragraph and the image from the collection space into the editing area, as shown in Figure A. Figure 13 As shown in Figure B. After dragging and dropping into the editing area, the user releases the card group, and the entire text and image are displayed in the editing area according to the order of the cards in the group, as shown below. Figure 13 As shown in Figure C, the editing cursor is moved to the last position to provide a starting position for the next paste. Furthermore, after displaying all text and images in the editing area, the favorites space is displayed in a hidden state. The favorites space switches from a floating state to a hidden state, as shown... Figure 13 As shown in Figure D, this makes it easier for users to view the pasted content.

[0190] For data collected in card sets, users can also merge card sets, such as... Figure 14 As shown in Figure A, the floating area of ​​the collection space displays card group 1, card group 2, card group 3, and card group 4. When a user selects any card group (e.g., card group 1) to merge it, the user can drag card group 1 (containing 2 cards) into card group 3 (containing 4 cards) along the drag direction. Figure 14 As shown in Figure B. To facilitate the differentiation of merged card groups, Figure 4 The Sino-Russian B diagram shows the merged card group filled in gray. After dragging and dropping to the corresponding position in card group 3 (which can be a preset area around card group 3), the user releases card group 1, and card group 1 and card group 3 merge into one card group, as shown below. Figure 14As shown in Figure C, a new card group is displayed in the floating area. The new card group includes card group 1 and card group 3. The new card group can also display the application name of the merged card group (e.g., card group 3) or the application name of the merged card group (e.g., card group 1). The new card group can also display the number of cards in the new card group (e.g., 6 items). Of course, the user can also edit the new name of the new card group, and this embodiment of the application does not limit this.

[0191] After merging card group 1 and card group 3, card group 1 will no longer be displayed in the floating area. Correspondingly, if the new card group moves to the top or bottom along with other card groups, the number of card groups currently displayed in the floating area will decrease, thereby revealing other hidden card groups and improving the visibility of the data collection. Figure 14 Figure C in the illustration uses an example of an unhidden card group for demonstration purposes and does not constitute a limitation on this application. The display method of the card group in the floating area after card groups are merged is not limited in the embodiments of this application.

[0192] Because the display area on an electronic device's interface is limited, the device's interface uses a collection space to store multiple pieces of mixed text and image content. A limited number of these pieces are displayed in a floating area, essentially a limited number of card groups, while other card groups are hidden. Hidden card groups can be rotated by dragging or scrolling a progress bar. The floating area displays multiple objects (each object can be an image or paragraph) from each piece of mixed text and image content as a combination of cards. Each card displays all or part of each object, and the cards maintain a corresponding order within the card group. Multiple cards can be displayed vertically, side-by-side, or overlaid, etc., and this embodiment does not impose any limitations on this.

[0193] When multiple pieces of mixed text and images are displayed in a card group format in the floating area of ​​different electronic devices, they can be sorted according to the acquisition timestamp of each piece of mixed text and images.

[0194] It should be noted that if the card group appears from the bottom of the floating area, the text and image content with the most recent timestamp will be displayed as a card group at the bottom, while the other card groups at the top will be hidden. If the card group appears from the right of the floating area, the text and image content with the most recent timestamp will be displayed as a card group on the far right, while the other card groups on the far left will be hidden.

[0195] Due to the limited display area of ​​the interface, the number of card groups that can be displayed in the floating area is also limited. The upper limit of the number of card groups that can be displayed in the floating area can be appropriately set by those skilled in the art based on the size of the electronic device. When a new card group needs to be displayed, if the number of card groups already displayed in the floating area has reached the upper limit, the earliest displayed card group needs to be hidden, and the new card group needs to be displayed simultaneously. Since the display order of card groups in the floating area can be sorted according to the card group's acquisition timestamp, the card group with the earliest acquisition timestamp can be hidden here.

[0196] The hiding and showing of card sets can take several forms. First, the collection space can directly display the final new card set to the user without showing the hiding and showing process. Second, the hiding and showing process can be shown to the user. For example, the card set at the top of the floating area gradually hides, while the newly added card set gradually appears at the bottom of the floating area, until the top card set is completely hidden, at which point the newly added card set is fully displayed at the bottom. Alternatively, the top edge of the top card set can be completely hidden when it reaches the top edge of the floating area, while the middle card sets and the newly added card set move upwards simultaneously, with the newly added card set gradually appearing at the bottom of the floating area until it is fully displayed. Finally, this application does not limit the form of hiding and showing card sets.

[0197] When the number of card groups displayed in the floating area reaches its limit, the first card group shown will be hidden. A progress bar will appear in the floating area, used to adjust the display of card groups. Users can drag or scroll the progress bar to reveal other hidden card groups. Figure 15 As shown in Figure A, the phone responds to the user's drag operation, determines the drag displacement based on the drag direction, and moves the card group in the floating area according to the drag displacement to achieve the rotation of card group display. Figure 15 As shown in Figure B, the topmost card group in the floating area gradually hides, while the hidden card group gradually appears at the bottom of the floating area. As the finger continues to drag, the progress bar shows the topmost card group completely hidden, and the hidden card group fully displayed at the bottom of the floating area, as shown. Figure 15 As shown in Figure C.

[0198] It should be noted that the method of hiding or showing card sets can be done as a whole. For example, when the topmost card set reaches the top edge of the floating area, it skips the gradual hiding and becomes completely hidden. Simultaneously, the middle card sets move upwards until they reach the top edge, at which point the hidden card set is fully displayed at the bottom of the floating area. Alternatively, when the topmost card set reaches the top edge of the floating area, it can be completely hidden, while the middle and hidden card sets move upwards simultaneously, with the hidden card set gradually appearing at the bottom of the floating area until it is fully displayed. This application does not limit the form of hiding and showing card sets.

[0199] The image and text mixed content collected in the collection space can be displayed as an offline web page file. The collection space obtains the rich text structure from the instance storage unit. The rich text structure includes data and the data data type (e.g., HTML tags). The HTML tags reflect the position of the images and text on the page. The collection space downloads or copies the images and text, and simultaneously arranges the images and text according to their original relative positions on the page, forming a standard HTML text style to generate an offline web page file. After obtaining the rich text structure, the collection space can also modify some data types in the HTML tags. However, in this application, the collection space preserves the original tag style as much as possible, typesets the data according to the original information (original relative positions and data data types), and generates an offline web page file according to the style of HTML text on the web page.

[0200] For example, such as Figure 16 As shown, Figure 16 In the above Figure 5 The suspended region is shown based on the D diagram in the figure. Figure 5 The mixed text and image content collected in image D comes from application a. Figure 16 A single card is shown in a floating area. This card displays the icon of application 'a' and the title of the webpage corresponding to the mixed text and image content. The icon of application 'a' indicates that the offline webpage file in this card comes from application 'a's' collection. Of course, this is just an example, and it can be displayed in other forms, which this application does not limit. The single card can also display the size of the offline webpage file (e.g., 544.18KB) and the generation timestamp of the offline webpage file (e.g., 2023.1.30).

[0201] It should be noted that, Figure 16The single card shown is the one displayed for this collection of mixed text and image content. It's understandable that mixed text and image content was already collected in the collection space before this collection, and the floating area can still display previously collected single cards. Of course, after this collection, more mixed text and image content can be collected in the collection space, and the floating area can still display newly collected single cards. Figure 16 This is merely an example of a single card and does not constitute a limitation of this application.

[0202] After a single card is displayed in the floating area, users can view it through a browser, file manager, or other applications that can browse HTML styles, such as... Figure 17 As shown in Figure A, users can open a single card by clicking, pressing, or touching it. In response to the user's click, press, or touch action on the card, the electronic device displays options for opening offline web page files on the interface, such as... Figure 17 As shown in Figure B, this option includes, but is not limited to, "Third-Party Applications," "Browser," and "More" controls. The "Third-Party Applications" option refers to applications that can browse HTML-style content. Users can also click, press, or touch the "More" control to display more third-party applications that can open offline web page files. Figure 17 As shown in Figure C, when a user clicks, presses, or touches the browser, selecting "Open Offline Web Page Files" as the browser's default action. Figure 17 As shown in Figure D, open the offline page file in the browser.

[0203] It should be noted that when saved text and image content is displayed as individual cards in the floating area, the floating area includes a progress bar. This progress bar is used to adjust the progress of the individual cards displayed in the floating area. Users can drag or scroll the progress bar to reveal other hidden individual cards. See the above explanation... Figure 15 The description will not be repeated here.

[0204] The bookmarking space allows users to save text and image content as offline webpage files. This makes it easier for users to review content of interest and improves data intuitiveness. Compared to saving the entire page, it reduces the need for users to search through the entire webpage to find what they are interested in. The saved offline webpage files only include the text and image content of interest and retain the original layout, enhancing the user experience when viewing saved data.

[0205] Collection space can be adopted as follows Figures 9-15 The floating area shown allows for the collection of mixed text and image content in a hash format. Alternatively, it can be done using methods such as... Figure 16 and Figure 17 The floating area shown allows for offline collection of text and image content in a webpage format. The implementation of the collection space can be appropriately configured by those skilled in the art based on actual circumstances, and this embodiment does not impose any limitations on this.

[0206] Understandably, the collection space can also collect mixed text and image content separately in a hash format and as an offline webpage format, that is, collect mixed text and image content into a card group and a single card. In this way, users can reuse the collected text and images through card groups, and view the collected mixed text and image content through single cards, which increases the diversity of data collection formats, meets users' personalized needs, and improves the user experience.

[0207] Figure 18 An exemplary flowchart of a data collection method provided in this application embodiment.

[0208] In some possible cases, exemplary processes involving the collection of graphic materials on electronic devices can be referred to in the following description of S101-S104.

[0209] S101. Display the first interface, which includes the first material.

[0210] It is understandable that the first interface can be the interface that displays text and images as mentioned above.

[0211] In one possible implementation, the first interface can be as described above. Figure 5 The display interface is shown in Figure A. At this point, the first material includes text and images.

[0212] S102. In response to a selection operation on the first material, first selection data is obtained, the first selection data including the first image and the first text in the first material.

[0213] In one possible implementation, the selection operation of the first material can be done through... Figure 5 In Figure A, the start and end cursors are set, and the first selected data can be... Figure 5 The image and text shown in gray fill in Figure A are from this diagram.

[0214] S103. In response to a first operation on the first selected data, the first selected data is stored in the instance storage unit in a rich text structure. The first operation refers to an operation with the intention of saving.

[0215] In one possible implementation, the first operation could be a long press or a touch operation.

[0216] S104. In response to a drag operation on the first selected data, the first selected data is saved in the save space according to the rich text structure.

[0217] In one possible implementation, the drag-and-drop operation can be... Figure 5 The drag-and-drop operation in diagram C.

[0218] The relevant content in S101-S104 can be referred to the above description of collecting mixed text and image content, and will not be repeated here. The exemplary interfaces involved in S101-S104 can be referred to the above description of... Figures 5-17 The relevant content will not be described in detail here.

[0219] In this application, the user selects text and image materials displayed on the interface. The electronic device responds to the selection operation by retrieving the selected data. The user's first operation on the selected data indicates a potential intention to save it. The electronic device responds to this first operation, preserving the distribution and order of the images and text within the page containing the text and image materials, and storing the selected data in an instance storage unit as a rich text structure. The user's drag-and-drop operation on the selected data triggers the save space to retrieve the rich text structure from the instance storage unit. The rich text structure includes the data and its data type. By encapsulating the data in a rich text structure for storage, key information in the mixed text and image format can be preserved to the greatest extent. Saved data or offline web page files include images, text, the relationships between images and text, and the relationships between text elements, thus enabling the saving of mixed text and image format data without requiring users to save text and images separately, improving saving efficiency.

[0220] The following is combined Figure 19 This section introduces the data storage process for the storage space of electronic devices.

[0221] This explanation uses the View framework to represent view components, the drag-and-drop framework to represent drag-and-drop components, and the OCR detection engine (OCR DetectEngine) to represent text recognition components.

[0222] S11. The user selects graphic materials in the application on the interface displayed on the electronic device.

[0223] Taking a mobile phone as an example, as mentioned above... Figure 5 As shown in Figure A, users select text and images in the application using the start and end cursors.

[0224] S12. In response to a long press operation performed by the user on the selected mixed text and image content, the application stores the mixed text and image content in ClipData as HTML text.

[0225] The application can be a native app on the electronic device or a third-party app. Some applications support drag-and-drop functionality for text and images (e.g., the built-in browser of the electronic device, third-party browsers, etc.). In this case, when a user wants to save the text and image content, they can initiate a long-press operation. The application recognizes the user's intention to save and stores the selected text and image content as HTML text in ClipData. Simultaneously, a scaled-down version of the text and image content is displayed on the interface, as described above. Figure 5 As shown in Figure B, users can drag and drop a smaller version of the mixed text and image content.

[0226] The scaled-down version of the mixed text and image content can be moved in the direction the user drags their finger.

[0227] S13. In response to the user's drag-and-drop operation on the mixed text and image content, the application starts the drag-and-drop framework.

[0228] Users begin dragging and dropping text and image content. The application responds to the drag-and-drop action by launching the drag-and-drop framework.

[0229] After S13, S18-S19 can be executed.

[0230] S14. If the application itself does not support dragging and dropping of text and images, and inherits the operating system's native content selection implementation method, then the application starts the View framework.

[0231] If an application inherits the functionality of selecting a content area with a cursor, it can use the operating system's native content selection method to identify the selected area. When a user wants to save a piece of mixed text and images, they can perform a long press operation on the content, launching the View framework. The View framework recognizes the long press event and, based on the system's native content selection method inherited by the application, obtains the selected area corresponding to the selected mixed text and images.

[0232] S15. The View framework recognizes the long press event and, based on the operating system's native content selection implementation method inherited by the application, obtains the selected area corresponding to the mixed text and image content selected by the user.

[0233] S16. The View framework retrieves the URIs of the selected area's text and image from the View, encapsulates the retrieved data into HTML text, and stores it in ClipData.

[0234] Images include pictures and videos. An image's URI represents a download link, allowing you to retrieve the image from a specific location on the network. HTML text uses a rich text structure, and ClipData is a storage container within the drag-and-drop frame, also known as an instance storage unit. The View frame recognizes text and images within the selected area.

[0235] S17. In response to the user's drag-and-drop operation on the mixed text and image content, the View framework starts the drag-and-drop framework.

[0236] As mentioned above Figure 5 As shown in Figure C, the interface displays a smaller version of the mixed text and image content, which the user can drag and drop.

[0237] After S17, S30-S31 can be executed.

[0238] S18. The user takes a screenshot of the interface.

[0239] As mentioned above Figure 7 As shown, users can take a screenshot of the current interface by swiping down with three fingers. If the user directly performs a three-finger swipe down on the interface, the View framework will target the entire page containing the image assets. The interface displays the application's image and text assets. When a user takes a screenshot, regardless of whether the application itself supports dragging and dropping image and text content or inherits the operating system's native content selection implementation method, the View framework will target the entire page containing the image assets.

[0240] S19. In response to the user's screenshot action, the application launches the View framework.

[0241] In this embodiment, the three-finger swipe down operation can trigger the launch of the View framework.

[0242] In this example, a screenshot is taken in response to the user's three-finger swipe down gesture, thereby launching the View framework.

[0243] After S19, S21-S31 can be executed.

[0244] S20. In response to a user's long-press operation on the selected mixed text and image content, if the application itself does not support dragging and dropping text and image content and does not inherit the operating system's native content selection implementation method, then the application starts the View framework.

[0245] Following S11 above, if the application is a third-party application and its content selection method uses custom logic (meaning the application does not inherit the functionality of selecting content areas with the cursor), then the user initiates a long-press operation on the selected mixed text and image content, launching the View framework. The View framework targets the entire page containing the image material.

[0246] After S20, S21-S31 can be executed.

[0247] S21. The View framework traverses the entire current page View, obtains all extractable images in the current page and their positions in the current page through the View, temporarily stores the images, and obtains the image URIs.

[0248] Images include pictures and videos. The View framework recognizes text and images throughout the current page.

[0249] S22. The View framework sends the bitmap corresponding to the screenshot of the entire current page to the OCR detection engine.

[0250] The View framework passes the bitmap corresponding to the screenshot of the entire current page to the OCR detection engine.

[0251] S23. The OCR detection engine recognizes the text in the screenshot of the current page and obtains the text and its position in the current page.

[0252] S24. The OCR detection engine sends the text and its position on the entire page to the View frame.

[0253] The OCR detection engine recognizes the text in a screenshot of the entire current page, obtains the text and its position on the current page, and returns the text and its position on the entire page to the View frame, which can be returned in paragraph or line format.

[0254] S25. The View framework adds an additional view above the current page, and draws the images and text on the additional view according to their positions on the current page.

[0255] As mentioned above Figure 6 As shown in Figure A, the View framework generates an additional view above the current page, and the content displayed in the additional view is the same as the page content of the page containing the text and image materials.

[0256] The image is extracted based on its URI. The image and the text returned by the OCR detection engine are then drawn onto the additional View according to the positions of the text and the image on the current page, and presented to the user for further selection.

[0257] S26. The user re-executes the selection operation in the additional view.

[0258] As mentioned above Figure 6 As shown in Figure B, the user in Figure 6 Select the desired text and image content again in the additional view shown in Figure A.

[0259] When the aforementioned OCR detection engine recognizes text in a screenshot, it also recognizes text from the image. Therefore, the generated additional view includes text from the image. Based on this, the text selected by the user can also include text from the image, such as... Figure 8 As shown, the text selected by the user can include text from an image.

[0260] S27. In response to the user's long press operation on the reselected mixed text and image content, the View framework recognizes the long press event and obtains the selected area corresponding to the mixed text and image content selected by the user in the attached view.

[0261] As mentioned above Figure 6 As shown in Figure C, when a user wants to save the selected text and image content in the supplementary view, they can long-press to display a smaller version of the text and image content. The smaller version of the text and image content can be moved in the direction the user drags their finger.

[0262] Since the attached view is generated by the View framework, the View framework can obtain the selected area corresponding to the mixed text and image content selected by the user in the attached view.

[0263] S28. The View framework obtains the URIs of the text and images in the selected area, encapsulates the obtained data into HTML text, and stores it in ClipData.

[0264] The View framework identifies the URI of the image (i.e., the download link of the image) and the text in the selected area, constructs ClipData with the text and image styled as HTML text, and launches the drag-and-drop framework. The text retains paragraph information, and the image is placed with its URI.

[0265] S29. In response to a user's drag-and-drop operation on the mixed text and graphics content in the attached view, the View framework starts the drag-and-drop framework.

[0266] As mentioned above Figure 6 As shown in Figure D, users can drag and drop a smaller version of the mixed text and image content.

[0267] Users begin dragging and dropping text and image content. The electronic device responds to the drag-and-drop action by activating the drag-and-drop framework.

[0268] S30. In response to the user's operation of placing mixed text and image content into the corresponding area of ​​the collection space, drag the frame to start the collection space.

[0269] When a user drags and drops text and image content into the hidden area of ​​the collection space, it indicates that the user wants to collect the text and image content. In this case, the drag-and-drop frame will launch the collection space.

[0270] The sidebar of the terminal displays a hidden area corresponding to the favorites space. When data is saved, the favorites space is attached to the sidebar in a hidden state, without affecting the user's viewing and operation of the interface. Of course, the favorites space can be displayed in a floating state, which is not a limitation of this application. As mentioned above. Figure 6 As shown in Figure D, when a user drags a scaled-down version of the mixed text and image content to the hidden area corresponding to the collection space (which can be the sidebar where the collection space is located), the collection space detects the new data and displays a card with the added data.

[0271] S31. The collection space retrieves HTML text from ClipData and collects content with mixed text and images.

[0272] As mentioned above Figure 5 As shown in Figure D, the collection space is used to collect content that combines text and images.

[0273] The HTML text carries HTML tags, which reflect the position of the images and text on the page. The collection space retrieves the HTML text from ClipData and downloads or copies the images and text. In one approach, the collection space hashes the images and text according to their original relative positions on the page, as described above. Figure 9 The card set shown is displayed in the corresponding floating area of ​​the collection space. Alternatively, the collection space styles images and text according to their original relative positions on the page into standard HTML text, generating an offline webpage file, as described above. Figure 16 The single card shown is displayed in the floating area corresponding to the collection space.

[0274] In this example, the content containing mixed text and images is saved via a bookmarking space. This bookmarking allows users to save the content as offline web pages. Images (such as pictures or videos) are downloaded or copied based on their URIs. Simultaneously, the text and images are arranged in their original relative positions to form standard HTML text, preserving the original tag styles as much as possible. Users can subsequently view the content through a browser or other applications that can view HTML text styles, facilitating easy review of the mixed text and image content, improving data intuitiveness, and enhancing the user experience.

[0275] In this example, the collection space allows users to save mixed text and images. This can be done in a hashed format, downloading or copying images (such as pictures or videos) based on their URIs. Simultaneously, text and images are grouped together using their original relative positions. This separate collection of images and text facilitates user access, improving data efficiency and user experience. Furthermore, the collection space maintains the association between images and text through card groups, enhancing the visibility of the saved data. Moreover, these card groups can display the applications corresponding to the text and image content, increasing the diversity and richness of the interface.

[0276] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer, or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0277] The above-described embodiments are optional embodiments provided by this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. A data collection method, characterized in that, When applied to electronic devices, the method includes: Display a first interface, the first interface including a first material; In response to a selection operation on the first material, first selection data is obtained, the first selection data including the first image and the first text in the first material; the first selection data is data in a mixed image and text format; In response to a first operation on the first selected data, the first selected data is stored in an instance storage unit in a rich text structure to preserve the distribution position and order of the first image and the first text on the page where the first material is located. The first operation refers to an operation with the intention of collecting. In response to a drag operation on the first selected data, when the first selected data is dragged and dropped to the area corresponding to the collection space in the first interface, the first selected data is collected in the collection space according to the rich text structure; the collection space includes images, text, the correlation between images and text, and the correlation between text.

2. The method as described in claim 1, characterized in that, The step of storing the first selected data in an instance storage unit in response to a first operation on the first selected data includes: In response to the first operation on the first selected data, if the application to which the first material belongs supports drag-and-drop of text and image content, the first selected data is stored in the instance storage unit in the rich text structure.

3. The method as described in claim 1, characterized in that, The step of storing the first selected data in an instance storage unit in response to a first operation on the first selected data includes: In response to the first operation on the first selected data, if the application to which the first material belongs does not support dragging and dropping of text and image content, and the application inherits the function of selecting a content area by cursor in the operating system of the electronic device, then the selected area corresponding to the first selected data is obtained. The resource identifier and first text of the first image in the selected area are identified by the view component to obtain identification data; The identification data is stored in the instance storage unit in a rich text structure.

4. The method as described in claim 1, characterized in that, The step of obtaining first selection data in response to a selection operation on the first material includes: In response to a selection operation on the first material, second selection data is obtained, the second selection data including any data in the first material; In response to the second operation on the second selected data, if the application to which the first material belongs does not support dragging and dropping of text and image content, and the application does not inherit the function of selecting content area by cursor in the operating system of the electronic device, then an additional view is generated above the page where the first material is located. The additional view includes the page content of the page where the first material is located and supports selecting content area by cursor. In response to a selection operation on data in the additional view, the first selected data is retrieved.

5. The method as described in claim 1, characterized in that, Before obtaining the first selection data in response to a selection operation on the first material, the method further includes: In response to a third operation on the first interface, an additional view is generated above the page where the first material is located. The additional view includes the page content of the page where the first material is located and supports selection of the content area by the cursor. In response to a selection operation on the first material, first selection data is obtained, including: In response to a selection operation on the first material in the additional view, the first selection data is obtained.

6. The method as described in claim 4 or 5, characterized in that, The step of storing the first selected data in an instance storage unit in response to a first operation on the first selected data includes: In response to the first operation on the first selected data, obtain the selected area corresponding to the first selected data; The resource identifier and first text of the first image in the selected area are identified by the view component to obtain identification data; The identification data is stored in the instance storage unit in a rich text structure.

7. The method as described in claim 4 or 5, characterized in that, The step of generating an additional view above the page containing the first material includes: The resource identifier of the image set in the page where the first material is located and the position of each image in the image set on the page are obtained through the view component; the image set includes the first image; The text recognition component is used to perform text recognition on the screenshot corresponding to the page to obtain a text set and the position of each text in the text set on the page; the text set includes the first text. Based on the image set extracted from the resource identifier of the image set, the text set, and the position of each text in the text set on the page, the additional view is generated above the page.

8. The method as described in claim 7, characterized in that, If the first image includes the second text, then the text set also includes the second text.

9. The method as described in claim 8, characterized in that, The additional view includes at least the first text, the first image, and the second text in the first image; The second or third selected data includes the first text, the first image, the second text, and / or other objects, wherein the other objects are objects on the page other than the first text, the first image, and the second text.

10. The method according to any one of claims 1-5, characterized in that, After adding the first selected data to the collection space according to the rich text structure, the method further includes: The second interface is displayed; the second interface includes a floating area corresponding to the collection space, and the floating area displays a first card group corresponding to the first selected data. The first card group includes a first card corresponding to the first image and a second card corresponding to the first text.

11. The method as described in claim 10, characterized in that, The first card group also includes a first icon and / or a first name, both of which indicate the application source identifier of the first selected data.

12. The method as described in claim 10, characterized in that, The first text includes at least one paragraph, and the first image includes at least one picture; the line spacing between any two paragraphs is greater than a preset spacing, or the first image is located between any two paragraphs; The first card group includes a card corresponding to each of the images and a card corresponding to each of the paragraphs; the cards corresponding to each of the images and the cards corresponding to each of the paragraphs are arranged in order of their position on the page where the first material is located.

13. The method as described in claim 10, characterized in that, The first card includes the first image, or a thumbnail or partial screenshot of the first image; The second card includes all or part of the characters in the first text.

14. The method as described in claim 10, characterized in that, After displaying the second interface, the method further includes: In response to a trigger operation on the first card group, the first card and the second card are displayed in the floating area.

15. The method as described in claim 14, characterized in that, After displaying the first card and the second card in the suspended area, the method further includes: In response to a drag operation on the first target card, a third interface is displayed; the third interface includes a first editing area, in which edit data corresponding to the first target card is displayed, and the first target card is any card in the first card group.

16. The method as described in claim 10, characterized in that, After displaying the second interface, the method further includes: In response to a drag operation on the second card group, a fourth interface is displayed; the fourth interface includes a second editing area, which displays the editing data corresponding to the multiple cards included in the second card group, wherein the second card group is any card group in the floating area.

17. The method as described in claim 10, characterized in that, After displaying the second interface, the method further includes: In response to the merging operation of the third and fourth card groups, a fifth card group is displayed in the floating area. The fifth card group includes the cards included in the third card group and the cards included in the fourth card group, which are any two card groups in the floating area.

18. The method according to any one of claims 1-5, characterized in that, After adding the first selected data to the collection space according to the rich text structure, the method further includes: The fifth interface is displayed; the fifth interface includes a floating area corresponding to the collection space, and a single card corresponding to the first selected data is displayed in the floating area. The single card includes an offline web page file corresponding to the first selected data, and the offline web page file is generated by the collection space according to the rich text structure.

19. The method as described in claim 18, characterized in that, After displaying the fifth interface, the following is also included: In response to a triggering operation on a second target card, at least one option is displayed; the second target card is any single card in the floating area, and the option indicates a browsing engine or an application that supports web browsing; In response to a triggering operation on a target option, the webpage corresponding to the second target card is displayed, wherein the target option is any one of at least one options.

20. The method according to any one of claims 1-5, characterized in that, The method further includes: In response to the first operation on the first selected data, the collection space is opened and displayed in a hidden state on the first interface; Alternatively, in response to a drag operation on the first selected data, the collection space is opened and displayed in a hidden state on the first interface.

21. The method as described in claim 10, characterized in that, The upper limit of the number of elements displayed in the floating area is a preset number; the elements are card groups or single cards. The method further includes: If the number of displayed area elements in the floating area reaches a preset number, then the area element with the earliest timestamp among the displayed area elements will be hidden.

22. The method as described in claim 21, characterized in that, The floating area includes a display progress bar, which is used to adjust the progress of the area elements displayed in the floating area; The method further includes: In response to the fourth operation on the display progress bar, the hidden area elements in the floating area are displayed.

23. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1-22.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-22.

25. A computer program product containing instructions, characterized in that, When it is run on a computer, it causes the computer to perform the method as described in any one of claims 1-22.

Images