Cross-application material collection method and electronic equipment

Abstract

The invention provides a cross-application material collection method and electronic equipment. The method comprises the following steps: running a first application on the electronic equipment and displaying a first interface; in response to a first operation of the user on the first interface, adding a first material in the first interface to the second application by using the first processing framework, the first operation being a three-finger glide operation in the first interface; or in response to a second operation of the user on the first interface, adding the second material in the first interface to the second application by using the second processing framework, the second operation being a long-press dragging operation on the second material selected in the first interface. In the scheme, two processing frameworks are designed, so that the purpose of cross-application collection can be achieved by calling different processing frameworks based on different user operations, the whole collection process is more flexible and diverse, various forms of collection requirements of the user can be met, and the user can conveniently collect different types of materials as required.

Description

Technical Field

[0001] This application relates to the technical field of electronic devices, and in particular, to a method and an electronic device for collecting materials across applications. Background Art

[0002] In traditional electronic devices, collections can only be scattered in different applications, which is inconvenient to search, and the collected elements can also be diverse. For example, they may include various elements such as text, pictures, videos, audios, web pages, and files. To facilitate users to manage and use these elements scattered in various applications in different forms, R & D personnel have designed a new application. In this application, users can put all the content they want to collect across applications into it for centralized management, that is, to achieve global collection.

[0003] However, since global collection involves different applications and different collection elements, the processing process will be very cumbersome and time-consuming. Therefore, how to optimize the processing process is a technical problem to be solved urgently. Summary of the Invention

[0004] This application provides a method and an electronic device for collecting materials across applications. This solution designs two processing frameworks, so that different processing frameworks can be called based on different user operations to achieve the purpose of collecting materials across applications, making the entire collection process more flexible and diverse, and facilitating users to collect different types of materials as needed.

[0005] In a first aspect, a method for collecting materials across applications is provided, which is applied to an electronic device. The electronic device includes a first processing framework and a second processing framework. The method includes: running a first application on the electronic device and displaying a first interface; in response to a first operation of the user on the first interface, using the first processing framework to add a first material in the first interface to a second application, where the first operation is a three-finger downward sliding operation in the first interface; or, in response to a second operation of the user on the first interface, using the second processing framework to add a second material in the first interface to the second application, where the second operation is a long-press and drag operation on the selected second material in the first interface.

[0006] In the technical solution of this application, two processing frameworks are designed, so that different processing frameworks can be called based on different user operations to achieve the purpose of collecting materials across applications, making the entire collection process more flexible and diverse, capable of meeting the collection needs of users in various forms, and also facilitating users to collect different types of materials as needed.

[0007] In combination with the first aspect, in some implementations of the first aspect, when adding the first material in the first interface to the second application in response to a first operation of the user on the first interface by using the first processing framework, it may include: calling an interface of the Activity Management Service (AMS) in the first processing framework to obtain a screenshot of the first interface; calling the global collection middle platform of the first processing framework to determine the view type included in the screenshot of the first interface; displaying a second interface on the electronic device, where the second interface includes the screenshot of the first interface and at least one control, and the at least one control is determined according to the view type included in the screenshot of the first interface; in response to a click operation of the user on the first control among the at least one control, adding the first material to the second application in the collection form corresponding to the first control by using the first processing framework. In this implementation, cross-application collection is triggered by a three-finger swipe-down operation. It is necessary to first obtain a screenshot of the interface, determine the view type therein, then provide the user with corresponding controls to indicate in what form to collect the first material, and then collect it in the collection manner indicated by the clicked control according to the way the user clicks the control.

[0008] In combination with the first aspect, in certain implementations of the first aspect, the first control is a content extraction control. When, in response to a user's click operation on the first control among at least one control, the first material is added to the second application in the favorite form corresponding to the first control by using the first processing framework, the following steps may be included: traversing and filtering all views in the screenshot of the first interface by using the global favorite middle platform to obtain a first subset of sub-views, where the first subset of sub-views includes at least one sub-view capable of extracting content; using the global favorite middle platform to extract content for each sub-view in the first subset of sub-views according to the view type of the sub-view to obtain the extracted content corresponding to the sub-view; when the content extraction of all sub-views in the first subset of sub-views is completed, displaying a third interface on the electronic device, where the third interface includes the extracted content of the first subset of sub-views; in response to a user's selection operation on the target content in the extracted content of the first subset of sub-views in the third interface, determining the selected target content as the first material; and in response to a user's click operation on the confirmation favorite control in the third interface, adding the first material to the second application by using the global favorite middle platform. In this implementation, the user clicks on the content extraction control, triggering the extraction of the content in the entire screenshot. First, all sub-views capable of extracting content in the screenshot are traversed and filtered, then the content of each sub-view is extracted according to the type of each sub-view, and then all the extracted content is displayed in the third interface for the user to select, and the user's selection is used for favoriting. It should be understood that in this implementation, since it is not known whether the user hopes to store the entire page content or only a part of it, nor which part the user wants to store, all the content of all sub-views in the entire screenshot is first extracted and set to a selectable state, and then it is convenient for the user to select which part they want to favorite and store.

[0009] In combination with the first aspect, in some implementations of the first aspect, when traversing and filtering all views within the extractable range using the global collection middleware platform to obtain the first subset of views, the following operations may be included: during traversal, for each view in the screenshot of the first interface, perform at least one of the following operations: when the view is a standard view, obtain the view type of the view using the acquisition method of the standard view; or, when the view is a custom view, determine the view type of the view using the suffix of the view; and, read the size and / or position of the view; during filtering, perform at least one of the following operations: only retain the views visible to the user in the screenshot of the first interface, remove the hidden views in the screenshot of the first interface, and, remove the views partially or completely blocked in the screenshot of the first interface. In this implementation, some or all of the information such as the type, size, and position of the view will be obtained during traversal. When obtaining the view type, different acquisition methods will be adopted according to whether the view is a standard view or a custom view. In this implementation, the views will also be filtered to a certain extent. In addition to the views visible to the user in an interface, there may also be hidden views or other views that are not displayed in the interface but belong to the interface. Therefore, only the views visible to the user can be retained, or the hidden views can be removed, and the two effects are equivalent. In addition, the views partially or completely blocked can also be removed.

[0010] In combination with the first aspect, in some implementations of the first aspect, the first subset of views includes sub-views of at least one view type among text, pictures, offline web pages, files, videos, and links; when using the global collection middleware platform to extract content for each sub-view in the first subset of views according to the view type of the sub-view, it may include: for each sub-view, when the view type of the sub-view is text, call the text extraction module in the global collection middleware platform to obtain the text data of the sub-view, and determine the obtained text data as the extracted content of the sub-view; or, for each sub-view, when the view type of the sub-view is a picture, call the picture extraction engine in the global collection middleware platform to obtain the bitmap of the picture from the sub-view, and determine the bitmap as the extracted content of the sub-view, and / or call the text processing module in the global collection middleware platform to obtain the text data contained in the picture by means of optical character recognition (OCR); or, for each sub-view, when the view type of the sub-view is an offline web page, call the web page extraction engine in the global collection middleware platform to perform title extraction and content parsing on the web page to obtain the extracted content of the sub-view, and the extracted content of the sub-view includes the extracted title and the parsed web page content; or, for each sub-view, when the view type of the sub-view is a file, call the file extraction engine in the global collection middleware platform to obtain the file path of the file, and determine the copied file of the file or the file path of the file as the extracted content of the sub-view, and determine the file information of the file as the extracted content of the sub-view; or, for each sub-view, when the view type of the sub-view is a video, call the video extraction engine in the global collection middleware platform to identify the video playback area by reading the video identifier (ID) written by the first application to the accessibility node, and perform screen recording, screenshotting, and / or adding tags to obtain the extracted content of the sub-view; or, for each sub-view, when the view type of the sub-view is a link, call the online link extraction engine in the global collection middleware platform to obtain link information according to the uniform resource locator (URL) field carried in the activity or the way of getting the URL of the web view, and the obtained link information is the extracted content of the sub-view. In this implementation, there are different content extraction strategies for different types of views, enabling the solution of the present application to support the content extraction of multiple views simultaneously, rather than just a single view.

[0011] In connection with the first aspect, in some implementations of the first aspect, for each sub-view, when the view type of the sub-view is text, when calling the text extraction module in the global collection middleware platform to obtain the text data of the sub-view and determining the obtained text data as the content after extraction of the sub-view, it may include: using the text extraction module to call the intelligent recognition platform, so that the intelligent recognition platform obtains the text data of the sub-view by means of optical character recognition (OCR), or, using the text extraction module to obtain the text data of the sub-view by reading the information written by the first application into the accessibility node. That is to say, for sub-views of the text type, there are two ways to extract content, OCR recognition or reading from the accessibility node.

[0012] In connection with the first aspect, in some implementations of the first aspect, in response to the user's click operation on the confirmation collection control in the third interface, when using the global collection middleware platform to add the first material to the second application, it may include at least one of the following operations: directly adding the text material in the first material to the second application; storing the picture material in the first material in the gallery application and adding the thumbnail of the picture to the second application, where the picture stored in the gallery application is associated with the thumbnail of the picture in the second application, and the thumbnail of the picture is the mapping element of the picture in the second application; storing the offline web page material in the first material in the note application and adding the introduction content of the offline web page to the second application, where the offline web page stored in the note application is associated with the introduction content of the offline web page in the second application, and the introduction content of the offline web page is the mapping element of the offline web page in the second application; storing the video material in the first material in the gallery application and adding the thumbnail of the first frame image of the video to the second application, where the video stored in the gallery application is associated with the thumbnail of the first frame image of the video in the second application, and the thumbnail of the first frame image of the video is the mapping element of the video in the second application; storing the file material in the first material in the file management application and adding the file information of the file to the second application, where the file stored in the second application is associated with the file information of the file in the second application, and the file information of the file is the mapping element of the file in the second application, and the file information includes the icon of the file type and the file name; and, directly adding the link material in the first material to the second application. In this implementation, it is described that for different view types of materials (different types of materials) in the first material, the collection will be completed according to different addition schemes.

[0013] In combination with the first aspect, in some implementations of the first aspect, the first control is a web page favorite control, a link favorite control, or a file favorite control. When, in response to a user's click operation on the first control among at least one control, the first material is added to the second application in the favorite form corresponding to the first control by using the first processing framework, it may include: when the first control is a web page favorite control, adding the first material to the second application in the form of an offline web page, where the first material is the content of the entire web page displayed in the first interface; or, when the first control is a web page favorite control, adding the first material to the second application in the form of an offline web page, where the first material is the content displayed on the interface of the web page displayed in the first interface; or, when the first control is a link favorite control, adding the first material to the second application in the form of a link, where the first material is the online link of the web page displayed in the first interface; or, when the first control is a file favorite control, adding the first material to the second application in the form of a file, where the first material is the file displayed in the first interface, and the file is an open document, a video displayed in full screen, or a picture displayed in full screen. In this implementation, examples of other favorite forms are given, enriching the form of favorite content and facilitating users to make favorites according to different needs. That is to say, users can make favorites of the content they want to favorite in different ways according to different favorite needs.

[0014] In combination with the first aspect, in some implementations of the first aspect, when calling the global favorite middle platform of the first processing framework to determine the view types included in the screenshot of the first interface, it may include: determining the view types included in the screenshot of the first interface according to the whitelist configuration of the first application or the whitelist configuration of the first interface. In this implementation, determining the view types included in the screenshot according to the whitelist configuration, the introduction of the whitelist configuration can help determine the extractable range in the interface, making the extracted content have customized constraint conditions, and different whitelists can be set for different applications or different interfaces as needed.

[0015] In combination with the first aspect, in certain implementations of the first aspect, when a second interface is displayed on an electronic device and the second interface includes a screenshot of the first interface and at least one control, it may include: when the view types included in the screenshot of the first interface include web pages, at least one of the controls includes at least one of a web page favorites control, a link favorites control, and a content extraction control; or, when the view types included in the screenshot of the first interface include files, at least one of the controls includes at least one of a file favorites control and a content extraction control; or, when the view types included in the screenshot of the first interface only include pictures or videos, at least one of the controls includes at least one of a file favorites control and a content extraction control. In this implementation, the type of control displayed on the second interface is determined based on the different view types included in the screenshot, which facilitates the user to select according to their needs. For example, assuming the first interface does not contain a web view, there is no need to display a web page favorites control on the second interface.

[0016] In combination with the first aspect, in certain implementations of the first aspect, when, in response to a second operation of the user on the first interface, adding a second material in the first interface to a second application using a second processing framework, it may include: using the content recognition service in the second processing framework to obtain the second material selected by the user in the first interface; using the content recognition service, based on the long-press coordinates of the second operation, to determine the view type included in the second material; using the view recognizer in the second processing framework to traverse and search the view tree to obtain the sub-views of the second material that contain extractable content, obtaining a second sub-view set, and the second sub-view set includes at least one sub-view with extractable content; using the content recognition service, for each sub-view in the second sub-view set, performing content extraction according to the view type of the sub-view to obtain the extracted content corresponding to the sub-view; when all the sub-views in the second sub-view set are extracted, obtaining the extracted content of the second sub-view set; using the window management service (WMS) in the second processing framework, based on the view type included in the extracted content of the second sub-view set, adding the extracted content of the second sub-view set to the second application. In this implementation, it is a global favorite triggered under a drag operation. Since dragging requires first selecting an object and then dragging, and the starting coordinates of the drag (the coordinates at the long press) can be obtained, the sub-views of the selected content are determined by traversing and searching all sub-views using the view tree, rather than identifying all sub-views in the entire interface. This implementation first allows the user to select the object (second material) they want to favorite, and then performs view recognition and content extraction in the second material, as well as favoriting the extracted content.

[0017] In combination with the first aspect, in some implementations of the first aspect, when using the window management service in the second processing framework to add the extracted content of the second subset of views to the second application according to the view types included in the extracted content of the second subset of views, the following operations may be included: directly adding the text content in the extracted content of the second subset of views to the second application; storing the image content in the extracted content of the second subset of views in the gallery application and adding the thumbnail of the image to the second application, where the image stored in the gallery application is associated with the thumbnail of the image in the second application, and the thumbnail of the image is a mapped element of the image in the second application; storing the offline web page content in the extracted content of the second subset of views in the note application and adding the brief introduction content of the offline web page to the second application, where the offline web page stored in the note application is associated with the brief introduction content of the offline web page in the second application, and the brief introduction content of the offline web page is a mapped element of the offline web page in the second application; storing the video content in the extracted content of the second subset of views in the gallery application and adding the thumbnail of the first frame image of the video to the second application, where the video stored in the gallery application is associated with the thumbnail of the first frame image of the video in the second application, and the thumbnail of the first frame image of the video is a mapped element of the video in the second application; storing the file content in the extracted content of the second subset of views in the file management application and adding the file information of the file to the second application, where the file stored in the second application is associated with the file information of the file in the second application, and the file information of the file is a mapped element of the file in the second application, and the file information includes the icon of the file type and the file name; and directly adding the link content in the extracted content of the second subset of views to the second application.

[0018] In combination with the first aspect, in some implementations of the first aspect, after adding the first material or the second material to the second application, the above method further includes: displaying a floating capsule (snackbar) on the electronic device, where the snackbar includes a prompt indicating successful collection and a viewing control; in response to a click operation of the user on the viewing control, starting the second application and displaying the addition result in the interface of the second application. In this implementation, the execution process of prompting the user that the collection is successful after successful collection and facilitating the user to view the collection result is given.

[0019] In a second aspect, an apparatus for cross-application collection of materials is provided, and the apparatus includes units composed of software and / or hardware for executing any one of the methods in the first aspect.

[0020] In a third aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it can implement any one of the methods in the first aspect.

[0021] In a fourth aspect, a chip is provided, including a processor configured to read and execute a computer program stored in a memory. When the computer program is executed by the processor, any method of the first aspect can be implemented.

[0022] Optionally, the chip further includes a memory electrically connected to the processor.

[0023] Optionally, the chip may further include a communication interface.

[0024] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, any method of the first aspect can be implemented.

[0025] In a sixth aspect, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, any method of the first aspect can be implemented. Description of the Drawings

[0026] Figure 1 is a schematic diagram of an interaction process for cross-application collection of materials in an embodiment of the present application.

[0027] Figure 2 and Figure 3 is a schematic diagram of another interaction process for cross-application collection of materials in an embodiment of the present application.

[0028] Figure 4 is a schematic diagram of another interaction process for cross-application collection of materials in an embodiment of the present application.

[0029] Figure 5 is a schematic diagram of another interaction process for cross-application collection of materials in an embodiment of the present application.

[0030] Figure 6 is a schematic diagram of a processing framework for cross-application collection of materials in an embodiment of the present application.

[0031] Figure 7 is a schematic diagram of another processing framework for cross-application collection of materials in an embodiment of the present application.

[0032] Figure 8 is a schematic flowchart of a method for cross-application collection of materials in an embodiment of the present application.

[0033] Figure 9 is a schematic diagram of a device for cross-application collection of materials in an embodiment of the present application.

[0034] Figure 10 is a schematic diagram of the hardware structure of an electronic device in an embodiment of the present application. Detailed implementation manners

[0035] The solutions of the embodiments of the present application will be introduced below with reference to the accompanying drawings.

[0036] Figure 1 It is a schematic diagram of an interaction process for collecting materials across applications in an embodiment of the present application. As Figure 1 shown, assume that the user opens Application A on an electronic device and enters the interface shown in Interface 101, which includes some graphic and text data of a news item. Assume that the user selects a part of the content by pressing and sliding in Interface 101. As shown in Interface 102, a part of the text content and a picture content are selected. Assume that the user raises the hand after selecting the content and then long-presses on the selected content area. In response to this long-press operation, the electronic device displays Interface 103, in which the selected content is superimposed in a floating form on the interface. Assume that the user drags the floating selected content to the side of the electronic device without releasing the long-press in Interface 103, and the selected content can be added to the collection space application. As shown in Interface 104, the display interface of the collection space application is superimposed on the interface of Application A. Here, it is a long-strip interface, and it can be seen from the display interface of the collection space that the selected content has been added to the collection space, and the text and the picture are stored as a material respectively.

[0037] It should also be understood that Figure 1 taking Application A as an example, the page content in Application A is added to the collection space for collection. For other applications, the page content therein can also be added to the collection space, which will not be elaborated. For example, the chat content in the chat application can be selected and dragged and then added to the collection space. Another example is that the mutually transmitted files in the chat application can be dragged to the collection space. Another example is that the attachments in the email application can be dragged to the collection space. Another example is that the email body content of the email application can be selected and dragged to the collection space. Another example is that the pictures opened in the shopping application can be dragged to the collection space. Another example is that the pictures and videos in the gallery application can be dragged to the collection space. Other situations will not be listed one by one.

[0038] Figure 2 and Figure 3 It is a schematic diagram of another interaction process for collecting materials across applications in an embodiment of the present application. As Figure 2As shown, assume that the user has opened Application A on the electronic device and entered the interface shown in Interface 104, which includes some graphic and text data of a news item. Assume that the user performs an operation of three-finger long press followed by three-finger swipe down in Interface 104. In response to this three-finger swipe down operation, the electronic device displays Interface 105, which includes, in addition to the interface content of Interface 104, three controls: web page collection, content extraction, and screenshot collection. It should be understood that in practice, the number, type, and form of the controls included in Interface 105 can be set as needed. It should also be understood that as the content included in Interface 104 varies, the number and type of controls in Interface 105 can also be different. For example, here Interface 104 is a web page, so Interface 105 includes web page collection and may also include link collection. Assume that what is displayed in Interface 104 is an open file. Taking a pdf document as an example, assume that an open pdf is displayed in Interface 104, then Interface 105 can include at least one of file collection, content extraction, and screenshot collection. Assume again that a picture is full-screen displayed in Interface 104. Then Interface 105 can include picture collection (equivalent to file collection) and content extraction (here specifically referring to the extraction of the content in the picture). Other situations will not be listed one by one.

[0039] Assume that the user clicks the content extraction control in Interface 105. In response to this click operation, the electronic device displays Interface 106, in which the background color of the selectable content and the non-selectable content in the entire interface are different. Here, take the non-selectable background color as the gray background color and the selectable one as the white background color as an example. Interface 105 also includes two controls: exit and collection. Assume that the user selects a part of the selectable content in Interface 105 in a way of holding and sliding, as shown in Interface 107. It can be seen from Interface 107 that here, take the selection of a piece of text content (Text Content #1) and two pictures (Picture Content #1 and Picture Content #2) as an example. The background color of the selected content is different from the background color of the unselected content and the non-selectable content. That is to say, in Interface 107, the selected content is marked by the background color. However, it should be understood that in practice, other methods can also be used to mark the selected content. For example, the color of the selected target content can be different from the color of other selectable content and also different from the color of other non-selectable content.

[0040] As Figure 3As shown, assume that the user raises their hand to end the selection operation after holding and sliding on interface 107, clicks the favorite control. In response to this click operation, the electronic device adds the selected content to the favorite space. After the addition is complete, interface 108 is displayed. Interface 108 includes floating capsule (snackbar) #1. Floating capsule #1 includes a prompt for successful favoriting and a click-to-view control for indicating to view the favorite result. Assume that the user clicks the "click to view" control in interface 108. In response to this click operation, the electronic device displays interface 109. As shown in interface 109, the display interface of the favorite space application is superimposed on the interface of application A. Here, the favorite space application is a long strip interface, and it can be seen from the display interface of the favorite space that the selected content has been added to the favorite space, and the text and pictures are stored as one material respectively. The selected piece of text and two pictures shown in interface 107 have all been favorited to the favorite space.

[0041] It should also be understood that Figure 2 and Figure 3 take application A as an example. After being triggered by a three-finger downward swipe, the page content in application A is added to the favorite space for favoriting. For other applications, the page content therein can also be added to the favorite space, which will not be elaborated further. For example, a three-finger downward swipe can be performed in the chat interface of a chat application, and after selecting content extraction and then selecting the target content, the target content can be favorited to the favorite space by clicking the favorite control. Another example is that a three-finger downward swipe can be performed in the display interface of a mail application, and then select the material to add to the favorite space. The interaction process is similar to Figure 2 and Figure 3 and will not be elaborated further. Another example is that a three-finger downward swipe can be performed in a shopping application, and then select materials such as pictures to add to the favorite space. Another example is that a three-finger downward swipe can be performed in the display interface of a gallery application to add all the pictures and / or videos in the current interface to the favorite space. Other situations will not be listed one by one.

[0042] Figure 4 is a schematic diagram of another interaction process for cross-application favorite materials in an embodiment of the present application. Figure 4 It can be regarded as being in Figure 2The interactive operation performed after the interface 105 is displayed is shown. Assuming that the user clicks on the webpage collection in the interface 105, in response to the click operation, the electronic device displays the interface 110, and the interface 110 includes the floating capsule #2. Assuming that the user clicks on "Click to view" in the floating capsule #2, in response to the click operation, the electronic device displays the interface 111. It can be seen in the interface 111 that the material added to the collection space is collected in the form of a webpage, that is, the offline webpage material. Comparing the interface 111 with the interface 109, it can be seen that when the webpage is collected, the valid content displayed in the entire interface 101, or the valid content displayed in the entire interface 105, is added to the collection space, and is used as a group of materials in the collection space, rather than storing text and pictures separately.

[0043] It should also be understood that webpage collection means collecting the content shown in the interface 105 in the form of a webpage, rather than collecting all the contents of the entire webpage involved in the interface 105. However, webpage collection may also mean collecting all the contents of the entire webpage involved in the interface 105 in the form of a webpage, rather than being limited to the contents visible to the user through the screen in the interface 105. The above two methods can be preset according to the actual needs of the electronic device.

[0044] In addition, assuming that the user clicks on the screenshot collection in the interface 105, the screenshot obtained after the screenshot of the interface 105 can also be collected in the collection space.

[0045] Figure 5 It is a schematic diagram of another interactive process of collecting materials across applications according to an embodiment of the present application. Figure 5 It can be seen as Figure 1 The interactive process performed after the interface 103 shown is displayed. Assuming that the user presses and holds the floating selected content in the interface 103 and drags it to the side of the electronic device, the electronic device follows the hand to display the door opening animation shown in the interface 112, and the portal application includes the icon of the collection space application. Assuming that the user continues to drag to the collection space icon and then raises his hand, the electronic device responds to the operation to perform the step of adding the selected content to the collection space, and displays the interface 113 after adding it. In the interface 113, the user is prompted that the collection has been successfully collected through the floating capsule #3. Assuming that the user clicks "Click to view" in the floating capsule #3, in response to the click operation, the electronic device displays the interface 114. As shown in the interface 114, the display interface of the collection space application is superimposed on the interface of application A. Here, the collection space application is a long strip interface, and it can be seen from the display interface of the collection space that the selected content has been added to the collection space, and the text and the picture are stored as a material respectively. The selected text and the picture shown in the interface 103 are both collected in the collection space, the text as one material, and the picture as another material.

[0046] It should be understood that Figures 1 - 5 only an example of the interaction process for cross - application collection is given. In practice, the specific content of the interface, the included controls, and the number of controls can all vary. As can be seen from Figures 1 - 5 the process of cross - application collecting materials can collect different contents of different applications into the collection space, that is, to achieve global collection, which is convenient for users to view and use the collected materials in the collection space later. And in the above - mentioned interaction process, different materials will be added to the collection space in different ways under different trigger operations. Therefore, the electronic device needs to have the ability to extract the content of different types of materials. For the above - mentioned problems, the cross - application material collection solution provided in this application will extract the content according to different types of views respectively, so as to achieve the content extraction of the above - mentioned different materials.

[0047] Figure 6 is a schematic diagram of a processing framework for cross - application material collection in an embodiment of this application. As Figure 6 shown, the InputManager is used to obtain the three - finger press operation (for pulling up the global collection service) and the three - finger swipe - down operation (for triggering the global collection service). The HwFingersSnapshooter module in the input processor is used to recognize the gesture operations of three - finger press and three - finger swipe - down.

[0048] The input processor sends the various operations obtained to the global collection service (global collection middleware). The global collection middleware includes a CollectionService module, a SensorManager, a gesture module, and extraction engines / modules for various views. For example, it includes a TexExtractClient module, a PictureExtractEngine, a FileExtractEngine, a VideoExtractEngine, an OnlinelinkExtractEngine, and a WebPageExtractEngine. That is to say, each view corresponds to an extraction engine / extraction module. Therefore, the number of extraction engines / extraction modules included in the global collection middleware can be set according to the different types of views to be extracted.

[0049] Each extraction engine / module in the global collection middleware extracts the view under the trigger of the gesture module, and starts and schedules the corresponding processes through the activity manager service (AMS) to perform subsequent operations.

[0050] The global collection middleware can request a node (requestContentNode) from AMS. AMS feeds back the node according to the request and passes the activity instance, so as to send the screenshot of the interface during the three-finger swipe-down operation to the global collection middleware.

[0051] After the global collection middleware extracts the content to be collected (which may be the entire web page, file, etc., or a small part of the content in the first interface), according to the type of the view, it stores different types of materials in the file manager, gallery, notes or collection space respectively, and stores mapping elements such as thumbnails in the collection space. The collection space application can display the content of the collected content, organize and search, preview / share, and can share the collected content with other electronic devices through multi-device sharing. When it is necessary to extract the text in the picture, the global collection platform sends the initial data, that is, the data to be recognized, to the intelligent recognition platform AIPlugin and receives the recognition result. The intelligent recognition platform includes VisionService, TextConfiguration, and OcrDetectEngine. VisionService is used to recognize the views in the interface, TextConfiguration is used to recognize text objects, and OcrDetectEngine is used to recognize the text in the picture, that is, to recognize the text in the picture according to the glyphs. Since the global collection middleware receives the interface screenshot sent by AMS, the text in it, including the text in the picture in the screenshot, can be recognized through OCR.

[0052] The global collection platform reads the whitelist configuration from the cloud storage or other storage modules or memories to guide the collection process.

[0053] Combined with Figure 2 and Figure 3, when the user performs a three - finger press operation in interface 104, this press operation will be captured by the input processor and transmitted to the global collection service middleware. Then the global collection service middleware launches the global collection service. When the user performs a three - finger swipe - down operation based on the three - finger press operation, this three - finger swipe - down operation will be captured by the input processor and transmitted to the global collection service middleware. Then the collection service module of the global collection service middleware collects the content in the interface, requests a node from the AMS, and displays interface 105 in the form of an activity instance. When the user clicks on content extraction in interface 105, the input processor captures this click operation and transmits it to the global collection service middleware. The various extraction engines / extraction modules and the intelligent recognition platform in the global service middleware are linked, and combined with the received whitelist configuration, corresponding engines / modules are called according to the types of each view in interface 105 to perform content extraction, so as to determine the display result shown in interface 106. When the user performs an operation of holding and sliding to select content in interface 106, the input processor captures this operation and transmits it to the global collection service middleware. The global collection service middleware marks the selected content. When the user clicks on the favorite control in interface 107, the input processor captures this click operation and sends it to the global collection service middleware. The global service middleware stores the element (material) at the corresponding position according to the view type corresponding to each element in the selected content, and adds it to the favorite space. Taking the text view as an example, the global collection service middleware directly sends the text material to the favorite space for display and storage. Taking the picture view as an example, the global collection service platform stores the picture in the gallery, sends the thumbnail of the picture to the favorite space for storage, and associates the two. Other situations will not be listed one by one. For example, an offline web page can be stored in a note, and only the thumbnail is displayed in the favorite space; a file can be stored in the file manager, and only the file information is displayed in the favorite space, etc.

[0054] Figure 7 It is a schematic diagram of another processing framework for cross - application favorite materials in an embodiment of the present application. Figure 6 and Figure 7 It can be understood as introducing the execution process of the solution of the embodiment of the present application from the perspective of the internal execution process of the electronic device. Figure 6 It is an example of the first processing framework. Figure 7 It is an example of the second processing framework.

[0055] In the application process, there is a view tree (viewtree) and a view recognizer (viewrecognizer). The viewrecognizer includes local strategies (localstrategies), recognizers, and stubs. The viewtree includes the tree-like relationship between all subclass views and parent views. The root refers to the root view. Below the root view, there are text views (textview), image views (imageview), and other subviews, such as file views, web page views, etc., which will not be listed one by one here. Below the subviews, there can also be custom views, which will not be elaborated here. These subviews can be recognized through the recognizers module in the viewrecognizer. This recognition can include recognizing the type, size, and whether it is occluded of the view. The localstrategies, after determining the view type of the subview, send the corresponding processing strategy of the subview to the Window Manager Service (WindowManagerService, WMS) in the native system service framework (SystemServer) to guide how to display different views when showing in the collection space. The Stubs are used to store relevant data. The viewrecognizer can also send the information such as the type of the recognized subview and the subview to the content recognition service (contentrecognizerserver) in the self-developed system service framework (HnSystemServer)

[0056] The recognition management in HnSystemServer triggers the content recognition service for the collection space application and the Any Door application. The policy management in HnSystemServer is used to manage the processing policies in viewrecognizer. The dump provider is used to determine the storage location and mapping elements of the extracted content for each sub-view according to the view type. The configuration management in HnSystemServer is used to determine the extractable content during content extraction according to the received whitelist configuration and / or blacklist configuration. The whitelist is used to customize the custom controls in the application. For example, it can be seen from interface 106 that the follow control in application A and the like are non-extractable content, but they can be changed to extractable content through the whitelist. It can be understood that the whitelist configuration is used to determine which interfaces can be extracted, and / or, determine which views in the interface can be extracted. The blacklist configuration is used to determine which interfaces cannot be extracted, and / or, determine which views in the interface cannot be extracted. The basis for non-extraction can be set as required, that is, the blacklist configuration can be determined according to actual needs.

[0057] The file drag and drop module in WMS is used to implement the transfer of files. For example, text views will be stored in the collection space, picture views will be stored in the gallery, and files will be stored in the file manager. Other cases will not be listed one by one.

[0058] Figure 7 It mainly introduces the execution process under the drag and drop operation, that is Figure 1 and Figure 5 An example of the internal execution process of the shown collection method. Since the pressing position (coordinates) can be obtained during long-press drag and drop, and the sub-views in the selectable content can be traversed and found using the view tree.

[0059] Figure 8 It is a schematic flowchart of a method for cross-application collection of materials in an embodiment of the present application. Next, Figure 8 each step shown will be introduced.

[0060] S801. Run the first application on the electronic device and display the first interface.

[0061] The electronic device includes a first processing framework and a second processing framework. Figure 6 and Figure 7 Examples of the first processing framework and the second processing framework are given respectively.

[0062] In the embodiments of the present application, it is mainly to implement collecting the content in the first application into the second application. The first application can be different applications, and the second application is an application that can perform global collection. The so-called global collection means that the content of many applications in the entire electronic device can be collected together, that is, collected into the second application. Application A in the above text is an example of the first application, but it should be understood that the first application can also be other applications such as chat applications, email applications, shopping applications, browser applications, etc., and will not be listed one by one. Both interface 101 and interface 104 can be regarded as an example of the first interface.

[0063] The collection space mentioned above can be regarded as an example of the second application.

[0064] S802. In response to a first operation of the user on the first interface, use the first processing framework to add the first material in the first interface to the second application. The first operation is a three-finger swipe-down operation on the first interface; or, in response to a second operation of the user on the first interface, use the second processing framework to add the second material in the first interface to the second application. The second operation is a long-press and drag operation on the selected second material in the first interface.

[0065] The first material can correspond to part or all of the content in the first interface.

[0066] In the embodiments of the present application, there will be differences in the subsequent processing flows of these two operations, namely the long-press and drag operation and the three-finger swipe-down operation, and different processing frameworks will be called for different processing. Figure 1 The drag operation in interface 103 can be regarded as an example of the second operation. Figure 2 The three-finger swipe-down operation in interface 104 can be regarded as an example of the first operation.

[0067] In one implementation, step S802 may include: calling the interface of AMS in the first processing framework to obtain a screenshot of the first interface; calling the global collection middleware platform of the first processing framework to determine the view type included in the screenshot of the first interface; displaying a second interface on the electronic device, and the second interface includes the screenshot of the first interface and at least one control, and the at least one control is determined according to the view type included in the screenshot of the first interface; in response to a click operation of the user on the first control among the at least one control, use the first processing framework to add the first material to the second application in the collection form corresponding to the first control. In this implementation, cross-application collection is triggered by the three-finger swipe-down operation. It is necessary to first obtain a screenshot of the interface, determine the view type therein, then provide the corresponding control for the user to indicate in what form to collect the first material, and then collect it according to the collection method indicated by the clicked control according to the user's way of clicking the control.

[0068] The interface 105 can be regarded as an example of the second interface. The "content extraction", "webpage collection", and "screenshot collection" controls in the interface 105 can be regarded as examples of at least one control.

[0069] In one implementation, when the first control is a content extraction control, when, in response to a user's click operation on the first control in at least one control, adding the first material to the second application in the collection form corresponding to the first control by using the first processing framework, it may include: traversing and filtering all views in the screenshot of the first interface by using the global collection middle platform to obtain a first subset of views, where the first subset of views includes at least one sub-view capable of extracting content; using the global collection middle platform to perform content extraction for each sub-view in the first subset of views according to the view type of the sub-view to obtain the extracted content corresponding to the sub-view; when all sub-views in the first subset of views are extracted, displaying a third interface on the electronic device, where the third interface includes the extracted content of the first subset of views; in response to a user's selection operation on the target content in the extracted content of the first subset of views in the third interface, determining the selected target content as the first material; and in response to a user's click operation on the confirmation collection control in the third interface, adding the first material to the second application by using the global collection middle platform. In this implementation, the user clicks the content extraction control, triggering the extraction of the content in the entire screenshot. First, all sub-views capable of extracting content in the screenshot are traversed and filtered, then the content of each sub-view is extracted according to the type of each sub-view, and then all the extracted content is displayed in the third interface for the user to select and collect based on the user's selection. It should be understood that in this implementation, since it is not known whether the user hopes to store the entire page content or only a part of it, nor which part the user wants to store, all the content of all sub-views in the entire screenshot is first extracted and set to a selectable state, and then it is convenient for the user to select which part to collect and store.

[0070] In an example, in the above implementation, during traversal, for each view in the screenshot of the first interface, perform at least one of the following operations: when the view is a standard view, obtain the view type of the view by using the acquisition method of the standard view, or, when the view is a custom view, determine the view type of the view by using the suffix of the view; and read the size and / or position of the view. In this example, some or all of the information of the view type, size, and position will be obtained during traversal, and different acquisition methods will be adopted respectively according to whether the view is a standard view or a custom view when obtaining the view type.

[0071] In another example, in the above implementation, during filtering, at least one of the following operations can be performed: only retain the views visible to the user in the screenshot of the first interface, remove the hidden views from the screenshot of the first interface, and remove the views that are partially or completely blocked in the screenshot of the first interface. In this example, certain filtering of views will be performed. In addition to the views visible to the user in an interface, there may also be hidden views or other views that belong to the interface but are not displayed in the interface. Therefore, it is possible to only retain the views visible to the user, or remove the hidden views, and the two effects are equivalent. In addition, views that are partially or completely blocked can also be removed.

[0072] In this example, when removing the views that are partially or completely blocked, it is further possible to determine whether to remove the view based on the proportion of the blocked part. That is, for the blocked views, when the proportion of the blocked part of the view in the entire view is greater than or equal to the preset proportion threshold, the blocked view is removed. This method can facilitate the content extraction of views that are blocked but not severely blocked and have a lot of effective content.

[0073] In the above example, different identification methods are given for different views. For standard views, it is the view definition method standardized in the field, so the types of these views can be obtained according to the standardized acquisition method in the field. For the views customized by individual manufacturers, the view type can be determined according to the suffix set during customization. For example, for the suffix of textview, it can be determined that it is a text-type view, and for the suffix of imageview, it can be determined that it is a picture-type view, etc., and will not be listed one by one. In addition, due to the limitation of the screen size, some views cannot be presented on the interface. For example, in interface 104, the news content may be very long and not fully displayed on this page. Then, when identifying views, only the views visible to the user, that is, the views within the interface, need to be identified. Another example is that at the boundary of the screen, some views may be partially blocked. In this case, these views also need to be filtered out. For example, assume that there is a picture that is only half shown at the bottom of interface 104. The view corresponding to this picture is a view that is partially blocked.

[0074] Both interface 106 and interface 107 can be regarded as an example of the third interface. The white-background content in interface 106 can be regarded as an example of the content (selectable content) extracted from the above-mentioned first subset of views. The hold-and-slide selection operation in interface 106 can be regarded as an example of the selection operation for the target content. The content with a dark gray background in interface 107 can be regarded as an example of the target content.

[0075] As can be seen from the above, during the collection process triggered by the three-finger swipe-down operation, after determining the content to be collected, the user only needs to click the confirm collection control in the third interface to add the content to be collected to the second application.

[0076] The "Collect" controls in interface 106 and interface 107 can be regarded as examples of the above-mentioned confirm collection controls. Figure 3 The click operation on the "Collect" control in interface 107 can be regarded as an example of the click operation on the confirm collection control in the third interface above.

[0077] In another example, during traversal, for each view, first determine whether the view is a standard view; when the view is a standard view, obtain the view type of the view using the acquisition method of the standard view; when the view is not a standard view, determine whether the webview is a custom view; when the view is a custom view, determine the view type of the view using the suffix of the view; when the view is neither a standard view nor a custom view, do not add the view to the first subset of views. In this example, it is equivalent to setting a judgment priority, first checking whether it is a standard view, and if not, then checking whether it is a custom view, and discarding the view if neither is the case. This can make the recognition of views more refined.

[0078] In one implementation, the first subset of views includes sub-views of at least one view type among text, pictures, offline web pages, files, videos, and links; when using the global collection middleware platform to extract content according to the view type of each sub-view in the first subset of views to obtain the extracted content corresponding to the sub-view, it may include: for each sub-view, when the view type of the sub-view is text, call the text extraction module in the global collection middleware platform to obtain the text data of the sub-view, and determine the obtained text data as the extracted content of the sub-view; or, for each sub-view, when the view type of the sub-view is a picture, call the picture extraction engine in the global collection middleware platform to obtain the bitmap of the picture from the sub-view, and determine the bitmap as the extracted content of the sub-view, and / or, call the text processing module in the global collection middleware platform to obtain the text data contained in the picture by means of optical character recognition (OCR) (the execution area of OCR corresponds to the picture); or, for each sub-view, when the view type of the sub-view is an offline web page, call the web page extraction engine in the global collection middleware platform to perform title extraction and content parsing on the web page to obtain the extracted content of the sub-view, and the extracted content of the sub-view includes the extracted title and the parsed web page content; or, for each sub-view, when the view type of the sub-view is a file, call the file extraction engine in the global collection middleware platform to obtain the file path of the file, and determine the copied file of the file or the file path of the file as the extracted content of the sub-view, and determine the file information of the file as the extracted content of the sub-view; or, for each sub-view, when the view type of the sub-view is a video, call the video extraction engine in the global collection middleware platform to identify the video playback area by reading the video identifier (ID) written by the first application into the accessibility node, and perform screen recording, screenshotting, and / or adding tags to obtain the extracted content of the sub-view; or, for each sub-view, when the view type of the sub-view is a link, call the online link extraction engine in the global collection middleware platform to obtain link information according to the uniform resource locator (URL) field carried in the activity or in the way of getting the URL of the web view, and the obtained link information is the extracted content of the sub-view.In this implementation, different content extraction strategies are adopted for views of different types, enabling the solution of the present application to support content extraction of multiple views simultaneously, rather than just single-view content extraction.

[0079] In an example, for each sub-view, when the view type of the sub-view is text, when calling the text extraction module in the global collection middleware platform to obtain the text data of the sub-view and determining the obtained text data as the content after extraction of the sub-view, it may include: using the text extraction module to call the intelligent recognition platform, enabling the intelligent recognition platform to obtain the text data of the sub-view through OCR (the execution area of OCR corresponds to a screenshot of the entire first interface), or using the text extraction module to obtain the text data of the sub-view by reading the information written by the first application into the accessibility node. That is to say, for text-type sub-views, there are two ways to extract content, OCR recognition or reading from the accessibility node.

[0080] In one implementation, when responding to the user's click operation on the confirmation collection control in the third interface and using the global collection middleware platform to add the first material to the second application, it may include at least one of the following operations: directly adding the text material in the first material to the second application; storing the picture material in the first material in the gallery application and adding the thumbnail of the picture to the second application, where the picture stored in the gallery application is associated with the thumbnail of the picture in the second application, and the thumbnail of the picture is the mapping element of the picture in the second application; storing the offline web page material in the first material in the note application and adding the brief content of the offline web page to the second application, where the offline web page stored in the note application is associated with the brief content of the offline web page in the second application, and the brief content of the offline web page is the mapping element of the offline web page in the second application; storing the video material in the first material in the gallery application and adding the thumbnail of the first-frame image of the video to the second application, where the video stored in the gallery application is associated with the thumbnail of the first-frame image of the video in the second application, and the thumbnail of the first-frame image of the video is the mapping element of the video in the second application; storing the file material in the first material in the file management application and adding the file information of the file to the second application, where the file stored in the second application is associated with the file information of the file in the second application, and the file information of the file is the mapping element of the file in the second application, and the file information includes the icon of the file type and the file name; and directly adding the link material in the first material to the second application. In this implementation, it is described that for materials of different view types (different types of materials) in the first material, the collection will be completed according to different addition schemes. Figure 6 and Figure 7An exemplary introduction to this part of the content is also given and will not be elaborated here.

[0081] In another implementation, the first control is a web page favorite control, a link favorite control, or a file favorite control. When responding to a click operation of the user on the first control among at least one control and adding the first material to the second application in the favorite form corresponding to the first control by using the first processing framework, it may include: when the first control is a web page favorite control, adding the first material to the second application in the form of an offline web page, where the first material is the content of the entire web page displayed in the first interface; or, when the first control is a web page favorite control, adding the first material to the second application in the form of an offline web page, where the first material is the content displayed on the interface of the web page displayed in the first interface; or, when the first control is a link favorite control, adding the first material to the second application in the form of a link, where the first material is the online link of the web page displayed in the first interface; or, when the first control is a file favorite control, adding the first material to the second application in the form of a file, where the first material is the file displayed in the first interface, and the file is an open document, a video displayed full screen, or a picture displayed full screen. In this implementation, examples of other favorite forms are given, enriching the form of favorite content and facilitating users to make favorites according to different needs. That is to say, users can make favorites of the content they want to favorite in different ways according to different favorite needs.

[0082] In another implementation, when calling the global favorite middle platform of the first processing framework to determine the view types included in the screenshot of the first interface, it may include: determining the view types included in the screenshot of the first interface according to the whitelist configuration of the first application or the whitelist configuration of the first interface. In this implementation, determining the view types included in the screenshot according to the whitelist configuration, the introduction of the whitelist configuration can help determine the extractable range in the interface, making the extracted content have customized constraint conditions, and different whitelists can be set for different applications or different interfaces as needed.

[0083] In another implementation, when a second interface is displayed on an electronic device and the second interface includes a screenshot of a first interface and at least one control, it may include: when the view types included in the screenshot of the first interface include web pages, at least one of the controls includes at least one of a web page favorite control, a link favorite control, and a content extraction control; or, when the view types included in the screenshot of the first interface include files, at least one of the controls includes at least one of a file favorite control and a content extraction control; or, when the view types included in the screenshot of the first interface only include pictures or videos, at least one of the controls includes at least one of a file favorite space and a content extraction control. In this implementation, the type of control displayed on the second interface is determined based on the different view types included in the screenshot, which is convenient for users to select according to their needs. For example, assuming that the first interface does not contain a web view, there is no need to display a web page favorite control on the second interface.

[0084] It should be understood that the view types included in the screenshot of the first interface refer to the types of views in the first interface that allow extraction. For example, for interface 104, which displays an online web page and does not include videos or files, the types of views in the first interface that allow extraction include offline web pages, online links, text, and pictures. That is to say, the view types included in interface 104 include offline web pages, online links, text, and pictures. Assuming that the interface displays a full-screen picture, the view types allowed for extraction in this interface include pictures. Assuming that the picture also includes text, the view types allowed for extraction in this interface also include text. Other cases will not be listed one by one.

[0085] In another implementation, when adding the second material in the first interface to the second application in response to a second operation of the user on the first interface using the second processing framework, it may include: obtaining the second material selected by the user in the first interface using the content recognition service in the second processing framework; determining the view type included in the second material according to the long-press coordinates of the second operation using the content recognition service; traversing the view tree using the view recognizer in the second processing framework to obtain the sub-views with extractable content in the second material, obtaining a second sub-view set, where the second sub-view set includes at least one sub-view with extractable content; using the content recognition service to perform content extraction for each sub-view in the second sub-view set according to the view type of the sub-view, obtaining the extracted content corresponding to the sub-view; after all the sub-views in the second sub-view set are extracted, obtaining the extracted content of the second sub-view set; using the WMS in the second processing framework to add the extracted content of the second sub-view set to the second application according to the view type included in the extracted content of the second sub-view set. In this implementation, the global collection is triggered under a drag operation. Since dragging requires selecting an object first and then dragging, and the starting coordinates of the drag (the coordinates at the long press) can be obtained, the sub-views of the selected content are determined by traversing and searching all sub-views in the view tree, rather than identifying all sub-views in the entire interface. This implementation first allows the user to select the object to be collected (the second material), and then performs view recognition and content extraction in the second material, and collects the extracted content. Figure 1 The drag operation on the interface 103 can be regarded as an example of a long-press drag operation. Figure 7 Examples of the internal execution process of this implementation and an example of the second processing framework are given.

[0086] In one example, the second sub-view set includes sub-views of at least one view type among text, picture, offline web page, file, video, and link. That is, it includes at least one of textview, imageview, webview, fileview, videoview, and onlinelinkview.

[0087] When using the content recognition service to extract content for each sub-view in the second sub-view set according to the view type of the sub-view, the obtained content after extraction corresponding to the sub-view may include: for each sub-view, when the view type of the sub-view is text, obtain text data through OCR (the execution area of OCR corresponds to the second material rather than the first interface), or obtain the text data of the sub-view by reading the information written by the first application into the accessibility node, and determine the obtained text data as the content after extraction of the sub-view; or, for each sub-view, when the view type of the sub-view is a picture, obtain the bitmap of the picture from the sub-view, and determine the bitmap as the content after extraction of the sub-view, and / or obtain the text data contained in the picture through OCR (the execution area of OCR corresponds to the picture); or, for each sub-view, when the view type of the sub-view is a file, obtain the file path of the file, and determine the copied file of the file or the file path of the file as the content after extraction of the sub-view; or, for each sub-view, when the view type of the sub-view is a video, identify the playback area of the video by reading the video ID written by the first application into the accessibility node, and perform screen recording, screenshotting, and / or adding tags; or, for each sub-view, when the view type of the sub-view is a link, obtain link information according to the URL field carried in the activity framework activity or the getURL method of the webview, and the obtained link information is the content after extraction of the sub-view. In this example, under the trigger of the second operation, different content extraction strategies are applied to views of different types, enabling the solution of this application to support content extraction of multiple views simultaneously, rather than just single-view content extraction.

[0088] It should be understood that in the case of a drag operation, generally, there will be no situation where it is necessary to collect an offline web page. For example, in interface 104, it is impossible to select the entire web page and then drag it.

[0089] It should be noted that the rules for traversing and filtering sub-views and the whitelist configuration described above also apply here and will not be elaborated further.

[0090] In one implementation, when using the window management service in the second processing framework to add the extracted content of the second subset of views to the second application according to the view types included in the extracted content of the second subset of views, at least one of the following operations may be included: directly adding the text content in the extracted content of the second subset of views to the second application; storing the image content in the extracted content of the second subset of views in the gallery application and adding the thumbnail of the image to the second application, where the image stored in the gallery application is associated with the thumbnail of the image in the second application, and the thumbnail of the image is a mapped element of the image in the second application; storing the offline web page content in the extracted content of the second subset of views in the note application and adding the summary content of the offline web page to the second application, where the offline web page stored in the note application is associated with the summary content of the offline web page in the second application, and the summary content of the offline web page is a mapped element of the offline web page in the second application; storing the video content in the extracted content of the second subset of views in the gallery application and adding the thumbnail of the first frame image of the video to the second application, where the video stored in the gallery application is associated with the thumbnail of the first frame image of the video in the second application, and the thumbnail of the first frame image of the video is a mapped element of the video in the second application; storing the file content in the extracted content of the second subset of views in the file management application and adding the file information of the file to the second application, where the file stored in the second application is associated with the file information of the file in the second application, and the file information of the file is a mapped element of the file in the second application, and the file information includes the icon of the file type and the file name; and directly adding the link content in the extracted content of the second subset of views to the second application. That is to say, the method of storing and displaying in the second application according to different material / content types described above is applicable to this step under both triggering operations.

[0091] In one example, the process of identifying views and extracting content from child views may include: confirming the whitelist for application page content extraction through offline touch testing (i.e., predefining the whitelist configuration according to actual requirements) to initially lock down the scope of extractable content on the current page (i.e., determining the extractable views on the page); if it is a global collection triggered by a long - press and drag operation, traverse the view tree with the coordinates of the current long - press to find all child views with extractable content; if it is a three - finger swipe down, determine the scope of extractable content according to the whitelist configuration by priority, and at the same time traverse and filter the views with extractable content on the current page to obtain the child views with extractable content as needed; according to the child views with extractable content obtained in the previous step, for each type of view, extract content according to multiple different rules. If text is involved, the OCR ability can be used to recognize the text on the current page or read the information written by the application into the accessibility node; if a web page is involved, use the readability method to extract the title and parse the content; if a video is involved, by reading the video ID written by the application in the accessibility node, automatically identify the video playback area and perform screen recording / screen capture / adding tags, etc.; if a picture is involved, obtain the bitmap of the picture from the child view to generate a picture; if a file is involved, obtain the file path and perform copy or path mapping as needed; if an online link (URL) is involved, obtain the link according to the URL field carried in the activity or the way of getting the uniform resource locator (getURL) of the web view (webview).

[0092] In one implementation,[ Figure 8 The method shown further includes steps S803 and S804.

[0093] S803. Display a snackbar on the electronic device, where the snackbar includes a prompt indicating successful collection and a view control.

[0094] S804. In response to the user's click operation on the view control, start a second application and display the addition result of the content to be collected in the interface of the second application.

[0095] Figure 8 In the solution shown, two processing frameworks are designed, so that different processing frameworks can be called based on different user operations to achieve the purpose of cross - application collection, making the entire collection process more flexible and diverse, capable of meeting the user's various collection needs, and also facilitating the user to collect different types of materials as needed. The characteristics of the two trigger operations are different, and designing two processing frameworks can effectively improve the processing efficiency.

[0096] The above mainly introduced the method of the embodiments of the present application in combination with the accompanying drawings. It should be understood that although the steps in the flowcharts involved in the above-mentioned embodiments are shown in sequence, these steps are not necessarily executed in the order shown in the figures. Unless there is a clear description in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps. The device of the embodiments of the present application will be introduced below in combination with the accompanying drawings.

[0097] Figure 9 It is a schematic diagram of a device for collecting materials across applications according to an embodiment of the present application. As Figure 9 shown, the device 2000 includes a display unit 2001 and a processing unit 2002. The device 2000 may be integrated in electronic devices such as mobile phones, tablet computers, and touch-screen laptop computers.

[0098] The device 2000 can be used to execute any of the above methods. For example, the display unit 2001 can be used to execute step S801, and the processing unit 2002 is used to execute steps S802-S804. The device 2000 can also be used to execute Figures 1 - 7 the interaction process or execution process shown.

[0099] In one implementation, the device 2000 may further include a storage unit for storing relevant data. The storage unit may be integrated in any of the above units, or may be a unit independent of all the above units.

[0100] Figure 10 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. As Figure 10As shown, the electronic device 900 may include a processor 910, an external memory interface 920, an internal memory 921, a universal serial bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna 1, an antenna 2, a mobile communication module 950, a wireless communication module 960, an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, a headphone interface 970D, a sensor module 980, a key 990, a motor 991, an indicator 992, a camera 993, a display screen 994, and a subscriber identification module (SIM) card interface 995, etc. Among them, the sensor module 980 may include a pressure sensor 980A, a gyroscope sensor 980B, a barometric pressure sensor 980C, a magnetic sensor 980D, an acceleration sensor 980E, a distance sensor 980F, a proximity light sensor 980G, a fingerprint sensor 980H, a temperature sensor 980J, a touch sensor 980K, an ambient light sensor 980L, a bone conduction sensor 980M, etc.

[0101] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 900. In other embodiments of the present application, the electronic device 900 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0102] Exemplarily, Figure 10 The illustrated processor 910 may include one or more processing units. For example, the processor 910 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0103] Among them, the controller may be the nerve center and command center of the electronic device 900. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0104] A memory can also be set in the processor 910 for storing instructions and data. In some embodiments, the memory in the processor 910 is a cache memory. This memory can store the instructions or data that the processor 910 has just used or recycled. If the processor 910 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 910, and thus improves the efficiency of the system.

[0105] The electronic device 900 realizes the display function through the GPU, the display screen 994, and the application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 994 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 910 may include one or more GPUs, which execute program instructions to generate or change display information.

[0106] The display screen 994 is used to display images, videos, etc. The display screen 994 includes a display panel. In some embodiments, the electronic device 900 may include 1 or N display screens 994, where N is a positive integer greater than 1.

[0107] The electronic device 900 can realize the shooting function through the ISP, the camera 993, the video codec, the GPU, the display screen 994, and the application processor, etc.

[0108] The NPU is a neural-network (NN) computing processor. By learning from the structure of biological neural networks, such as the transmission mode between human brain neurons, it can quickly process input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 900 can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0109] The pressure sensor 980A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 980A can be disposed on the display screen 994. There are many types of pressure sensors 980A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 980A, the capacitance between the electrodes changes. The electronic device 900 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 994, the electronic device 900 detects the intensity of the touch operation according to the pressure sensor 980A. The electronic device 900 can also calculate the position of the touch according to the detection signal of the pressure sensor 980A. In some embodiments, touch operations with the same touch position but different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0110] The touch sensor 980K, also known as the "touch panel". The touch sensor 980K can be disposed on the display screen 994, and the touch sensor 980K and the display screen 994 form a touch screen, also known as the "touch screen". The touch sensor 980K is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 994. In some other embodiments, the touch sensor 980K can also be disposed on the surface of the electronic device 900, at a different position from the display screen 994.

[0111] It should be noted that for the information interaction, execution process, etc. between the above-mentioned devices / units, since they are based on the same concept as the method embodiments of the present application, for their specific functions and the technical effects brought, please refer to the method embodiment part for details, and will not be elaborated here.

[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0113] An embodiment of this application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, the electronic device can implement the steps in any of the above methods.

[0114] An embodiment of this application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by an electronic device, it can implement the steps in each of the above method embodiments.

[0115] An embodiment of this application provides a computer program product, which includes a computer program. When the computer program is executed by an electronic device, it can implement the steps in each of the above method embodiments.

[0116] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the photographing device / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0117] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0118] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0119] In the embodiments provided in this application, it should be understood that the disclosed device / equipment and method can be implemented in other ways. For example, the device / equipment embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical, mechanical, or other form.

[0120] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] It should be understood that when used in the specification and claims of this application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0122] It should also be understood that the term "and / or" as used in the specification and claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0123] As used in the specification and claims of this application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" according to the context.

[0124] In addition, in the description of the specification and claims of this application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0125] The reference to "one embodiment" or "some embodiments" etc. described in the specification of this application means that a specific feature, structure or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0126] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for collecting materials across applications, characterized in that, Applied to an electronic device, which includes a first processing framework and a second processing framework, the method includes: Run a first application on the electronic device and display a first interface; In response to a first operation of the user on the first interface, use the first processing framework to add a first material in the first interface to a second application, where the first operation is a three-finger swipe-down operation in the first interface; or, In response to a second operation of the user on the first interface, use the second processing framework to add a second material in the first interface to the second application, where the second operation is a long-press drag operation on the selected second material in the first interface.

2. The method according to claim 1, characterized in that The step of, in response to a first operation of the user on the first interface, using the first processing framework to add a first material in the first interface to a second application, includes: Call the interface of the Activity Management Service (AMS) in the first processing framework to obtain a screenshot of the first interface; Call the global collection middleware platform of the first processing framework to determine the view types included in the screenshot of the first interface; Display a second interface on the electronic device, where the second interface includes the screenshot of the first interface and at least one control, and the at least one control is determined according to the view types included in the screenshot of the first interface; In response to a click operation of the user on a first control among the at least one control, use the first processing framework to add the first material to the second application in the collection form corresponding to the first control.

3. The method according to claim 2, wherein The first control is a content extraction control. The step of, in response to a click operation of the user on the first control among the at least one control, using the first processing framework to add the first material to the second application in the collection form corresponding to the first control, includes: Use the global collection middleware platform to traverse and filter all views in the screenshot of the first interface to obtain a first subset of views, and the first subset of views includes at least one sub-view that can extract content; Use the global collection middleware platform to perform content extraction for each sub-view in the first subset of views according to the view type of the sub-view to obtain the extracted content corresponding to the sub-view; After all sub-views in the first subset of views are extracted, display a third interface on the electronic device, and the third interface includes the extracted content of the first subset of views; In response to a selection operation of the user on the target content in the extracted content of the first subset of views in the third interface, determine the selected target content as the first material; In response to a click operation of the user on the confirmation collection control in the third interface, use the global collection middleware platform to add the first material to the second application.

4. The method according to claim 3, wherein The step of using the global collection middleware platform to traverse and filter all views in the extractable range to obtain a first subset of views, includes: During traversal, for each view in the screenshot of the first interface, perform at least one of the following operations: when the view is a standard view, obtain the view type of the view using the acquisition method of the standard view, or, when the view is a custom view, determine the view type of the view using the suffix of the view; and, read the size and / or position of the view. During filtering, perform at least one of the following operations: only retain the views visible to the user in the screenshot of the first interface, remove the hidden views from the screenshot of the first interface, and remove the views that are partially or completely blocked in the screenshot of the first interface.

5. The method according to claim 3, wherein The first subset of sub-views includes sub-views of at least one view type among text, picture, offline web page, file, video, and link; using the global collection middleware, for each sub-view in the first subset of sub-views, perform content extraction according to the view type of the sub-view to obtain the extracted content corresponding to the sub-view, including: For each sub-view, when the view type of the sub-view is text, call the text extraction module in the global collection middleware to obtain the text data of the sub-view, and determine the obtained text data as the extracted content of the sub-view; or, For each sub-view, when the view type of the sub-view is a picture, call the picture extraction engine in the global collection middleware to obtain the bitmap of the picture from the sub-view, and determine the bitmap as the extracted content of the sub-view, and / or, call the text processing module in the global collection middleware to obtain the text data contained in the picture by means of optical character recognition (OCR); or, For each sub-view, when the view type of the sub-view is an offline web page, call the web page extraction engine in the global collection middleware to perform title extraction and content parsing on the web page to obtain the extracted content of the sub-view, and the extracted content of the sub-view includes the extracted title and the parsed web page content; or, For each sub-view, when the view type of the sub-view is a file, call the file extraction engine in the global collection middleware to obtain the file path of the file, and determine the copied file of the file or the file path of the file as the extracted content of the sub-view, and determine the file information of the file as the extracted content of the sub-view; or, For each sub-view, when the view type of the sub-view is a video, call the video extraction engine in the global collection middleware, identify the playing area of the video by reading the video identification ID written by the first application to the accessibility node, and perform screen recording, screenshot, and / or add tags, so as to obtain the extracted content of the sub-view; or, For each sub-view, when the view type of the sub-view is a link, the online link extraction engine in the global collection middleware platform is called to obtain link information according to the uniform resource locator (URL) field carried in the activity or the way of getting the URL by reading the web view of the web page. The obtained link information is the content after extraction of the sub-view.

6. The method according to claim 5, wherein For each sub-view, when the view type of the sub-view is text, the text extraction module in the global collection middleware platform is called to obtain the text data of the sub-view, and the obtained text data is determined as the content after extraction of the sub-view, including: The text extraction module is used to call the intelligent recognition platform, so that the intelligent recognition platform obtains the text data of the sub-view by means of OCR, or, the text extraction module obtains the text data of the sub-view by reading the information written into the accessibility node by the first application.

7. The method according to claim 3, characterized in that, In response to the user's click operation on the confirmation collection control in the third interface, the global collection middleware platform is used to add the first material to the second application, including at least one of the following operations: Directly add the text material in the first material to the second application; Store the picture material in the first material in the gallery application, and add the thumbnail of the picture to the second application. The picture stored in the gallery application is associated with the thumbnail of the picture in the second application, and the thumbnail of the picture is the mapped element of the picture in the second application; Store the offline web page material in the first material in the note application, and add the brief content of the offline web page to the second application. The offline web page stored in the note application is associated with the brief content of the offline web page in the second application, and the brief content of the offline web page is the mapped element of the offline web page in the second application; Store the video material in the first material in the gallery application, and add the thumbnail of the first frame image of the video to the second application. The video stored in the gallery application is associated with the thumbnail of the first frame image of the video in the second application, and the thumbnail of the first frame image of the video is the mapped element of the video in the second application; Store the file material in the first material in the file management application, and add the file information of the file to the second application. The file stored in the second application is associated with the file information of the file in the second application, and the file information of the file is the mapped element of the file in the second application. The file information includes the icon of the file type and the file name; And, Directly add the link material in the first material to the second application.

8. The method according to claim 2, wherein The first control is a web page collection control, a link collection control, or a file collection control. Responding to a click operation by the user on the first control among the at least one control, and using the first processing framework to add the first material to the second application in the collection form corresponding to the first control includes: When the first control is a web page collection control, adding the first material to the second application in the form of an offline web page, where the first material is the content of the entire web page displayed in the first interface; or, When the first control is a web page collection control, adding the first material to the second application in the form of an offline web page, where the first material is the content displayed on the interface of the web page displayed in the first interface; or, When the first control is a link collection control, adding the first material to the second application in the form of a link, where the first material is the online link of the web page displayed in the first interface; or, When the first control is a file collection control, adding the first material to the second application in the form of a file, where the first material is the file displayed in the first interface, and this file is an open document, a full-screen video, or a full-screen picture.

9. The method according to claim 2, wherein Invoking the global collection middle platform of the first processing framework to determine the view types included in the screenshot of the first interface includes: determining the view types included in the screenshot of the first interface according to the whitelist configuration of the first application or the whitelist configuration of the first interface; Displaying a second interface on the electronic device, where the second interface includes the screenshot of the first interface and at least one control, includes: When the view types included in the screenshot of the first interface include a web page, at least one of the at least one control includes at least one of a web page collection control, a link collection control, and a content extraction control; or, When the view types included in the screenshot of the first interface include a file, at least one of the at least one control includes at least one of a file collection control and a content extraction control; or, When the view types included in the screenshot of the first interface only include a picture or a video, at least one of the at least one control includes at least one of a file collection space and a content extraction control.

10. The method according to claim 1, characterized in that Responding to a second operation by the user on the first interface, and using the second processing framework to add the second material in the first interface to the second application, includes: Using the content recognition service in the second processing framework to obtain the second material selected by the user in the first interface; Using the content recognition service to determine the view types included in the second material according to the long-press coordinates of the second operation; Using the view recognizer in the second processing framework to traverse and search the view tree to obtain the sub-views with extractable content included in the second material, obtaining a second sub-view set, where the second sub-view set includes at least one sub-view with extractable content; Using the content recognition service, for each sub-view in the second sub-view set, content extraction is performed according to the view type of the sub-view to obtain the extracted content corresponding to the sub-view; After all sub-views in the second sub-view set are extracted, the extracted content of the second sub-view set is obtained; Using the window management service WMS in the second processing framework, according to the view types included in the extracted content of the second sub-view set, the extracted content of the second sub-view set is added to the second application.

11. The method according to claim 10, wherein The step of using the window management service WMS in the second processing framework to add the extracted content of the second sub-view set to the second application according to the view types included in the extracted content of the second sub-view set includes at least one of the following operations: Directly add the text content in the extracted content of the second sub-view set to the second application; Store the picture content in the extracted content of the second sub-view set in the gallery application, and add the thumbnail of the picture to the second application. The picture stored in the gallery application is associated with the thumbnail of the picture in the second application, and the thumbnail of the picture is the mapped element of the picture in the second application; Store the offline web page content in the extracted content of the second sub-view set in the note application, and add the brief content of the offline web page to the second application. The offline web page stored in the note application is associated with the brief content of the offline web page in the second application, and the brief content of the offline web page is the mapped element of the offline web page in the second application; Store the video content in the extracted content of the second sub-view set in the gallery application, and add the thumbnail of the first frame image of the video to the second application. The video stored in the gallery application is associated with the thumbnail of the first frame image of the video in the second application, and the thumbnail of the first frame image of the video is the mapped element of the video in the second application; Store the file content in the extracted content of the second sub-view set in the file management application, and add the file information of the file to the second application. The file stored in the second application is associated with the file information of the file in the second application, and the file information of the file is the mapped element of the file in the second application. The file information includes the icon of the file type and the file name; And, Directly add the link content in the extracted content of the second sub-view set to the second application.

12. The method according to any one of claims 1 to 11, characterized in that, After adding the first material or the second material to the second application, the method further includes: Display a floating capsule snackbar on the electronic device. The snackbar includes a prompt indicating successful collection and a viewing control; In response to the user's click operation on the viewing control, start the second application and display the addition result on the interface of the second application.

13. An electronic device, comprising a memory, one or more processors, and a computer program stored in the memory and executable on the processors, characterized in that, When the one or more processors execute the computer program, the electronic device implements the method according to any one of claims 1 to 12.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by an electronic device, the method according to any one of claims 1 to 12 is implemented.

Images