Information collection method and device and electronic equipment

Abstract

The invention discloses an information collection method and device and electronic equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: receiving collection input of program information in a program interface, wherein the program information comprises at least one of the following items: a file and a link; displaying an information collection interface in response to the collection input; wherein the information collection interface comprises a first collection option corresponding to the program information, and the first collection option comprises an information identifier of the program information and collection detail information of the program information; collection detail information in the information collection interface comprises at least one of the following items: a keyword of program information, at least one application icon, a collection source identifier and a sharing control, and the at least one application icon is an icon of an application program matched with a user intention; the user intention is obtained based on keyword prediction.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to an information collection method, device and electronic device. Background Art

[0002] Most current applications have a favorites feature, which meets the needs of temporarily storing and managing files.

[0003] In related technologies, after a user views a file of interest in an application, he or she may not be able to process the file in a timely manner due to various reasons. The user can trigger the electronic device to save the file of interest to the user in the favorites so that the user can subsequently manually edit the file or process the file through other applications.

[0004] However, in the above method, since the electronic device requires the user to manually edit the files collected in the favorites; or, the electronic device requires the user to manually copy the files collected in the favorites to other applications so that the files can be processed by other applications, the file editing process is cumbersome and time-consuming. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide an information collection method, device and electronic device, which can simplify the process of file editing by the electronic device, thereby improving the efficiency of file editing by the electronic device.

[0006] In a first aspect, an embodiment of the present application provides an information collection method, which includes: receiving a collection input for program information in a program interface, the program information including at least one of the following: a file, a link; in response to the collection input, displaying an information collection interface; wherein the information collection interface includes a first collection option corresponding to the program information, the first collection option including an information identifier of the program information and collection details information of the program information; the collection details information in the information collection interface includes at least one of the following: a keyword of the program information, at least one application icon, a collection source identifier, and a sharing control; the at least one application icon is an icon of an application that matches the user's intention, and the user intention is predicted based on the keyword.

[0007] In a second aspect, an embodiment of the present application provides an information collection device, comprising: a receiving module for receiving a collection input for program information in a program interface, wherein the program information includes at least one of the following: a file, a link. A display module for displaying an information collection interface in response to the collection input received by the receiving module; wherein the information collection interface includes a first collection option corresponding to the program information, wherein the first collection option includes an information identifier of the program information and collection details information of the program information; the collection details information in the information collection interface includes at least one of the following: a keyword of the program information, at least one application icon, a collection source identifier, and a sharing control; wherein the at least one application icon is an icon of an application that matches the user's intention, and the user intention is predicted based on the keyword.

[0008] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, a collection input for program information in a program interface is received, the program information including at least one of the following: a file, a link; then, in response to the collection input, an information collection interface is displayed; wherein the information collection interface includes a first collection option corresponding to the program information, the first collection option including an information identifier of the program information and collection details of the program information; the collection details in the information collection interface include at least one of the following: a keyword of the program information, at least one application icon, a collection source identifier, and a sharing control; the at least one application icon is an icon of an application that matches the user's intent, the user intent being predicted based on the keyword. In this solution, since the information collection interface already includes the first collection option corresponding to the program information, it can be understood that when the program information is collected, the program information can be deeply processed to achieve comprehensive content extraction of the program information and the extracted content is displayed in the information collection interface, thereby avoiding the need for additional editing operations when the user edits the program information; and the user's next intention can be predicted, facilitating the user's convenient use after finding the first collection option corresponding to the program information, thereby simplifying the processing steps after the electronic device collects the program information, significantly improving the user experience and the efficiency of file editing in the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flowchart of an information collection method provided for some embodiments of the present application;

[0014] Figure 2A A schematic diagram of an evaluation application interface provided for some embodiments of the present application;

[0015] Figure 2B A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0016] Figure 3A A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0017] Figure 3B A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0018] Figure 4 A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0019] Figure 5A A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0020] Figure 5B A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0021] Figure 6A schematic diagram of a collection dialogue interface provided for some embodiments of the present application;

[0022] Figure 7 A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0023] Figure 8 A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0024] Figure 9 A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0025] Figure 10 A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0026] Figure 11A A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0027] Figure 11B A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0028] Figure 12A A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0029] Figure 12B A schematic diagram of a collection dialogue interface provided for some embodiments of the present application;

[0030] Figure 13A A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0031] Figure 13B A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0032] Figure 14A A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0033] Figure 14B A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0034] Figure 15A A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0035] Figure 15B A schematic diagram of a favorites editing interface provided for some embodiments of the present application;

[0036] Figure 16 A schematic diagram of a favorites editing interface provided for some embodiments of the present application;

[0037] Figure 17 A schematic diagram of an information collection interface provided for some embodiments of the present application;

[0038] Figure 18A A schematic diagram of a favorites editing interface provided for some embodiments of the present application;

[0039] Figure 18B A schematic diagram of a collection details interface provided in some embodiments of the present application;

[0040] Figure 19 A model structure diagram of a multimodal reasoning model provided for some embodiments of the present application;

[0041] Figure 20 is a schematic structural diagram of an information collection device provided by some embodiments of the present application;

[0042] Figure 21 is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application;

[0043] Figure 22 This is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0045] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0046] The terms "at least one" and "at least one of" in the specification and claims of this application refer to any one, any two, or a combination of more than two of the objects included. For example, at least one of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c", and "a, b, and c", where a, b, and c can be single or multiple. Similarly, "at least two" means two or more, and its meaning is similar to "at least one".

[0047] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0048] The following is an explanation of the terms involved in the embodiments of the present application.

[0049] Multimodality: Multimodality refers to the multiple representations of data or information. A single piece of information can exist in multiple forms. To enable computers to analyze internet data and mimic human cognition, multimodal information processing technologies, which simultaneously process data from multiple modalities, have emerged. In current AI tasks, multimodality primarily refers to support for the 3V tasks: text (verbal), voice (vocal), and vision (visual). Many classic tasks in deep learning are based on the conversion between these three tasks. For example, image generation tasks generate images based on text descriptions, while image description tasks, conversely, generate text based on images. Multimodal learning is a learning method that utilizes multimodal data, which may include text, images, audio, video, and more. By integrating multiple data modalities to train models, the model's perception and understanding capabilities are improved, enabling cross-modal information interaction and fusion.

[0050] Model: A model is a simulation or abstraction of certain characteristics and internal connections of an objective reality. Model is a role. The concept of a model can be defined as follows: an object is called a "model" because of its role or purpose in a specific situation—in that situation, it directly or indirectly carries certain attributes of another object and, based on these attributes, serves as a stand-in or representation of that object. Thus, by using the attributes obtained from the model, operations can be correlated with the corresponding attributes of the object.

[0051] Model training: Model training involves adjusting model parameters by learning from large amounts of data, enabling the model to accurately predict unknown data. Model training involves continuously adjusting model parameters. This process requires fully utilizing the dataset to evaluate model performance, ultimately resulting in a model with good performance and strong generalization capabilities.

[0052] Application: An application is a computer program that is used to complete one or more specific tasks. It runs in user mode, can interact with the user, and has a visual user interface.

[0053] Icons: Icons are computer graphics with a clear meaning. They can also be graphic symbols with referential meaning. They are more than just a graphic; they are also symbols, conveying information in a highly condensed and quick manner, making them easy to remember.

[0054] User Interface: The User Interface (UI) bridges the gap between the system and the user for interaction and communication. UI design refers to the stylistic design of the graphical user interface (GUI), including color schemes, content arrangement, and layout. It encompasses not only the graphical interfaces of everyday desktop and mobile devices, but also new interaction methods like touchscreens, motion sensing, and voice control. The three key principles of UI design are consistency, flexibility, and simplicity.

[0055] Control: A control (also called part, component, widget or control) is a graphical user interface element and the basic building block of the user interface, such as a window or text box, displayed in the program interface of any application. A control can be a button, text box, label, etc., used to control all data processed by each application and the interactive operations on this data.

[0056] Deep thinking: Deep thinking is about constantly approaching the essence of a problem. It trains the brain to have information retrieval capabilities, enabling it to grasp the key points and see the essence. Deep thinking is all about exploring the essence of things.

[0057] Deep Inference: Deep inference generally refers to the inference phase of deep learning, which is the process of using the trained model to make predictions or generate results for new data after training. The inference phase is a critical step in the application of deep learning models. During this phase, the model no longer updates weights, but instead uses the knowledge learned during the training phase to process new, previously unseen data. Specifically, the model receives input data and, through a forward propagation operation, produces output results. These output results can be category labels, target location coordinates, descriptions of the input data, and more, depending on the model's task and design. Deep learning models are widely used in the inference phase and can be embedded in various applications such as computer vision systems, natural language processing systems, and speech recognition systems to process new data in real time or in batches.

[0058] File: A file is the basic unit of data storage and organization in a computer. Files can contain various types of data, such as program code, text documents, audio files, video files, and images. These files are stored on disks or other storage media to facilitate user access, editing, and management of the data.

[0059] Recall: Recall refers to the ratio of all true positive samples successfully detected by the model to the total number of true positive samples. Recall and precision are two metrics widely used in information retrieval and statistical classification to evaluate the quality of results. Recall is the ratio of the number of relevant documents retrieved to the total number of relevant documents in the document library, measuring the recall rate of the retrieval system. Precision is the ratio of the number of relevant documents retrieved to the total number of documents retrieved, measuring the precision rate of the retrieval system.

[0060] Link: A link is a link that jumps from one place to another, or from one page to another, on the internet. Links can be expressed in several ways: 1. Plain text links: These simply display the link address and text on the page, for example, "textdomain.com"; 2. Anchor text links: Anchor text links can contain both hyperlinks and text links, for example, "a href = link address text"; 3. Image / icon links: Similar to anchor text links, for example, "a href = link address image / icon."

[0061] The information collection method, device, and electronic device provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0062] The information collection method, device, and electronic device provided in the embodiments of the present application can be applied to file editing scenarios. Specifically, the information collection method provided in the embodiments of the present application can be applied to scenarios where a file is collected and then edited or searched.

[0063] In a specific application scenario, the information collection method provided in the embodiment of the present application can be applied to a scenario in which navigation is performed based on the restaurant name in the collected picture: The boots, for example, scenario 1.

[0064] Scenario 1, when the user needs to navigate based on the restaurant name: The boots in the collection picture, the user can trigger the electronic device to display the collection interface of the collection application, which displays the collection entry corresponding to the restaurant picture. The user can click and input the collection entry so that the electronic device can display the collection details interface corresponding to the collection entry, which includes the restaurant picture, and the restaurant picture includes: restaurant name: The boots, restaurant location: Kerry Center, Gongshu District, average consumption per person: 115 / person; after viewing the restaurant name, the user can trigger the electronic device to exit the collection application, and trigger the electronic device to run the navigation application, and manually enter the restaurant name: Theboots into the navigation interface in the navigation application, so that the electronic device can determine the navigation route by the restaurant name.

[0065] In another specific application scenario, the information collection method provided in the embodiment of the present application can be applied to a scenario in which the document content of the collected document is summarized, for example, scenario 2.

[0066] In scenario 2, when a user needs to summarize the content of a favorite document, the user can trigger the electronic device to display the favorites interface of the favorites application. The favorites interface displays the document identifier "Essential Knowledge for Deep Learning" for the document. Then, the user clicks and enters the document identifier, causing the electronic device to display the favorites editing interface for the document. The favorites editing interface includes the document identifier "Essential Knowledge for Deep Learning" and the document content:

[0067] "1. Basic knowledge: The basic knowledge of deep learning includes mathematical foundations, such as linear algebra, matrix theory, probability statistics, optimization theory, etc., machine learning foundations, and programming foundations.

[0068] 2. Neural network: The core of deep learning is neural network, so it is necessary to have a deep understanding of the working principles of neural networks, including algorithms such as forward propagation and backpropagation.

[0069] 3. Deep network structure: In addition to basic neural networks, you also need to understand various deep network structures, such as convolutional neural networks, recurrent neural networks, etc.;

[0070] Finally, after viewing the document content, the user can manually enter "Deep learning requires a good foundation in mathematics and science and engineering" in the input box in the collection editing interface to complete the summary of the document.

[0071] The execution subject of the information collection method provided in the embodiment of the present application can be an information collection device, which can be an electronic device or a functional module in an electronic device. The following uses an electronic device as an example to illustrate the technical solution provided in the embodiment of the present application.

[0072] The embodiment of the present application provides an information collection method. Figure 1 FIG. 1 shows a flow chart of an information collection method provided by an embodiment of the present application. Figure 1 As shown, the information collection method provided in the embodiment of the present application may include the following steps 201 and 202.

[0073] Step 201: The electronic device receives a collection input of program information in a program interface.

[0074] In some embodiments of the present application, the program information includes at least one of the following: a file, a link.

[0075] In some embodiments of the present application, the above-mentioned file may include at least one of the following: documents, audio, video and pictures.

[0076] In some embodiments of the present application, the above-mentioned link can be any of the following: an interface link, a picture link, a video link, a text link, an audio link or a hyperlink.

[0077] In some embodiments of the present application, the above-mentioned program interface can be any interface of any application program in the electronic device.

[0078] In some embodiments of the present application, the above application may be any of the following: a chat application, a video application, an audio application, a document application, etc. The specific application may be determined according to actual use requirements and is not limited in the embodiments of the present application.

[0079] In some embodiments of the present application, the above interface may be any of the following: a main interface, a secondary interface, or a tertiary interface of an application, etc. The specific interface may be determined according to actual use requirements and is not limited in the embodiments of the present application.

[0080] Exemplarily, the main interface may be a chat identifier display interface of a chat application, the secondary interface may be a conversation interface corresponding to the chat identifier, and the tertiary interface may be a chat record interface corresponding to the conversation interface.

[0081] As another example, the main interface may be a video identification display interface of a video application, the secondary interface may be a video playback interface corresponding to the video identification, and the tertiary interface may be a barrage sending interface corresponding to the video playback interface.

[0082] In some embodiments of the present application, the collection input is used to trigger the electronic device to collect the program information in the program interface into a collection application.

[0083] In some embodiments of the present application, the above-mentioned collection input can be a click input of the above-mentioned program information by the user through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0084] In some embodiments of the present application, the above-mentioned specific gesture can be any one of a single-click gesture, a sliding gesture, a drag gesture, a pressure recognition gesture, a long press gesture, an area change gesture, a double-press gesture, and a double-click gesture.

[0085] In some embodiments of the present application, the click input may be a single click input, a double click input, or any number of click inputs, and may also be a long press input or a short press input. For example, the first input may be a user dragging the program information.

[0086] In some embodiments of the present application, the above-mentioned favorite input may include one or more inputs.

[0087] Example 1, taking a mobile phone as an example of an electronic device, when the mobile phone displays the image storage interface of the gallery application, if the user wants to collect the ID card picture in the gallery application in the collection application, the user can long press the picture identifier of the ID card picture to enable the mobile phone to display the collection control; then, the user can continue to drag the picture identifier to the collection control so that the mobile phone can collect the ID card picture in the collection application.

[0088] Example 2: When the mobile phone displays the audio storage interface of an audio application, if the user wants to collect an audio in the audio application in a favorite application, the user can long-press and input the audio identifier of the audio so that the mobile phone can display the sharing control; then, the user can click and input the sharing control so that the mobile phone can display the application identifier of the favorite application; finally, the user can click and input the application identifier of the favorite application so that the mobile phone can collect the audio in the favorite application.

[0089] Example 3: When the mobile phone displays the document identification display interface of the document application, the user can long press and input a document identifier in the document identification display interface so that the mobile phone can collect the control, and then the user can drag the document identifier to the collection control so that the mobile phone can collect the document corresponding to the document identifier in the collection application.

[0090] Example 4, combined with the above scenario 1, such as Figure 2A As shown, the mobile phone can display a food evaluation interface 10 of a food evaluation application, wherein the food evaluation interface 10 includes a restaurant picture 11 and user evaluation information. The restaurant picture includes the restaurant name: The boots, the restaurant location: Kerry Center, Gongshu District, and the average consumption per person: 115. The user evaluation information includes "Family, we had a beautiful meal near West Lake today. The restaurant name is in the picture." The user can long press the restaurant picture 11 in the food evaluation interface 10 to make the mobile phone display a thumbnail of the restaurant picture 11 and display a "Collection" control 12; then, the user can continue to drag the thumbnail to the "Collection" control 12 so that the mobile phone can collect the restaurant picture 11 to the collection application.

[0091] Step 202: The electronic device displays an information collection interface in response to the collection input.

[0092] In some embodiments of the present application, the above-mentioned information collection interface includes a first collection option corresponding to the program information, and the first collection option includes an information identifier of the program information and collection details information of the program information; the collection details information in the information collection interface includes at least one of the following: a keyword of the program information, at least one application icon, a collection source identifier, and a sharing control, and the at least one application icon is an icon of an application that matches the user's intention; the user's intention is predicted based on the keyword.

[0093] In some embodiments of the present application, the above-mentioned identifier can be any of the following: a text identifier, an image identifier, a special symbol identifier, etc. The specific identifier can be determined according to actual usage and is not limited in the embodiments of the present application.

[0094] In some embodiments of the present application, the electronic device can jump from the program interface to the information collection interface based on the above-mentioned collection input.

[0095] In some embodiments of the present application, the above-mentioned information collection interface may be a collection main interface in a collection application; or, the above-mentioned information collection interface may be an information collection interface in an application corresponding to the program interface.

[0096] For example, in combination Figure 2A ,like Figure 2B As shown, the mobile phone can jump from the food evaluation interface 10 to the information collection interface 20, and the collection interface 20 includes a first collection option 13 corresponding to the restaurant picture 11, and the first collection option 13 includes: the information identifier of the restaurant picture 11, Figure 2B The image 1 is used to represent the restaurant image 11; the keywords of the restaurant image 11 are: restaurant name: The boots, restaurant location: Kerry Center, Gongshu District, average consumption per person: 115 / person; navigation application icon 21, food review application icon 22, "Share from application A" logo 23 and sharing control 24.

[0097] As another example, the mobile phone can jump from the document display interface to the information collection interface, which includes collection options corresponding to the first document, and the collection options corresponding to the first document include: document identification, keywords in the first document: deep learning, neural network and deep network structure; chat program icon, document display interface identification and sharing controls.

[0098] In an information collection method provided in an embodiment of the present application, an electronic device can receive a collection input for program information in a program interface, the program information including at least one of the following: a file, a link; then, in response to the collection input, an information collection interface is displayed; wherein the information collection interface includes a first collection option corresponding to the program information, the first collection option including an information identifier of the program information and collection details of the program information; the collection details in the information collection interface include at least one of the following: a keyword of the program information, at least one application icon, a collection source identifier, and a sharing control; the at least one application icon is an icon of an application that matches a user's intent, the user intent being predicted based on the keyword. In this solution, since the information collection interface already includes the first collection option corresponding to the program information, it can be understood that when the program information is collected, the program information can be deeply processed to extract the comprehensive content of the program information and display the extracted content in the information collection interface, thereby avoiding the need for additional editing operations when the user edits the program information; and the user's next intention can be predicted, facilitating the user's convenient use after finding the first collection option corresponding to the program information, thereby simplifying the processing steps of the electronic device after collecting the program information, significantly improving the user experience and the efficiency of file editing of the electronic device.

[0099] In one example, combining Figure 2B The user can click the navigation application icon 21 in the first collection option 13 in the information collection interface 20, so that the mobile phone can run the navigation application in the foreground and display the navigation application interface, which includes an input box, and the mobile phone can automatically enter the restaurant name: The boots into the input box, so that the mobile phone can automatically obtain the navigation route according to the restaurant name: The boots.

[0100] In another example, combining Figure 2B The user can click the food application icon 22 in the first collection option 13 in the information collection interface 20, so that the mobile phone can run the food evaluation application in the foreground and display the food evaluation application interface, which includes an input box, and the mobile phone can automatically input the restaurant name: The boots into the input box, so that the mobile phone can automatically search for the evaluation information of The boots restaurant according to the restaurant name: The boots.

[0101] In yet another example, combining Figure 2BThe user can click the share control 24 in the first favorites option 13 in the information favorites interface 20, causing the mobile phone to display the program identifier of at least one third-party application in the information favorites interface, taking the program identifier of a chat application or a video application as an example. The user can then click and input the program identifier of the chat application, causing the mobile phone to run the chat application in the foreground and display the application interface of the chat application. The user can then select a contact in the application interface of the chat application, causing the mobile phone to share the application information in the first favorites option with the contact.

[0102] In yet another example, combining Figure 2B The user can click the "Share from A Application" logo 23 in the first collection option 13 in the information collection interface 20, so that the electronic device can display the collection source interface of the restaurant picture, which includes the restaurant picture.

[0103] In some embodiments of the present application, after the above step 202, the information collection method provided by the embodiment of the present application further includes the following steps 301 and 302.

[0104] Step 301: The electronic device receives a selection input for a first favorite option corresponding to program information.

[0105] In some embodiments of the present application, the selection input is used for the user to select a first favorite option from multiple favorite options in the information favorite interface.

[0106] In some embodiments of the present application, the above-mentioned selection input can be a click input of the user on the above-mentioned first favorite option through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0107] Exemplarily, the selection input may be a single-click input of the user on the first favorite option.

[0108] Step 302: The electronic device displays a collection details interface of the first collection option in response to the selection input.

[0109] In some embodiments of the present application, the above-mentioned collection details interface includes: information content of program information, collection details information of program information, deep dialogue control, deep thinking control and import control.

[0110] In some embodiments of the present application, the collection details information in the above-mentioned collection details interface includes at least one of the following: overview information of the program information, keywords of the program information, at least one application icon, a collection source identifier, and a sharing control.

[0111] For example, in combination Figure 2B ,like Figure 3A As shown, the user can click and input the first favorite option 13, as shown in Figure 3B As shown, the mobile phone can display the collection details interface 25 corresponding to the first collection option 13, and the collection details interface 25 includes the restaurant picture 11 and the summary information of the restaurant picture 11: Deep thinking: the address "The boots mud boots" is extracted from the picture, and it is identified as a popular barbecue restaurant located in Kerry Center, Gongshu District, with an average price of 115 yuan per person; the keywords of the restaurant picture 11 are "Theboots mud boots", Kerry Center, Gongshu District, 115 / person, Figure 3B The following are underlined: the navigation program icon 21 corresponding to the restaurant picture 11, the food review interface logo 22 and the sharing control 24; the deep conversation control 26, the deep thinking control 27 and the import control 28.

[0112] For example, combining Figure 3A The user can click the navigation application icon 21 on the favorites details interface 25 so that the mobile phone can run the navigation application in the foreground and display the navigation application interface, which includes an input box, and the mobile phone can automatically input the restaurant name: The boots into the input box, so that the mobile phone can automatically obtain the navigation route according to the restaurant name: Theboots.

[0113] For example, combining Figure 3A The user can click the food application icon 22 in the collection details interface 25 to enable the mobile phone to run the food evaluation application in the foreground and display the food evaluation application interface, which includes an input box, and the mobile phone can automatically input the restaurant name: The boots into the input box, so that the mobile phone can automatically search for the evaluation information of The boots restaurant according to the restaurant name: The boots.

[0114] For example, combining Figure 3A The user can click the share control 24 in the favorites details interface 25 to cause the mobile phone to display the program identifier of at least one third-party application in the favorites details interface 25, taking the program identifier of a chat application and the program identifier of a video application as examples. The user can then click and input the program identifier of the chat application to cause the mobile phone to run the chat application in the foreground and display the application interface of the chat application. The user can then select a contact in the application interface of the chat application to cause the mobile phone to share the application information of the first favorite option with the contact.

[0115] For example, combining Figure 3AThe user can click the import control 28 in the favorites details interface 25 to display the program identifier of at least one third-party application on the favorites details interface 25, such as the program identifier of a chat application or a video application. Next, the user can click and input the program identifier of the chat application to cause the mobile phone to run the chat application in the foreground and display its application interface. The user can then select a contact in the chat application interface to display the chat history between the user and the contact. Finally, the user can select a desired image from the chat history to cause the mobile phone to import the image into the favorites details interface 25.

[0116] As another example, in combination with the above scenario 2, if Figure 4 As shown, when the user needs to view the detailed information of the first document, the user can click and input the collection option corresponding to the first document, so that the mobile phone can display the collection details interface 30 corresponding to the first document, and the collection details interface 30 includes: "1. Basic knowledge: The basic knowledge of deep learning includes mathematical foundations, such as linear algebra, matrix theory, probability statistics, optimization theory, etc., machine learning foundations and programming foundations. 2. Neural network: The core of deep learning is neural network, so it is necessary to have an in-depth understanding of the working principle of neural network, including algorithms such as forward propagation and back propagation. 3. Deep network structure: In addition to basic neural networks, it is also necessary to understand various deep network structures, such as convolutional neural networks, recurrent neural networks, etc."; Deep thinking: According to the above text, the overview information of the first document is: deep learning requires a good mathematical foundation and science and engineering foundation; the keywords of the first document are deep learning, neural network and deep network structure, Figure 4 The browser application icon 31 , the document application icon 32 and the share icon 33 ; and the deep conversation control 26 , the deep thinking control 27 and the import control 28 .

[0117] For example, combining Figure 4 The user can click the browser application icon 31 in the collection details interface 30 to enable the mobile phone to run the browser application in the foreground and display the application interface of the browser application, which includes an input box. The mobile phone can automatically input the document content of the first document into the input box, so that the mobile phone can automatically search for information related to the document content of the first document through the browser application.

[0118] For example, combining Figure 4The user can click the document application icon 32 in the collection details interface 30 to enable the mobile phone to run the document application in the foreground and display the document interface of the document application, which includes the document content of the first document, so that the user can edit the first document through the document application.

[0119] For example, combining Figure 4 The user can click the share icon 33 in the favorites details interface 30 to display the program identifier of at least one third-party application on the favorites details interface 30, such as the program identifier of a chat application or a video application. The user can then click and input the program identifier of the chat application to cause the mobile phone to run the chat application in the foreground and display the application interface of the chat application. The user can then select a contact in the application interface of the chat application to cause the mobile phone to share the first document with the contact.

[0120] For example, combining Figure 4 , the user can click the import control 28 in the favorites details interface 30, so that the mobile phone can display the program identifier of at least one third-party application in the favorites details interface 30, taking the program identifier of a chat application and the program identifier of a video application as examples. Next, the user can click and input the program identifier of the chat application, so that the mobile phone can run the chat application in the foreground and display the application interface of the chat application. The user can select a contact in the application interface of the chat application, so that the mobile phone can display the chat history between the user and the contact. Finally, the user can select the second document required from the chat history, so that the mobile phone can import the second document into the favorites details interface 30.

[0121] In the embodiment of the present application, the electronic device can display the collection details interface of the collection option through the user's input of the collection option, so that the user can view the information related to the program information in real time.

[0122] In some embodiments of the present application, after the above step 202, the information collection method provided by the embodiment of the present application further includes the following steps 401 to 405.

[0123] Step 401: The electronic device receives a first input to a deep dialogue control.

[0124] In some embodiments of the present application, the first input is used by the user to select a deep conversation control from the collection details interface.

[0125] In some embodiments of the present application, the above-mentioned first input can be a click input of the user on the above-mentioned deep dialogue control through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0126] Exemplarily, the first input may be a single-click input of the deep dialogue control by the user.

[0127] Step 402: The electronic device displays a collection dialog interface in response to the first input.

[0128] In some embodiments of the present application, the electronic device may jump from the collection details interface to the collection dialogue interface based on the first input; or, the electronic device may display the collection details interface and the collection dialogue interface in split screen.

[0129] In some embodiments of the present application, the above-mentioned collection dialogue interface may include: a text input box, a voice input control, and an import control.

[0130] For example, in combination Figure 3B ,like Figure 5A As shown, the user can click and input the depth dialogue control 26 in the collection details interface 25, such as Figure 5B As shown, the mobile phone can be displayed in split screen, displaying the collection details interface 25 in the first display area 40 of the mobile phone screen, and displaying the collection dialogue interface 42 in the second display area 41 of the mobile phone screen, and the collection dialogue interface 42 includes a dialogue area 43, a text input box 44, a voice input control 45 and an import control 46.

[0131] Step 403: The electronic device receives the first conversation information input by the user in the collection conversation interface.

[0132] In some embodiments of the present application, the first dialogue information includes first task description information of the first task, and the first task description information lacks at least one task element.

[0133] In some embodiments of the present application, the first task description information may include at least one of the following: an application name and user intention information.

[0134] In some embodiments of the present application, the above-mentioned task elements can be any of the following: navigation destination, document, picture, audio or video, etc., which can be determined according to actual usage, and the embodiments of the present application do not limit it.

[0135] In some embodiments of the present application, the electronic device may receive the first conversation information input by the user by collecting a text input box or a voice input control in the conversation interface.

[0136] In some embodiments of the present application, when a user inputs the first dialogue information through a voice input control, the electronic device may perform text conversion on the acquired audio to obtain the above-mentioned first dialogue information.

[0137] For example, the first conversation information may be: I have an appointment with my best friend to go to this restaurant for dinner later, please help me navigate using B map, where "I have an appointment with my best friend to go to this restaurant for dinner later" is the user intent information, and "B map" is the application name.

[0138] For example, the user can click and input the text input box in the favorite dialogue interface so that the mobile phone can display a keyboard in the favorite dialogue interface. The user can input on the keyboard to obtain the first dialogue message: I have an appointment with my best friend to go to this restaurant for dinner later, please help me navigate with map B; or, the user can long press and input on the voice control in the favorite dialogue interface and say "I have an appointment with my best friend to go to this restaurant for dinner later, please help me navigate with map B", so that the mobile phone can convert the audio into text and obtain the first dialogue message: I have an appointment with my best friend to go to this restaurant for dinner later, please help me navigate with map B.

[0139] Step 404: The electronic device determines default description information of the element based on the collection details information of the program information.

[0140] In some embodiments of the present application, the above-mentioned element default description information is used to describe at least one task element missing in the first task description information.

[0141] In some embodiments of the present application, the above-mentioned element default description information can be any of the following: text of the flight destination, summary information of the document, description information of the picture, audio content information of the audio, or video content information of the video.

[0142] In some embodiments of the present application, the electronic device can input the first conversation information into the first model to determine the user intention through the first model, as well as the information missing from the user intention, that is, the default description information of the above-mentioned elements. At this time, the electronic device can search for the information missing from the user intention from the collection details information of the program information and fill it in the user intention.

[0143] In some embodiments of the present application, the first model may be any one of the following: an artificial intelligence module, a neural network model, or a large language module.

[0144] For example, in combination with the above scenario 1, taking the first conversation information as: I have an appointment with my bestie to go to this restaurant later, please help me use map B to navigate, the mobile phone inputs the first conversation information into the big language module. The big language module obtains the user's intention through semantic analysis to use map B to navigate to a certain place, and the certain place is the default description information of the element. At this time, the mobile phone can search for the place or name from the collection details information corresponding to the restaurant picture, find The boots, and then determine that the user's real intention is: to use map B to navigate to The boots.

[0145] As another example, combined with the above scenario 2, taking the first dialogue information as: Help me use document editing program A to delete the sentence "including forward propagation, backward propagation and other algorithms" in the above document as an example, the mobile phone inputs the first dialogue information into the large language module. The large language module obtains the user's intention through semantic analysis to delete the "including forward propagation, backward propagation and other algorithms" in a certain document through document editing program A. The certain document is the default description information of the element. At this time, the mobile phone can determine the first document from the collection details information corresponding to the first document. At this time, it can be determined that the user's true intention is: Help me use document editing program A to delete the sentence "including forward propagation, backward propagation and other algorithms" in the first document.

[0146] As another example, taking the first dialogue message as: Help me add a weather filter to the above picture using the C image editing program as an example, the mobile phone inputs the first dialogue message into the large language module. The large language module obtains the user's intention through semantic analysis, which is to add a weather filter to a certain image using the C image editing program. The certain image is the default description information of the element. At this time, the mobile phone can determine the first image from the collection details information corresponding to the first image. At this time, it can be determined that the user's true intention is: Help me add a weather filter to the first picture using the C image editing program.

[0147] Step 405: The electronic device executes the first task based on the element default description information and the first task description information.

[0148] In some embodiments of the present application, when the application is a navigation program, the element default description information includes a navigation destination, and the first task description information includes a navigation task element, the first task is to navigate to the navigation destination.

[0149] For example, in combination with the above scenario 1, after the mobile phone determines that the user intends to use B map to navigate to The boots, the mobile phone can call the B map application and automatically input The boots into the B map application so that the B map application can automatically perform navigation.

[0150] When the application is a document editing program, the element default description information includes document content, and the first task description information includes a document editing task element, the first task is to edit the document content using the document editing program.

[0151] For example, in combination with the above scenario 2, when the mobile phone determines that the user intends to delete the sentence "including forward propagation, back propagation and other algorithms" in the first document through document editing program A, the mobile phone can call document editing program A, create a new text document, copy the document content of the first document to the newly created text document, and delete the sentence "including forward propagation, back propagation and other algorithms" in the first document.

[0152] When the application is an image editing program, the element default description information includes an image, and the first task description information includes an image editing task element, the first task is to edit an image using the image editing program.

[0153] For example, when the mobile phone determines that the user intends to add a weather filter to the first image through the C image editing program, the mobile phone can call the C image editing program, copy the first image in the C image editing program, and add a weather filter to the first image.

[0154] In some embodiments of the present application, the electronic device may display the first conversation information and the reply information of the electronic device performing the first task in the message display area in the favorites conversation interface.

[0155] For example, in conjunction with FIG5, as Figure 6 As shown, the mobile phone can display "I have an appointment with my bestie to eat here later, please help me navigate with Map B" in the dialogue area 43 in the favorite dialogue interface 42, and display the reply message "OK, I will open Map B for you in 3 seconds" for the electronic device to perform the first task in the dialogue area 43.

[0156] In some embodiments of the present application, the electronic device can obtain the user's needs through dialogue and automatically perform corresponding operations based on the user's needs without the need for manual operation by the user, thereby improving the flexibility of the electronic device in performing tasks.

[0157] In some embodiments of the present application, before the above step 405, the information collection method provided in the embodiment of the present application further includes the following step 501, and the above step 405 can be specifically implemented through the following step 405a.

[0158] Step 501: The electronic device receives a second input on an application icon among at least one application icon.

[0159] In some embodiments of the present application, the second input is used for the user to select an application icon from at least one application icon.

[0160] In some embodiments of the present application, the above-mentioned second input can be a click input of the user on the above-mentioned application icon through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0161] Exemplarily, the second input may be a click input of the user on one of the at least one application icon.

[0162] It can be understood that the above-mentioned one application icon can be any one of the at least one application icon.

[0163] Step 405a: In response to the second input, the electronic device executes the first task based on the element default description information and the first task description information by the application corresponding to the application icon selected by the second input.

[0164] In some embodiments of the present application, the application icon selected by the user may be the same as or different from the application name included in the first task description information.

[0165] For example, in combination with the above scenario 1, the mobile phone determines that the user intends to use map B to navigate to The Boots, and after the user selects the D map application, the mobile phone can call the D map application and automatically input The Boots into the D map application, so that the D map application can automatically perform navigation.

[0166] As another example, in combination with the above scenario 2, the mobile phone determines that the user intends to delete the sentence "including forward propagation, back propagation and other algorithms" in the first document through the A document editing program, and after the user selects the E document editing program, the mobile phone can call the E document editing program, create a new text document, copy the document content of the first document to the newly created text document, and delete the sentence "including forward propagation, back propagation and other algorithms" in the first document.

[0167] In some embodiments of the present application, the electronic device can obtain the user's needs through dialogue and automatically perform corresponding operations based on the user's selected application, thereby improving the flexibility of the electronic device in performing tasks.

[0168] In some embodiments of the present application, the information collection method provided in the embodiments of the present application further includes the following steps 601 and 602.

[0169] Step 601: The electronic device receives a selection input for a collection source identifier.

[0170] In some embodiments of the present application, the above-mentioned collection source identifier is used to jump to the collection source interface of program information.

[0171] In some embodiments of the present application, the selection input is used to select a collection source identifier from an information collection interface; or, the selection input is used to select a collection source identifier from a collection details interface.

[0172] In some embodiments of the present application, the above-mentioned selection input can be a click input of the user on the above-mentioned application icon through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0173] Step 602: The electronic device displays a program interface including program information in response to a selection input.

[0174] In some embodiments of the present application, the electronic device may run an information source application of the program information in the foreground and display a program interface including the program information.

[0175] For example, in combination Figure 2B ,like Figure 7 As shown, when the user needs to display the source interface corresponding to the restaurant picture 11, the user can click and input the "Shared from A application" mark 23 corresponding to the restaurant picture 11 in the information collection interface 20, that is, the above-mentioned collection source mark, as shown in FIG. Figure 2A As shown, the mobile phone can display a food evaluation interface 10, which includes a restaurant picture 11.

[0176] As another example, when the user needs to display the source interface corresponding to the first document, the user can click and input the "Share from B application" icon corresponding to the first document in the information collection interface, so that the mobile phone can display the document page of the F document editing program, which displays the first document.

[0177] In an embodiment of the present application, the electronic device can predict the user's next intention and display the predicted application identifier in the collection option to facilitate the user's convenient use after finding the collection file, thereby simplifying the processing steps after the electronic device collects the file.

[0178] In some embodiments of the present application, the above-mentioned information collection interface includes at least one collection option.

[0179] For example, Figure 8As shown, the information collection interface 20 displays 5 collection options, and the first collection option 110 of the 5 collection options displays a thumbnail of a food picture. Figure 8 The image 1 shows the collection details corresponding to the collection option 110: Restaurant: The Boots, Address: Kerry Center, Gongshu District, Price per person: 115, navigation program icon, food program icon, sharing control, and "Share from App A" logo and sharing control. The second collection option 111 of the five collection options displays a thumbnail of a hot pot restaurant. Figure 8 The image 2 shows the collection details corresponding to the collection option 111: Restaurant: Donglaishun Hotpot, Address: EFC Square, Yuhang District, Price per person: 100 / person, navigation program icon, food program icon, "Share from B App" logo and sharing control. The third collection option 112 of the five collection options displays a thumbnail of an image of an iron pot stew restaurant. Figure 8 The image 3 shows the collection details corresponding to the collection option 112: Restaurant: Caoyuan Iron Pot Stew, Address: No. 2 Huaihua Road, Villa District, Price per person: 83 / person, navigation program icon, food program icon, "Share from C App" logo and sharing control. The fourth collection option 113 of the five collection options displays a thumbnail of the first document. Figure 8 The image 4 shows the collection details corresponding to the collection option 113: keywords: deep learning, neural network and deep network structure, document editing program icon, and "Share from G App" logo and sharing control. The fifth collection option 114 of the five collection items displays a thumbnail of the audio. Figure 8 This is shown in Figure 5.

[0180] Illustratively, after the above step 202, the information collection method provided in the embodiment of the present application further includes the following steps 701 and 702.

[0181] Step 701: The electronic device receives a favorite merging control input for a second favorite option and a third favorite option in at least one favorite option.

[0182] In some embodiments of the present application, the favorite merging control input is used to select a second favorite option and a third favorite option from at least one favorite option.

[0183] In some embodiments of the present application, the above-mentioned collection merging control input can be a click input of the second collection option and the third collection option by the user through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0184] Exemplarily, the favorite merging control input may be a single-click input of the user on the second favorite option and the third favorite option.

[0185] For example, in combination Figure 8 ,like Figure 9 As shown, if the user needs to merge the second collection option 111 and the third collection option 112 in the information collection interface 20, the user can long press the information collection interface 20 so that the mobile phone can display a selection box before each collection option in the information collection interface 20. At this time, the user can click and input the selection box 29 corresponding to the second collection option 111 and the selection box corresponding to the third collection option 112.

[0186] Step 702: The electronic device displays a fourth collection option on the information collection interface in response to the collection merging control input.

[0187] In some embodiments of the present application, the display area of the above-mentioned fourth collection option includes the first information identifier of the program information corresponding to the second collection option, the collection details information of the program information corresponding to the second collection option, the second information identifier of the program information corresponding to the third collection option, and the collection details information of the program information corresponding to the third collection option.

[0188] For example, in combination Figure 9 ,like Figure 10 As shown, the mobile phone merges the second favorite option 110 and the third favorite option 111 according to the user's click input on the selection box corresponding to the second favorite option 111 and the selection box corresponding to the third favorite option 112, cancels the display of the second favorite option 111 and the third favorite option 112, and displays the thumbnail of the food picture of the second favorite option 110 in the fourth favorite option 120. Figure 10 Image 1 shows the collection details of the second collection option. And the thumbnail of the hot pot restaurant picture of the third collection option 112, Figure 10 It is represented by image 2 and the collection details information corresponding to the collection option 112.

[0189] In some embodiments of the present application, the electronic device can merge multiple favorite options. The merged favorite options are more concentrated, which reduces the time for finding specific content and improves the efficiency of finding favorite options. The merged favorite options are more tidy, and further by merging repeated or similar favorite options, the occupied storage space can be reduced.

[0190] In some embodiments of the present application, after the above step 702, the information collection method provided by the embodiment of the present application further includes the following steps 801 and 802.

[0191] Step 801: The electronic device receives a selection input for a fourth favorite option.

[0192] In some embodiments of the present application, the above-mentioned selection input is the input of the user selecting the fourth collection option in the information collection interface.

[0193] In some embodiments of the present application, the above-mentioned selection input can be a click input of the user on the above-mentioned fourth favorite option through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0194] Exemplarily, the selection input may be a click input for the fourth favorite option.

[0195] Step 802: In response to the selection input, the electronic device displays a collection details interface for the fourth collection option.

[0196] In some embodiments of the present application, the above-mentioned collection details interface includes: information content of the program information corresponding to the second collection option, collection details information of the program information corresponding to the second collection option, information content of the program information corresponding to the third collection option, and collection details information of the program information corresponding to the third collection option.

[0197] For example, in combination Figure 10 ,like Figure 11A As shown, the user can click and input the fourth favorite option 120, as shown in Figure 11B As shown, the mobile phone can display the collection details interface 121 corresponding to the fourth collection item 120. This collection details interface 121 includes the food image of the second collection option and the collection details corresponding to the second collection option: "This hot pot looks pretty good. It's perfect for going to Movie A after eating it." Deep thinking: The address of Donglaishun Hot Pot is extracted from the image and identified as a traditional Beijing hot pot restaurant located at EFC Plaza in Yuhang District. The average price per person is 100. Recommended uses: navigation app icon 21, food app icon, and sharing control. Also included is the image of Caoyuan Iron Pot Stew Restaurant in the third collection option and the collection details of the third collection option: "Iron Pot Stew is also good, with a higher rating." Deep thinking: The address of Caoyuan Iron Pot Stew is extracted from the image and identified as a traditional iron pot stew restaurant with Northeastern characteristics located at No. 2 Huaihua Road in the Villa District. The average price per person is 83. Recommended uses: navigation app icon, food app icon, sharing control. Also included are the deep conversation control, deep thinking control, and import control.

[0198] In some embodiments of the present application, by merging information from multiple sources, the user's management process of multiple applications is simplified, avoiding the tedious steps of opening multiple applications and searching in their respective favorites to find the required information.

[0199] In some embodiments of the present application, after the above step 802, the information collection method provided by the embodiment of the present application further includes the following steps 901 to 906.

[0200] Step 901: The electronic device receives a selection input for a deep dialogue control.

[0201] In some embodiments of the present application, the above-mentioned selection input is used to select the deep dialogue control in the collection details interface of the fourth collection option.

[0202] In some embodiments of the present application, the above-mentioned selection input can be a click input of the user on the above-mentioned deep dialogue control through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0203] Exemplarily, the selection input may be a click input on the deep dialogue control.

[0204] Step 902: The electronic device displays a collection dialog interface in response to the selection input.

[0205] In some embodiments of the present application, the electronic device can jump from the collection details interface to the collection dialogue interface based on the selection input; or, the electronic device can display the collection details interface and the collection dialogue interface in split screen.

[0206] Step 903: The electronic device receives the second conversation information input by the user in the collection conversation interface.

[0207] In some embodiments of the present application, the second dialogue information includes second task description information of the second task, and the second task description information lacks at least one task element information.

[0208] In some embodiments of the present application, the first task description information may include at least one of the following: an application name and user intention information.

[0209] In some embodiments of the present application, the electronic device may receive the second conversation information input by the user by collecting a text input box or a voice input control in the conversation interface.

[0210] For example, the second conversation message may be: Please help me arrange a plan based on Xiaomei's appointment tomorrow afternoon.

[0211] Step 904: The electronic device displays the deep thinking information on the collection conversation interface.

[0212] In some embodiments of the present application, the above-mentioned deep thinking information includes descriptive information of the thinking process and thinking result information obtained through deep thinking based on the collection details information of the fourth collection option and the second task description information; the thinking result information includes at least one recommended plan recommendation option; each plan option in the at least one plan recommendation option includes keywords in the collection details information of the fourth collection option and at least one application icon.

[0213] For example, in combination Figure 11B ,like Figure 12A As shown, the user can click and input the deep dialogue control 15 in the collection details interface 31 corresponding to the fourth collection item 30, so that the mobile phone can display the collection dialogue interface 50, as shown in FIG. Figure 12B As shown, the user can then long-press the voice input control in the favorites dialog interface 121 and say the voice message "Tomorrow afternoon, I have a date with Xiaomei. Please help me arrange my plans." This allows the phone to perform voice recognition and display the voice recognition text "Tomorrow afternoon, I have a date with Xiaomei. Please help me arrange my plans." in the message display area of the favorites dialog interface. The user can then invoke the large language model for deep thinking and display the descriptive information of the thought process obtained from the deep thinking: "Deep thinking: The user mentioned two restaurants but planned an afternoon trip. However, they later stated that they felt hot pot had a better rating, so they recommended hot pot for dinner. They also mentioned wanting to watch Nezha 2. There's a movie theater upstairs from the hot pot restaurant. The comprehensive reasoning result is as follows:" and the plan recommendation option "Okay, combining the favorites, I'll arrange your plans as follows: 1. Donglaishun Hot Pot for dinner, 2. Watch Nezha 2 after dinner," along with the navigation app icon, food review app icon, and sharing app icon corresponding to option 1; and the movie app icon, food review app icon, and sharing app icon corresponding to option 2.

[0214] Step 905: The electronic device receives a selection input for one of the at least one application icon.

[0215] In some embodiments of the present application, the selection input is used to select an application icon from at least one application icon.

[0216] In some embodiments of the present application, the above-mentioned selection input can be a click input of one of the above-mentioned at least one application icon by the user through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0217] Exemplarily, the selection input may be a click input of a user selecting an application icon from at least one application icon.

[0218] Step 906: The electronic device executes the second task in response to the selection input by using the application corresponding to the application icon selected by the selection input.

[0219] In some embodiments of the present application, the second task is a task determined based on a plan option and an application program type corresponding to the application icon selected by the selection input.

[0220] Exemplarily, when the application program type is a navigation application, the second task is to navigate to Donglaishun Hotpot.

[0221] As another example, in the case where the application program type is a movie application, the second task is to reserve a movie ticket for Nezha A in the evening.

[0222] In some embodiments of the present application, the electronic device can obtain the user's needs through dialogue, and automatically determine and execute corresponding operations based on the user's selected application, thereby improving the flexibility of the electronic device in performing tasks.

[0223] In some embodiments of the present application, the above-mentioned information collection interface also includes a fifth collection option, which is obtained by merging the first collection option, the second collection option and the third collection option; the display area of the fifth collection option includes the information identifier and collection details information of the program information corresponding to the first collection option and the second collection option, and the information identifier and collection details information of the program information corresponding to the third collection option are hidden; the display area of the fifth collection option also includes an expansion control.

[0224] Illustratively, the information collection method provided in the embodiment of the present application further includes the following steps 1001 and 1002.

[0225] Step 1001: The electronic device receives a selection input for an expansion control.

[0226] In some embodiments of the present application, the selection input is used to select an expansion control from the information collection interface.

[0227] In some embodiments of the present application, the above-mentioned selection input can be a click input of the user on the above-mentioned expansion control through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0228] Exemplarily, the selection input may be a click input of the user on the expansion control.

[0229] Step 1002 : In response to a selection input, the electronic device displays, in a display area for a fifth favorite option, an information identifier and favorite details of the program information corresponding to the third favorite option.

[0230] For example, in combination Figure 8 ,like Figure 13A As shown, the fifth collection option 51 in the information collection interface 20 displays the collection details information of the first collection option and the collection details information of the second collection option, as well as an expansion control, such as Figure 13B As shown, the user can click and input the expansion control, so that the mobile phone can display the collection detail information of the third collection option in the fifth collection option 51.

[0231] In some embodiments of the present application, the electronic device can display the favorite options hidden in the fifth favorite option based on the user's input, thereby improving the flexibility of the electronic device in displaying the favorite options.

[0232] In some embodiments of the present application, the second favorite option corresponds to the first position identifier and the first page position identifier, and the third favorite option corresponds to the second position identifier and the second page position identifier.

[0233] Illustratively, the information collection method provided in the embodiment of the present application further includes the following steps 1003 to 1007.

[0234] Step 1003: The electronic device receives a selection input for the first information identifier.

[0235] In some embodiments of the present application, the selection input is used to select a first information identifier from a first favorite option.

[0236] In some embodiments of the present application, the above-mentioned selection input can be a click input of the user on the above-mentioned first information identifier through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0237] Exemplarily, the selection input may be a click input of the user on the first information identifier.

[0238] As another example, the first information identifier may be a keyword of the program information displayed in the first favorites option.

[0239] Step 1004: The electronic device determines a favorite option corresponding to the first location identifier in response to the selection input.

[0240] In some embodiments of the present application, the electronic device may determine the favorite option corresponding to the first location identifier based on a stored correspondence between the first location identifier and the favorite option.

[0241] Step 1005: The electronic device determines, based on the first page position identifier, a display position of the program information and the collection detail information of the second collection option in the collection detail interface of the fourth collection option.

[0242] In some embodiments of the present application, the electronic device can determine the display position of the program information and collection details information of the second collection option in the collection details interface of the fourth collection option based on the correspondence between the stored first page location identifier and the collection details information of the second collection option and the program information of the second collection option.

[0243] Step 1006: The electronic device displays the program information and favorite details information of the second favorite option at the top of the favorite details interface of the fifth favorite option based on the display position.

[0244] For example, in combination Figure 13B ,like Figure 14A As shown, the user can click and input "Donglaishun Hotpot" in the second favorite option, such as Figure 14B As shown, the mobile phone can display the program information of the second favorite option and the favorite details information of the second favorite option at the top in the favorite details interface of the fifth favorite option.

[0245] In some embodiments of the present application, the electronic device can quickly locate the program information and collection details information of the collection option that the user needs to view in the collection details interface based on the first position identifier and the first page position identifier corresponding to the collection option, without the user having to manually search in the collection details interface, thereby improving the efficiency of the electronic device in searching for the program information and collection details information of the collection option.

[0246] In some embodiments of the present application, the above-mentioned information collection interface includes a sixth collection option.

[0247] Illustratively, the information collection method provided in the embodiment of the present application further includes the following steps 1100 to 1103.

[0248] Step 1100: The electronic device receives a selection input for a sixth favorite option.

[0249] In some embodiments of the present application, the above-mentioned selection input is used to select the sixth collection option from the information collection interface.

[0250] In some embodiments of the present application, the above-mentioned selection input can be a click input of the user on the above-mentioned sixth favorite option through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0251] Exemplarily, the selection input may be a single-click input of the user for the sixth favorite option.

[0252] Step 1101: In response to a selection input, the electronic device displays a favorites editing interface for a sixth favorites option.

[0253] In some embodiments of the present application, the above-mentioned collection editing interface includes information content of program information, deep dialogue controls, deep thinking controls and import controls; the collection editing interface is used to edit the interface content of the collection details interface of the sixth collection option.

[0254] For example, in combination Figure 8 ,like Figure 15A As shown, the user can click and input the favorite option 114, as shown in Figure 15B As shown, the mobile phone can display the collection editing interface 52 corresponding to the collection option 114, and the editing interface includes a deep dialogue control 26, a deep thinking control 27 and an import control 28.

[0255] Step 1102: The electronic device receives a third input to the deep thinking control.

[0256] In some embodiments of the present application, the third input is used to select a deep thinking control from the favorites editing interface of the sixth favorites option.

[0257] In some embodiments of the present application, the above-mentioned third input can be a click input of the user on the above-mentioned deep thinking control through a touch device such as a finger or a stylus, or a voice command input by the user, or a specific gesture input by the user, or other feasible input. The specific input can be determined according to actual usage requirements and is not limited in the embodiments of the present application.

[0258] Exemplarily, the third input may be a single-click input of the deep thinking control by the user.

[0259] Step 1103: The electronic device displays a collection details interface in response to the third input.

[0260] In some embodiments of the present application, the above-mentioned collection details interface includes collection details information of the sixth collection option.

[0261] For example, in combination Figure 15B ,like Figure 16As shown, the user can click and input the deep thinking control so that the mobile phone can display the collection details information corresponding to the sixth collection option. The collection details information includes: Let’s go to Shahe Avenue Skating Rink today, it’s not expensive, 50 per person; Deep Thinking: Address Shahe Avenue Skating Rink, 50 per person, recommended use: navigation program icon, sharing control.

[0262] In some embodiments of the present application, after the electronic device obtains the collection detail information corresponding to the sixth collection option, the electronic device may update the sixth collection option according to the collection detail information corresponding to the sixth collection option.

[0263] For example, in combination Figure 16 ,like Figure 17 As shown, the user can trigger the mobile phone to jump from the collection details interface 52 of the sixth collection option to the information collection interface 20, where the sixth collection option 114 in the information collection interface 20 includes: Shahe Avenue Skating Rink, 50 / person, navigation application icon, sharing application icon and "Share from F application".

[0264] In some embodiments of the present application, the electronic device can manually trigger the electronic device to obtain the collection details of the collection option based on the user's input, thereby improving the flexibility of the electronic device in obtaining the collection details of the collection option.

[0265] In some embodiments of the present application, after the above step 1101, the information collection method provided by the embodiment of the present application further includes the following steps 1110 and 1111.

[0266] Step 1110: The electronic device receives the remark information input by the user in the information editing area of the favorites editing interface.

[0267] In some embodiments of the present application, the above-mentioned favorites editing interface includes a text input box or a voice control, so that the user can enter note information in the favorites editing interface through the text input box or voice control.

[0268] Step 1111: The electronic device updates the note information to the collection details interface of the sixth collection option.

[0269] For example, in combination Figure 2B ,like Figure 18A As shown, the user long presses the favorite option 13 to make the mobile phone display the favorite editing interface 54 corresponding to the favorite option 13. The favorite editing interface 54 includes a text input box 55 and a voice input box 56. The user long presses the voice input box 56 to make an input and says "My bestie said this restaurant is not bad". Figure 18BAs shown, the mobile phone can update the text corresponding to the voice "My bestie said this restaurant is not bad" to the collection details interface 25 of the collection option 13.

[0270] In some embodiments of the present application, the electronic device can update the remark information entered by the user in the favorites editing interface to the favorites details interface, thereby improving the convenience of the electronic device in editing the favorites details information.

[0271] In some embodiments of the present application, the information collection method provided in the embodiments of the present application further includes the following steps 1120 to 1123.

[0272] Step 1120: The electronic device inputs the program information into the multimodal reasoning model.

[0273] In some embodiments of the present application, the multimodal reasoning model includes a file encoder and a deep reasoning unit.

[0274] In some embodiments of the present application, the above-mentioned file encoder may include a visual encoder, an audio encoder, and a text encoder.

[0275] In some embodiments of the present application, the above-mentioned deep reasoning unit can be any of the following: a large language model, a neural network model, or an artificial intelligence model, etc. The specific method can be determined according to actual use requirements and is not limited by the embodiments of the present application.

[0276] For example, Figure 19 As shown, the multimodal reasoning model 60 may include a visual encoder 61 , an audio encoder 62 , and a text encoder 63 , wherein the visual encoder 61 , the audio encoder 62 , and the text encoder 63 are respectively connected to a deep reasoning unit 64 .

[0277] In some embodiments of the present application, when the program information is a picture or a video, the electronic device may input the picture or video into a visual encoder; or,

[0278] In the case where the program information is a document or text, the electronic device may input the document or text into a text encoder; or,

[0279] In the case where the program information is audio, the electronic device may input the audio into an audio encoder.

[0280] Step 1121: The electronic device extracts characteristic information of the program information through a file encoder.

[0281] In some embodiments of the present application, the electronic device may perform convolution processing on the program information through a file encoder to obtain feature information of the program information.

[0282] In some embodiments of the present application, when the program information is an image, the above-mentioned feature information includes at least one of the following: image content description information and text recognition information; wherein, the image content description information is used to describe the image content of the image, and the text recognition information is text information obtained by recognizing text in the image.

[0283] In the case where the program information is a document, the characteristic information includes at least one of the following: document summary information, document keywords, and document title; wherein the document summary information is used to summarize the document content of the document.

[0284] When the program information is a video, the characteristic information includes at least one of the following: video content description information, subtitle information of each video frame of the video, and video frame content information of each video frame of the video; wherein the video content description information is used to describe the video content of the video.

[0285] In the case where the program information is audio, the above-mentioned feature information includes at least one of the following: text summary information of the audio and audio content description information of the audio, wherein the text summary information is used to summarize the audio content of the audio, and the audio content description information is used to describe the audio content of the audio.

[0286] As another example, the above-mentioned feature information includes but is not limited to text extraction from images, recognition of addresses, extraction of link information, extraction of conversation content in audio, etc. The extraction of information presets multiple keys in advance to normalize the output, and makes it output in json format, so as to ensure that the information to be extracted is comprehensive enough without omissions, and standardized. For all types of data, a description is required, that is, an overall description. For image format files, text recognition results are required. For audio formats, speech content needs to be extracted, and so on. Of course, some other optional keys can also be added, which can be defined in advance. If it is involved in the collection file, the corresponding content will be returned. If it is not involved, Null will be returned to avoid hallucinations that affect subsequent processing.

[0287] For example, to parse the image content, use the prompt: "This is a user-provided image. Return it according to the given JSON structure. For keys where no information can be extracted, return Null." The returned result is as follows:

[0288]

[0289] For user-collected web pages, their contents will first be obtained through the Internet, and then input into the model for inference to obtain the results. The input format is similar to the JSON requirements for images.

[0290] For voice data, the text information after text conversion is obtained, and then the entire audio is described to obtain JSON data with a structure similar to the above.

[0291] It can be understood that the design of the json format can ensure that the important features are extracted and no important information is missed. At the same time, mixed input can better ensure the stability and quality of the subsequent model output results.

[0292] It should be noted that the multimodal reasoning model can automatically reason about the entire program information. For parts such as links that require network access, the content can be obtained online and then returned to the client for reasoning, thus protecting user privacy. Optionally, the electronic device can also send all data to the multimodal reasoning model in the server.

[0293] Step 1122: The electronic device uses a deep reasoning unit to predict intent based on the feature information and obtains at least one application that matches the feature information.

[0294] In an embodiment of the present application, the electronic device can use the intent label in the multimodal reasoning model to perform intent prediction based on the extracted feature information and obtain at least one application that matches the feature information.

[0295] In some embodiments of the present application, the above-mentioned intention tags may be preset; or user-defined.

[0296] For example, the output format is specified as json format, set to

[0297]

[0298] The conversation structure is a list with two roles: user and assistant. This structure supports multiple rounds of conversations, allowing users to dynamically adjust the model output based on their needs. The app_list retrieves publicly accessible software installed on the phone in real time, making it easy to provide users with quick options.

[0299] The model output contains various special tokens, which are used to call the underlying application and are then displayed to the user after front-end rendering. For example, in the above case, the model output is:

[0300]

[0301] in <app>< / app> It is a special token used to mark the top three apps in the app recommendation list.<app_list>< / app_list> To supplement possible other options.

[0302] In some embodiments of this application, when obtaining relevant information corresponding to a merged favorite, the multimodal reasoning model needs to add two new keywords: "collect_index" to mark which favorite the current input comes from, and "page_location" to locate the relative position of the current favorite on the details page for subsequent processing. The input format is as follows:

[0303]

[0304] Step 1123: The electronic device outputs collection detail information based on the feature information and at least one application.

[0305] In some embodiments of the present application, after obtaining the feature information and at least one application, the electronic device may sequentially output the feature information and at least one application, ie, the above-mentioned collection detail information.

[0306] In some embodiments of the present application, after the electronic device outputs the collection details information through the multimodal reasoning model, the electronic device may display the collection details information in the collection interface; or may display the collection details information in the collection conversation interface. For details, please refer to the above embodiments, and to avoid repetition, it will not be repeated here.

[0307] For example, for the above-mentioned conversation with the user and execution of the first task, the electronic device can put the relevant information into the conversations and then splice it with the user input. User input supports multiple formats, including but not limited to audio, text, pictures, etc. The content of the user's question will be converted into a dictionary format (the dictionary includes input content [content], input file format [type, optional audio-audio, text-text, picture-image, etc.], and role [because it is user input, so role = "user"]), and then spliced into the previous conversations.

[0308] The format is as follows:

[0309]

[0310]

[0311] Each input and output is a new text description prompt word added to the conversation list. To distinguish, the user's question role is user, while the model's response, when fed back into the model as the previous conversation, has the role of assistant. This allows the model to consider and meet user needs based on the user input and the model's response from the previous round.

[0312] In some embodiments of the present application, the information collection method provided by the embodiments of the present application further includes the following step 1130.

[0313] Step 1130: The electronic device inputs the training mark into the multimodal reasoning model for training and outputs collection details.

[0314] In some embodiments of the present application, the above-mentioned training mark includes at least one of the following: an application name mark, an application list mark, an application call mark, a placeholder mark, an audio content description mark, and a picture content description mark.

[0315] For example, the above application name is marked as <app>< / app> Used to contain the name of the application actually installed by the user, to facilitate association and provide an icon for the user to quickly click.

[0316] The above application list is marked as<app_list>< / app_list> Used to contain other applications that are not the highest priority and can be shared.

[0317] The above application calls are marked as<function app=”app_name”> Used for subsequent function call functions to quickly call up the application and execute the corresponding command function, such as setting an alarm at 10 o'clock.

[0318] The above placeholders are marked as Used to mark the position of the picture in the text, placeholder; <audio>Used to mark the position of audio in the text, placeholder.

[0319] The above audio content description is marked as<audio_sec>< / audio_sec> , the content description and related reasoning content obtained after reasoning on the audio.

[0320] The above picture content description is marked as<image_sec>< / image_sec> ,The content description and related reasoning content obtained after reasoning on the image.

[0321] In some embodiments of the present application, the above-mentioned training data may also include at least one of the following: for audio data, the basic data needs to collect audio data of various scenes and descriptions of the audio content, collect human voice samples, and annotate the speakers and speech content. Other types of audio data can be used as the basis for training.

[0322] For image data, you need to prepare general image recognition data, including but not limited to OCR, image description, object detection and other data.

[0323] For text data, a large amount of inference or summary data needs to be prepared.

[0324] In some embodiments of the present application, the electronic device trains the multimodal reasoning model based on the above-mentioned training data.

[0325] For example, the model is trained in two stages. First, the encoders in the multimodal reasoning model are trained, leveraging open-source data from the internet and in-house data from relevant business scenarios to enhance their general capabilities. Second, the training parameters of the previously trained encoders are frozen, and the focus is on training the large language model for deep reasoning. This ensures the stability of the basic capabilities while simultaneously improving the model's reasoning capabilities. This involves distilling the model with inference data generated by DeepSeek to enhance its reasoning capabilities.

[0326] In the embodiment of the present application, after integrating the inference effect of the large model, the electronic device returns a more intelligent result to the user, eliminating the need for the user to extract and edit important content, saving the user time and energy, and providing better results. Traditional image retrieval solutions are based on image classifiers, and their effectiveness depends on the data categories pre-defined during the model training phase. This results in poor results for image types that the model has never seen, and lacks a divergent function. The present invention, based on deep thinking, will compare the file content of the collection file with the user's search terms to improve the accuracy of recall.

[0327] The above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0328] It should be noted that the information collection method provided in the embodiment of the present application can be executed by an information collection device. In the embodiment of the present application, the information collection device performing the information collection method is taken as an example to illustrate the information collection device provided in the embodiment of the present application.

[0329] Figure 20 A possible structural diagram of the information collection device involved in the embodiment of the present application is shown. Figure 20 As shown, the information collection device 70 may include: a receiving module 71 and a display module 72.

[0330] The receiving module 71 is configured to receive a collection input for program information in the program interface, where the program information includes at least one of the following: a file or a link. The display module 72 is configured to display an information collection interface in response to the collection input received by the receiving module 71. The information collection interface includes a first collection option corresponding to the program information, where the first collection option includes an information identifier for the program information and collection details of the program information. The collection details in the information collection interface include at least one of the following: a keyword for the program information, at least one application icon, a collection source identifier, and a sharing control. The at least one application icon is an icon of an application that matches the user's intent, where the user's intent is predicted based on the keyword.

[0331] In one possible implementation, the receiving module 71 is further configured to receive a selection input for a first favorite option corresponding to the program information after the display module 72 displays the information favorites interface. The display module 72 is further configured to display a favorites details interface for the first favorite option in response to the selection input received by the receiving module 71. The favorites details interface includes: program information content, favorites details information for the program information, a deep conversation control, a deep thinking control, and an import control. The favorites details information in the favorites details interface includes at least one of the following: program information overview information, program information keywords, at least one application icon, a favorites source identifier, and a sharing control.

[0332] In a possible implementation, the information collection device 70 provided in the embodiment of the present application further includes: a processing module. The above-mentioned receiving module 71 is also used to receive a first input to the deep dialogue control after the display module 72 displays the collection details interface of the first collection option. The above-mentioned display module 72 is also used to display the collection dialogue interface in response to the first input received by the receiving module. The above-mentioned receiving module 71 is also used to receive the first dialogue information input by the user in the collection dialogue interface; the first dialogue information includes the first task description information of the first task, and the first task description information lacks at least one task element. The processing module is used to determine the element default description information based on the collection details information of the program information, and the element default description information is used to describe at least one task element missing in the first task description information; and execute the first task based on the element default description information and the first task description information.

[0333] In one possible implementation, the receiving module 71 is further configured to receive a second input for one of the at least one application icons before the processing module executes the first task based on the default element description information and the first task description information. The processing module is specifically configured to, in response to the second input received by the receiving module, cause the application corresponding to the application icon selected by the second input to execute the first task based on the default element description information and the first task description information.

[0334] In a possible implementation, the receiving module 71 is further configured to receive a selection input for a collection source identifier. The display module 72 is further configured to display a program interface including program information in response to the selection input received by the receiving module 71 .

[0335] In one possible implementation, the information collection interface includes at least one collection option; the receiving module 71 is further configured to, after displaying the information collection interface, receive a collection merging control input for a second collection option and a third collection option in the at least one collection option. The display module 72 is further configured to, in response to the collection merging control input received by the receiving module, display a fourth collection option on the information collection interface, wherein the display area for the fourth collection option includes a first information identifier for the program information corresponding to the second collection option, collection details information for the program information corresponding to the second collection option, a second information identifier for the program information corresponding to the third collection option, and collection details information for the program information corresponding to the third collection option.

[0336] In one possible implementation, receiving module 71 is further configured to receive a selection input for the fourth favorite option after display module 72 displays the fourth favorite option on the information favorites interface. Display module 72 is further configured to, in response to the selection input, display a favorite details interface for the fourth favorite option; the favorite details interface includes: information content of the program information corresponding to the second favorite option, favorite details information of the program information corresponding to the second favorite option, information content of the program information corresponding to the third favorite option, and favorite details information of the program information corresponding to the third favorite option.

[0337] In one possible implementation, the information collection device 70 provided in an embodiment of the present application further includes: a processing module. The receiving module 71 is further configured to receive a selection input for a deep conversation control after displaying the collection details interface corresponding to the fourth collection item. The display module 72 is further configured to display the collection conversation interface in response to the selection input received by the receiving module 71. The receiving module 71 is further configured to receive second conversation information entered by the user in the collection conversation interface, the second conversation information including second task description information for a second task, the second task description information lacking at least one task element information. The display module 72 is configured to display deep thinking information in the collection conversation interface; the deep thinking information includes descriptive information and thinking result information obtained through deep thinking based on the collection details information and second task description information of the fourth collection option; the thinking result information includes at least one recommended plan option; each plan option includes keywords from the collection details information of the fourth collection option and at least one application icon. The receiving module 71 is further configured to receive a selection input for one of the at least one application icon. The above-mentioned processing module is used to respond to the selection input received by the receiving module 71, and execute the second task by selecting the application corresponding to the application icon selected by the selection input; wherein, the second task is a task determined based on the plan option and application program type corresponding to the application icon selected by the selection input.

[0338] In one possible implementation, the information collection interface further includes a fifth collection option, which is obtained by combining the first, second, and third collection options. The display area of the fifth collection option includes information identifiers and collection details of the program information corresponding to the first and second collection options, while the information identifier and collection details of the program information corresponding to the third collection option are hidden. The display area of the fifth collection option also includes an expansion control. The receiving module 71 is further configured to receive a selection input for the expansion control. The display module 72 is further configured to, in response to the selection input, display the information identifier and collection details of the program information corresponding to the third collection option in the display area of the fifth collection option.

[0339] In a possible implementation, the information collection device 70 provided in the embodiment of the present application further includes: a determination module; the second collection option corresponds to the first position identifier and the first page position identifier, and the third collection option corresponds to the second position identifier and the second page position identifier; the above-mentioned receiving module 71 is also used to receive a selection input for the first information identifier. The above-mentioned determination module is used to determine the collection item corresponding to the first position identifier in response to the selection input received by the receiving module 71; and based on the first page position identifier, determine the display position of the program information and collection details information of the second collection option in the collection details interface of the fourth collection option. The above-mentioned display module 72 is also used to display the program information and collection details information of the second collection option at the top of the collection details interface of the fifth collection option based on the display position.

[0340] In one possible implementation, the information collection interface includes a sixth collection option. The receiving module 71 is further configured to receive selection input for the sixth collection option. The display module 72 is further configured to, in response to the selection input received by the receiving module 71, display a collection editing interface for the sixth collection option. The collection editing interface includes program information content, a deep conversation control, a deep thinking control, and an import control. The collection editing interface is used to edit the interface content of the collection details interface for the sixth collection option. The receiving module 71 is further configured to receive a third input for the deep thinking control. The display module 72 is further configured to display a collection details interface in response to the third input received by the receiving module 71.

[0341] In one possible implementation, the information collection device 70 provided in an embodiment of the present application further includes an updating module. The receiving module 71 is further configured to receive a user's note entered in the information editing area of the collection editing interface after the display module 72 displays the collection editing interface for program information. The updating module updates the note to the collection details interface for the sixth collection option.

[0342] In one possible implementation, the information collection device 70 provided in an embodiment of the present application further includes: an input module, an extraction module, a processing module, and an output module. The input module is configured to input program information into a multimodal reasoning model, which includes a file encoder and a deep reasoning unit. The extraction module is configured to extract feature information of the program information using the file encoder. The processing module is configured to perform intent prediction based on the feature information using the deep reasoning unit to obtain at least one application that matches the feature information. The output module is configured to output collection details based on the feature information and the at least one application.

[0343] In one possible implementation, when the program information is an image, the feature information includes at least one of the following: image content description information and text recognition information; wherein the image content description information is used to describe the image content of the image, and the text recognition information is text information obtained by recognizing text in the image;

[0344] In the case where the program information is a document, the characteristic information includes at least one of the following: document summary information, document keywords, and document title; wherein the document summary information is used to summarize the document content of the document;

[0345] In the case where the program information is a video, the characteristic information includes at least one of the following: video content description information, subtitle information of each video frame of the video, and video frame content information of each video frame of the video; wherein the video content description information is used to describe the video content of the video;

[0346] When the program information is audio, the feature information includes at least one of the following: text summary information of the audio and audio content description information of the audio. The text summary information is used to summarize the audio content of the audio, and the audio content description information is used to describe the audio content of the audio.

[0347] In one possible implementation, the above-mentioned processing module is also used to input training tags into the multimodal reasoning model for training and output collection detail information; wherein the training tags include at least one of the following: application name tag, application list tag, application call tag, placeholder tag, audio content description tag and image content description tag.

[0348] In a possible implementation, when the application is a navigation application, the element default description information includes a navigation destination, and the first task description information includes a navigation task element, the first task is to navigate to the navigation destination;

[0349] When the application is a document editing program, the element default description information includes document content, and the first task description information includes a document editing task element, the first task is to edit the document content using the document editing program;

[0350] When the application is an image editing program, the element default description information includes an image, and the first task description information includes an image editing task element, the first task is to edit an image using the image editing program.

[0351] An embodiment of the present application provides an information collection device. Since the information collection interface already includes a first collection option corresponding to program information, it can be understood that when collecting program information, the program information can be deeply processed to achieve comprehensive content extraction of the program information, and the extracted content can be displayed in the information collection interface, thereby avoiding the need for users to perform additional editing operations when editing the program information; and the user's next intention can be predicted, which is convenient for users to use after finding the first collection option corresponding to the program information, thereby simplifying the processing steps of the information collection device after collecting the program information, significantly improving the user's usage experience and the efficiency of the information collection device in file editing.

[0352] The information collection device in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or chip. The electronic device can be a terminal or other device other than a terminal. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiments of the present application are not specifically limited.

[0353] The information collection device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0354] The information collection device provided in the embodiment of the present application can implement each process implemented in the above embodiment. To avoid repetition, it will not be repeated here.

[0355] Alternatively, as Figure 21 As shown, an embodiment of the present application further provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores a program or instruction that can be run on the processor 91, and when the program or instruction is executed by the processor 91, the various steps of the above-mentioned information collection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0356] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0357] Figure 22 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0358] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101 , a network module 102 , an audio output unit 103 , an input unit 104 , a sensor 105 , a display unit 106 , a user input unit 107 , an interface unit 108 , a memory 109 , and a processor 110 .

[0359] Those skilled in the art will understand that the electronic device 100 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 22 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0360] The user input unit 107 is configured to receive a collection input for program information in the program interface, where the program information includes at least one of the following: a file or a link. The display unit 106 is configured to display an information collection interface in response to the collection input. The information collection interface includes a first collection option corresponding to the program information, where the first collection option includes an information identifier for the program information and collection details for the program information. The collection details in the information collection interface include at least one of the following: a keyword for the program information, at least one application icon, a collection source identifier, and a sharing control, where at least one application icon is an icon of an application that matches the user's intent. The user's intent is predicted based on the keyword.

[0361] In some embodiments of the present application, the user input unit 107 is further configured to receive a selection input for a first favorite option corresponding to the program information after displaying the information favorites interface. The display unit 106 is further configured to display a favorites details interface for the first favorite option in response to the selection input; the favorites details interface includes: the program information content, favorites details of the program information, a deep conversation control, a deep thinking control, and an import control. The favorites details information in the favorites details interface includes at least one of the following: a summary of the program information, keywords for the program information, at least one application icon, a favorites source identifier, and a sharing control.

[0362] In some embodiments of the present application, the user input unit 107 is further used to receive a first input to the deep dialogue control after displaying the collection details interface of the first collection option. The display unit 106 is further used to display the collection dialogue interface in response to the first input. The user input unit 107 is further used to receive first dialogue information input by the user in the collection dialogue interface; the first dialogue information includes first task description information of the first task, and the first task description information lacks at least one task element. The processor 110 is further used to determine element default description information based on the collection details information of the program information, the element default description information is used to describe at least one task element missing from the first task description information; and execute the first task based on the element default description information and the first task description information.

[0363] In some embodiments of the present application, the user input unit 107 is further configured to receive a second input for one of the at least one application icons before executing the first task based on the default element description information and the first task description information. The processor 110 is specifically configured to, in response to the second input, cause the application corresponding to the application icon selected by the second input to execute the first task based on the default element description information and the first task description information.

[0364] In some embodiments of the present application, the user input unit 107 is further configured to receive a selection input for a favorite source identifier. The display unit 106 is further configured to display a program interface including program information in response to the selection input.

[0365] In some embodiments of the present application, the information collection interface includes at least one collection option; the user input unit 107 is further configured to, after displaying the information collection interface, receive a collection merging control input for a second collection option and a third collection option in the at least one collection option. The display unit 106 is further configured to, in response to the collection merging control input, display a fourth collection option on the information collection interface, wherein the display area for the fourth collection option includes a first information identifier for the program information corresponding to the second collection option, collection details information for the program information corresponding to the second collection option, a second information identifier for the program information corresponding to the third collection option, and collection details information for the program information corresponding to the third collection option.

[0366] In some embodiments of the present application, the user input unit 107 is further configured to receive a selection input for the fourth favorite option after the fourth favorite option is displayed on the information favorites interface. The display unit 106 is further configured to display a favorites details interface for the fourth favorite option in response to the selection input; the favorites details interface includes: information content of the program information corresponding to the second favorite option, favorites details information of the program information corresponding to the second favorite option, information content of the program information corresponding to the third favorite option, and favorites details information of the program information corresponding to the third favorite option.

[0367] In some embodiments of the present application, the user input unit 107 is further configured to receive a selection input for the deep conversation control after displaying the collection details interface corresponding to the fourth collection item. The display unit 106 is further configured to display the collection conversation interface in response to the selection input. The user input unit 107 is further configured to receive second conversation information input by the user in the collection conversation interface, the second conversation information including second task description information for a second task, the second task description information lacking at least one task element information. The display unit 106 is further configured to display deep thinking information in the collection conversation interface; the deep thinking information includes descriptive information and thinking result information obtained through deep thinking based on the collection details information and second task description information of the fourth collection option; the thinking result information includes at least one recommended plan option; each plan option includes keywords and at least one application icon from the collection details information of the fourth collection option. The user input unit 107 is further configured to receive a selection input for one of the at least one application icon. The processor 110 is further configured to execute a second task in response to a selection input by executing an application corresponding to the application icon selected by the selection input; wherein the second task is a task determined based on the plan option and application program type corresponding to the application icon selected by the selection input.

[0368] In some embodiments of the present application, the information collection interface further includes a fifth collection option, which is obtained by combining the first, second, and third collection options. The display area of the fifth collection option includes information identifiers and collection details of the program information corresponding to the first and second collection options, while the information identifier and collection details of the program information corresponding to the third collection option are hidden. The display area of the fifth collection option further includes an expansion control. The user input unit 107 is further configured to receive a selection input for the expansion control. The display unit 106 is further configured to display, in response to the selection input, the information identifier and collection details of the program information corresponding to the third collection option in the display area of the fifth collection option.

[0369] In some embodiments of the present application, the second favorite option corresponds to the first location identifier and the first page location identifier, and the third favorite option corresponds to the second location identifier and the second page location identifier; the user input unit 107 is further configured to receive a selection input for the first information identifier. The processor 110 is further configured to, in response to the selection input, determine the favorite item corresponding to the first location identifier, and, based on the first page location identifier, determine the display position of the program information and favorite details information of the second favorite option in the favorite details interface of the fourth favorite option. The display unit 106 is further configured to, based on the display position, display the program information and favorite details information of the second favorite option at the top of the favorite details interface of the fifth favorite option.

[0370] In some embodiments of the present application, the information collection interface includes a sixth collection option; the user input unit 107 is further configured to receive a selection input for the sixth collection option. The display unit 106 is further configured to, in response to the selection input, display a collection editing interface for the sixth collection option. The collection editing interface includes program information content, a deep conversation control, a deep thinking control, and an import control. The collection editing interface is used to edit the interface content of the collection details interface for the sixth collection option. The user input unit 107 is further configured to receive a third input for the deep thinking control. The display unit 106 is further configured to display a collection details interface in response to the third input.

[0371] In some embodiments of the present application, the user input unit 107 is further configured to receive a user-entered note in the information editing area of the favorites editing interface after displaying the favorites editing interface for the program information. The processor 110 is further configured to update the note to the favorites details interface of the sixth favorites option.

[0372] In some embodiments of the present application, the above-mentioned processor 110 is also used to input program information into a multimodal reasoning model, and the multimodal reasoning model includes a file encoder and a deep reasoning unit; and extract feature information of the program information through the file encoder; and perform intent prediction based on the feature information through the deep reasoning unit to obtain at least one application matching the feature information; and output collection detail information based on the feature information and at least one application.

[0373] In some embodiments of the present application, the processor 110 is further configured to input training tags into a multimodal reasoning model for training and output collection detail information; wherein the training tags include at least one of the following: an application name tag, an application list tag, an application call tag, a placeholder tag, an audio content description tag, and a picture content description tag.

[0374] An embodiment of the present application provides an electronic device. Since the information collection interface already includes a first collection option corresponding to program information, it can be understood that when collecting program information, the program information can be deeply processed to achieve comprehensive content extraction of the program information, and the extracted content can be displayed in the information collection interface, thereby avoiding the need for additional editing operations when the user edits the program information; and the user's next intention can be predicted, which is convenient for the user to use after finding the first collection option corresponding to the program information, thereby simplifying the processing steps of the electronic device after collecting the program information, and significantly improving the user's usage experience and the efficiency of the electronic device in file editing.

[0375] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0376] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described here.

[0377] It should be understood that in an embodiment of the present application, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0378] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0379] Processor 110 may include one or more processing units. Optionally, processor 110 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 110.

[0380] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0381] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0382] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0383] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0384] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned information collection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0385] It should be noted that, in this document, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0386] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0387] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.< / audio>

Claims

1. A method for collecting information, characterized in that: The method comprises: Receiving a collection input of program information in a program interface, wherein the program information includes at least one of the following: a file, a link; In response to the collection input, displaying an information collection interface; Among them, the information collection interface includes a first collection option corresponding to the program information, and the first collection option includes the information identifier of the program information and the collection details information of the program information; the collection details information in the information collection interface includes at least one of the following: keywords of the program information, at least one application icon, a collection source identifier, and a sharing control, and the at least one application icon is an icon of an application that matches the user's intention; the user's intention is predicted based on the keyword.

2. The method according to claim 1, characterized in that After displaying the information collection interface, the method further includes: receiving a selection input of a first favorite option corresponding to the program information; In response to the selection input, displaying a collection details interface of the first collection option; the collection details interface includes: information content of the program information, collection details information of the program information, a deep conversation control, a deep thinking control, and an import control; The collection details information in the collection details interface includes at least one of the following: summary information of the program information, keywords of the program information, at least one application icon, a collection source identifier, and a sharing control.

3. The method according to claim 2, characterized in that After displaying the favorite details interface of the first favorite option, the method further includes: receiving a first input to the deep conversation control; In response to the first input, displaying a collection dialogue interface; receiving a first dialogue message input by a user in the collection dialogue interface; the first dialogue message including first task description information of a first task, the first task description information lacking at least one task element; determining element default description information based on the collection detail information of the program information, where the element default description information is used to describe the at least one task element missing from the first task description information; The first task is executed based on the element default description information and the first task description information.

4. The method according to claim 3, characterized in that Before executing the first task based on the element default description information and the first task description information, the method further includes: receiving a second input to one of the at least one application icon; The executing the first task based on the element default description information and the first task description information includes: In response to the second input, the application corresponding to the application icon selected by the second input executes the first task based on the element default description information and the first task description information.

5. The method according to claim 1, wherein The method further comprises: receiving a selection input of the collection source identifier; In response to the selection input, the program interface including the program information is displayed.

6. The method according to claim 1, characterized in that The information collection interface includes at least one collection option; After displaying the information collection interface, the method further includes: receiving a favorite merging control input for a second favorite option and a third favorite option in the at least one favorite option; In response to the collection merging control input, a fourth collection option is displayed on the information collection interface, and the display area of the fourth collection option includes a first information identifier of the program information corresponding to the second collection option, collection detail information of the program information corresponding to the second collection option, a second information identifier of the program information corresponding to the third collection option, and collection detail information of the program information corresponding to the third collection option.

7. The method according to claim 6, characterized in that After the fourth collection option is displayed on the information collection interface, the method further includes: receiving a selection input for the fourth favorite option; In response to the selection input, the collection details interface of the fourth collection option is displayed; the collection details interface includes: the information content of the program information corresponding to the second collection option, the collection details information of the program information corresponding to the second collection option, the information content of the program information corresponding to the third collection option, and the collection details information of the program information corresponding to the third collection option.

8. The method according to claim 7, characterized in that After displaying the favorite details interface corresponding to the fourth favorite entry, the method further includes: receiving a selection input of the deep conversation control; In response to the selection input, displaying a collection dialogue interface; receiving a second dialogue message input by the user in the collection dialogue interface, wherein the second dialogue message includes second task description information of the second task, and the second task description information lacks at least one task element information; Displaying deep thinking information on the collection session interface; wherein the deep thinking information includes descriptive information of a thinking process and thinking result information obtained through deep thinking based on the collection details information of the fourth collection option and the second task description information; the thinking result information includes at least one recommended plan option; each plan option includes a keyword in the collection details information of the fourth collection option and at least one application icon; receiving a selection input of one of the at least one application icon; In response to the selection input, the application corresponding to the application icon selected by the selection input executes a second task; wherein the second task is a task determined based on the plan option and application program type corresponding to the application icon selected by the selection input.

9. The method according to claim 1, characterized in that The information collection interface further includes a fifth collection option, the fifth collection option being obtained by combining the first collection option, the second collection option, and the third collection option; the display area of the fifth collection option includes information identifiers and collection details of the program information corresponding to the first collection option and the second collection option, while the information identifier and collection details of the program information corresponding to the third collection option are hidden; The display area of the fifth favorite option further includes an expansion control; The method further comprises: receiving a selection input for the expansion control; In response to the selection input, the information identifier and collection detail information of the program information corresponding to the third collection option are displayed in the display area of the fifth collection option.

10. The method according to claim 6, characterized in that The second favorite option corresponds to the first position identifier and the first page position identifier, and the third favorite option corresponds to the second position identifier and the second page position identifier; The method further comprises: receiving a selection input of the first information identifier; In response to the selection input, determining a favorite entry corresponding to the first location identifier; Determining, based on the first page position identifier, a display position of the program information and the favorite details information of the second favorite option in the favorite details interface of the fourth favorite option; Based on the display position, the program information and the favorite details information of the second favorite option are displayed at the top of the favorite details interface of the fifth favorite option.

11. The method according to claim 1, wherein The information collection interface includes a sixth collection option; The method further comprises: receiving a selection input for the sixth favorite option; In response to the selection input, displaying a favorites editing interface for the sixth favorites option, the favorites editing interface including information content of the program information, a deep conversation control, a deep thinking control, and an import control; the favorites editing interface is used to edit interface content of a favorites details interface for the sixth favorites option; receiving a third input to the deep thinking control; In response to the third input, a favorites details interface is displayed.

12. The method according to claim 11, characterized in that After displaying the favorites editing interface of the program information, the method further includes: Receiving the remark information input by the user in the information editing area of the favorites editing interface; The remark information is updated to the collection details interface of the sixth collection option.

13. The method according to claim 1, wherein The method further comprises: Inputting the program information into a multimodal reasoning model, the multimodal reasoning model comprising a file encoder and a deep reasoning unit; extracting characteristic information of the program information through the file encoder; Performing intent prediction based on the feature information by the deep reasoning unit to obtain at least one application matching the feature information; Based on the feature information and the at least one application, favorite detail information is output.

14. The method according to claim 13, characterized in that In the case where the program information is an image, the feature information includes at least one of the following: image content description information and text recognition information; wherein the image content description information is used to describe the image content of the image, and the text recognition information is text information obtained by recognizing text in the image; In the case where the program information is a document, the characteristic information includes at least one of the following: document summary information, keywords of the document, and a title of the document; wherein the document summary information is used to summarize the document content of the document; In the case where the program information is a video, the feature information includes at least one of the following: video content description information, subtitle information of each video frame of the video, and video frame content information of each video frame of the video; wherein the video content description information is used to describe the video content of the video; In the case where the program information is audio, the feature information includes at least one of the following: text summary information of the audio and audio content description information of the audio, wherein the text summary information is used to summarize the audio content of the audio, and the audio content description information is used to describe the audio content of the audio.

15. The method according to claim 13, characterized in that The method further comprises: Inputting the training mark into the multimodal reasoning model for training and outputting the collection details information; The training mark includes at least one of the following: an application name mark, an application list mark, an application call mark, a placeholder mark, an audio content description mark, and a picture content description mark.

16. The method according to claim 4, characterized in that When the application is a navigation application, the element default description information includes a navigation destination, and the first task description information includes a navigation task element, the first task is to navigate to the navigation destination; When the application is a document editing program, the element default description information includes document content, and the first task description information includes a document editing task element, the first task is to edit the document content using the document editing program; When the application is an image editing program, the element default description information includes an image, and the first task description information includes an image editing task element, the first task is to edit an image using the image editing program.

17. An information collection device, characterized in that: include: A receiving module, configured to receive a collection input of program information in a program interface, wherein the program information includes at least one of the following: a file, a link; A display module, configured to display an information collection interface in response to the collection input received by the receiving module; The information collection interface includes a first collection option corresponding to the program information, and the first collection option includes an information identifier of the program information and collection details of the program information; The collection details information in the information collection interface includes at least one of the following: a keyword of the program information, at least one application icon, a collection source identifier, and a sharing control; the at least one application icon is an icon of an application that matches the user's intention, and the user's intention is predicted based on the keyword.

18. The device according to claim 17, characterized in that The receiving module is further configured to receive a selection input of a first collection option corresponding to the program information after the display module displays the display information collection interface; The display module is further configured to display a collection details interface of the first collection option in response to the selection input received by the receiving module; The collection details interface includes: the information content of the program information, the collection details information of the program information, the deep conversation control, the deep thinking control and the import control; The collection details information in the collection details interface includes at least one of the following: summary information of the program information, keywords of the program information, at least one application icon, a collection source identifier, and a sharing control.

19. The device according to claim 18, characterized in that The information collection device further includes: a processing module; The receiving module is further configured to receive a first input to the deep dialogue control after the display module displays the collection details interface of the first collection option; The display module is further configured to display a collection dialogue interface in response to the first input received by the receiving module; The receiving module is further configured to receive a first dialogue message input by a user on the collection dialogue interface; the first dialogue message includes first task description information of a first task, the first task description information lacking at least one task element; The processing module is configured to determine element default description information based on the collection details information of the program information, wherein the element default description information is used to describe the at least one task element missing from the first task description information; And based on the element default description information and the first task description information, execute the first task.

20. The device according to claim 19, characterized in that The receiving module is further configured to receive, before the processing module executes the first task based on the element default description information and the first task description information, a second input on one of the at least one application icon; The processing module is specifically configured to respond to the second input received by the receiving module, and execute the first task based on the element default description information and the first task description information, by the application corresponding to the application icon selected through the second input.

21. The device according to claim 17, characterized in that The receiving module is further configured to receive a selection input of the collection source identifier; The display module is further configured to display the program interface including the program information in response to the selection input received by the receiving module.

22. The device according to claim 17, characterized in that The information collection interface includes at least one collection option; The receiving module is further configured to receive a collection merging control input for a second collection option and a third collection option in the at least one collection option after the information collection interface is displayed; The display module is also used to display a fourth collection option on the information collection interface in response to the collection merging control input received by the receiving module, and the display area of the fourth collection option includes a first information identifier of the program information corresponding to the second collection option, collection details information of the program information corresponding to the second collection option, a second information identifier of the program information corresponding to the third collection option, and collection details information of the program information corresponding to the third collection option.

23. The device according to claim 22, characterized in that The receiving module is further configured to receive a selection input of the fourth collection option after the display module displays the fourth collection option on the information collection interface; The display module is further configured to display a collection details interface of the fourth collection option in response to the selection input; The collection details interface includes: information content of the program information corresponding to the second collection option, collection details information of the program information corresponding to the second collection option, information content of the program information corresponding to the third collection option, and collection details information of the program information corresponding to the third collection option.

24. The device according to claim 23, characterized in that The information collection device further includes: a processing module; The receiving module is further configured to receive a selection input of the deep conversation control after displaying the favorites details interface corresponding to the fourth favorites item; The display module is further configured to display a collection dialogue interface in response to the selection input received by the receiving module; The receiving module is further configured to receive a second dialogue message input by the user in the collection dialogue interface, the second dialogue message including second task description information of the second task, the second task description information lacking at least one task element information; The display module is configured to display deep thinking information on the collection session interface; wherein the deep thinking information includes descriptive information of a thinking process and thinking result information obtained through deep thinking based on the collection details information of the fourth collection option and the second task description information; the thinking result information includes at least one recommended plan option; each plan option includes a keyword in the collection details information of the fourth collection option and at least one application icon; The receiving module is further configured to receive a selection input of one of the at least one application icon; The processing module is used to execute a second task in response to the selection input received by the receiving module through the application corresponding to the application icon selected by the selection input; wherein the second task is a task determined based on the plan option and application program type corresponding to the application icon selected by the selection input.

25. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the information collection method according to any one of claims 1 to 16.

Images